"Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson

What Hacker News said

Strong rec teeray · 2015-03 · on “Apple Acquires FoundationDB”

One of their engineers gave a great talk at Strangeloop ( [this video] ) that shows some of the amazing simulations they ran FoundationDB through. It's well worth a watch if you want to understand the punishment that they put this database through to make it stand out.

Strong rec chubot · 2018-05 · on “The Problem with Threads (2006) [pdf]”

This talk was about Foundation DB was brought up recently, and it's pretty amazing. I recommend watching the whole thing, but to be brief they are taming the "infinite interleavings" problem through determinism. "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson [this video] They wrote an interesting Actor DSL that compiles to C++ and is completely deterministic, and they torture this deterministic engine with generated test cases on a cluster every night. I guess you could say that the whole cluster is necessarily non-deterministic, but an individual node is deterministic, given an ordering of the messages it receives.

Strong rec rubyn00bie · 2018-11 · on “FoundationDB Document Layer”

Soooo stoked. This should make futzing with it to get started a lot easier. I've been thinking about going to the FoundationDB summit to hopefully learn more... Does anyone have any good resources for learning about it? I've tried a few times but find the documentation a bit hard to grok. Maybe I'm just slow. Also if you haven't seen it, this is one of the most amazing talks on distributed systems (because it's about testing them) I've ever seen (by Will Wilson from FoundationDB pre-Apple merger): [this video] Truly mind blowing how good FoundationDB is supposed to be for its use cases, and this Mongo layer is just the business when it comes to helping adoption.

Strong rec sgk284 · 2021-06 · on “FoundationDB: A distributed, unbundled, transactional key value store [pdf]”

Here's another excellent talk at Strangeloop on FoundationDB's simulation testing by Will Wilson in 2014: [this video]

Strong rec LAC-Tech · 2023-09 · on “How FoundationDB works and why it works”

"FDB can tolerate f failures with only f+1 replicas." Wait a minute, I know that formula... Viewstamped Replication?? I need to read the foundation DB paper. (I mainly read CRDT stuff so hopefully it's understandable). --- In general I'm really impressed foundation DB folks. The talk "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson blew my mind. TL;DR they spent the majority of their initial dev effort into making a simulation of the database, then when they were happy with that plugged in real storage, time and networks at the end. Well worth a watch for anyone interested in distributed systems & reliability. [this video]

Recommends itp · 2014-12 · on “Databases at 14.4Mhz”

I'll give you some brief thoughts from my experience working at FoundationDB, but if you really want the in-depth answer to what makes our simulation framework an enabling technology for everything we do, you should take a look at the talk my colleague Will Wilson gave at Strange Loop [0]. Everything in our core is written in Flow, our asynchronous programming environment. The architecture of our server is essentially single threaded, with (pseudo-)concurrent actors allowing a single process to satisfy multiple roles at once. The interfaces that these actors use to communicate -- with the disk, over the network, with the OS in general -- are abstracted and implemented in Flow as well. Each o

Recommends StreamBright · 2018-09 · on “CrimsonDB: A Self-Designing Key-Value Store”

>>> But for the Web site for my startup, early on it was clear that I needed a good key-value store as a Web user session state store (server). So, I designed one, coded it up, and have been using it. It seems to work fine. Well depending on what error scenarios you might run into your code will work until it runs into those scenarios. Few ideas what could go wrong you can get from this video: [this video]

Recommends drej · 2020-02 · on “Challenges with distributed systems”

There is a great talk about this testing, it goes all the way to deterministic seeds and other basics. The level of determinism is admirable. [this video]