Databases, Ambitions, and a Testing Silver Bullet? (With Joran Dirk Greef)

How far would you go to get the kind of database you want? How deep into the stack would you dive to re-architect a system for the kind of performance, reliability and scale you believe in? Today's guest has decided to go all in, as he’s tackling the database problem from the fsync up.

In this week’s Developer Voices we talk to Joran Dirk Greef, whose ambitions—combined with the lacklustre performance of his project's payment system—have led him to build a new database called TigerBeetle, that tackles some meaty problems. They’re attempting to build a database that can be durable in the face of fsync-corner cases, highly available in the face of all kinds of hidden network problems, and performant enough to outpace existing financial systems. And on top of all those goals, they’re doing it with an interesting new language you may not have heard of - Zig.

What makes him want to take on this big a challenge? What problems keep him awake at night? And what is he doing to turn all that ambition into an achievable launch strategy? Listen on and find out…

–

TigerBeetle on Twitter: https://twitter.com/TigerBeetleDB
TigerBeetle on YouTube: https://www.youtube.com/channel/UC3TlyQ3h6lC_jSWust2leGg
Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Joran’s QCon ‘23 Talk: https://www.youtube.com/channel/UC3TlyQ3h6lC_jSWust2leGg
Viewstamped Replication Revisited (paper): https://pmg.csail.mit.edu/papers/vr-revisited.pdf
Github Test Cases for Journal recovery code: https://github.com/tigerbeetle/tigerbeetle/blob/b4dd441502894cbe9d48cb90ff0bc6a12c378591/src/vsr/journal.zig#L1181-L1213MySQL transactions per second vs fsyncs per second: https://sirupsen.com/napkin/problem-10-mysql-transactions-per-second