Our database had reached a point where failure scenarios were becoming increasingly complex and time consuming. A single node could take up to 15 minutes to recover. It was expensive to run and operate, and it simply couldn’t scale to meet the customer demand we were facing. It became clear that we needed a new design. By leveraging a modern architecture and the latest open-source technologies, we rebuilt our database for the cloud era. Recoveries that once took 15 minutes now complete in seconds. Operational costs dropped by 50%, and query latencies improved by 200%. These gains weren’t the result of any single change, but of a holistic redesign powered by technologies like Vortex, DataFusion, Delta Lake, and Rust.

In this talk, Thor will walk you through the end-to-end journey of this evolution the failure patterns and scaling limits that forced a rethink,

the architectural principles that guided the redesign,

the trade-offs and dead ends along the way,

how modern open-source components were evaluated and integrated, and

the concrete performance and reliability improvements unlocked by the new design.

You’ll leave with a blueprint for modernizing a legacy data system: how to identify when your architecture is holding you back, and how to apply today’s open-source ecosystem to build a cloud-native database that’s fast, resilient, and ready for the future.