Systems Recipes - Issue #1
Hello and welcome to the first issue of Systems Recipes! This is a newsletter about Software and Systems Architecture. We aim to bring you stories that inspire us to be better at our craft.
Every two weeks, we’ll send our top 3 picks that we’re reading at the moment 🤓. We had a lot of fun curating this first issue and hope you enjoy it as much 🎉.
Centrifuge: a reliable system for delivering billions of events per day by Segment
Centrifuge is a system that Segment built to take in customer events and fan them out to hundreds of various endpoints like Google Analytics, Salesforce and more.
The post takes us through the architectural choices and trade-offs of each iteration and how they got to their eventual end result. They rolled it out to production where it happily serves billions of events daily.
Rewriting the heart of our sync engine by Dropbox
Dropbox did something which a lot of companies would seldom do and completely rewrote a core part of Dropbox, their Sync engine from scratch.
We especially loved the emphasis on testing strategies and how they validated behaviour using randomized testing with a seed that can reproduce the behaviour if it results in a failure.
Splash the cache: How caching improved our reliability - Pusher
Pusher implemented a cache after an incident to deal with spiky load. They drastically reduced the number of database queries that went to DynamoDB as an added benefit.
Pusher acknowledges data consistency and cache invalidation as hard problems and talks about how setting a very small 3 second time to live allows them to minimize any inconsistencies.