Building a Skyscraper with Legos: The Anatomy of a Distributed System Архитектуры, масштабируемость
Tyler McMullen is CTO at Fastly, where he’s responsible for the system architecture and leads the company’s technology vision. As part of the founding team, Tyler built the first versions of Fastly’s Instant Purging system, API, and Real-time Analytics. Before Fastly, Tyler worked on text analysis, web apps, bash scripts, and kickflips.
Much has been written and said about distributed systems of many different sizes, scales, and complexities. Most of the time though, we either talk about algorithms and theoretical ideas or entirely concrete specific systems: Abstract or Concrete. Gossip, failure detection, and leader election are discussed as if they’re a magical salve that can be rubbed onto a project to make it distributed. But the practical realities of distributed systems are far from straightforward.
In this talk, I will take a new approach, talking through the components and design of a real system, as well as the theory behind them. The system we’ll discuss is built to perform very high volumes of health checks, done across a cluster of machines for reliability and scalability. It’s complex enough that we need to worry about things like gossip, failure detection, leader election, logical clocks, and consistency tradeoffs. We’ll walk through each of the major components, see how they are practically built, and learn about the pain and compromises that each bring.
The audience will leave the talk with a better understanding of how the basic components fit together in a real-world system. With that knowledge, they will be better equipped to approach such projects themselves with confidence.