Distributed Systemsreading

Designing Data-Intensive Applications

Martin Kleppmann

5/5

The definitive guide to building reliable, scalable, and maintainable data systems. Covers everything from database internals to distributed consensus.

Read on January 20, 2024

Favorite Quotes

“A system is only as reliable as its least reliable component.”
“The truth is defined by what the data says, not by what the code says.”
“Exactly-once semantics can be achieved through idempotent operations.”

Why I Read This

Every senior engineer I respected had this book on their desk. After working on distributed systems at Oracle OCI, I wanted a deeper theoretical understanding of the tradeoffs I was making daily — consistency models, replication strategies, and partitioning schemes.

Key Takeaways

Part 1: Foundations of Data Systems

The book starts by building mental models for comparing data systems. The key insight is that there's no "best" database — only databases that are better suited for specific access patterns and consistency requirements.

The chapter on data models was eye-opening. Understanding the difference between document databases, relational databases, and graph databases at a fundamental level (not just "NoSQL vs SQL") changed how I approach data modeling.

Part 2: Distributed Data

This is where the book shines. The chapters on replication and partitioning provided the theoretical framework for decisions I'd made intuitively at Oracle. Understanding why leader-based replication works for some workloads and multi-leader for others gave me confidence in architecture discussions.

The chapter on consistency and consensus is dense but essential. The progression from linearizability → causal consistency → eventual consistency finally clicked after reading Kleppmann's examples.

Part 3: Derived Data

The batch and stream processing chapters connected dots between systems I'd used in isolation. Understanding that batch processing (MapReduce) and stream processing (Kafka Streams) are points on a spectrum — not fundamentally different paradigms — was a major insight.

How It Changed My Work

After reading DDIA, I started asking better questions in design reviews:

  • "What happens when this leader fails mid-replication?"
  • "Are we okay with eventual consistency here, or do we need linearizability?"
  • "How does this partitioning scheme handle hot keys?"

Who Should Read This

Every backend engineer building systems that store or process data at scale. This isn't a book about a specific technology — it's about the principles that underlie all data systems. The concepts will remain relevant long after any specific database version is obsolete.

Rating: 5/5

The single most impactful technical book I've read. I re-read specific chapters before every system design interview.

Back to all books