# Data-Intensive Apps

> Designing reliable, scalable data systems — an AI agent skill based on *Designing Data-Intensive Applications* by Martin Kleppmann.

Published 2026-02-23 · Updated 2026-08-17 · Architecture & Systems · Free and MIT-licensed
Canonical: https://skills.wondel.ai/skills/ddia-systems/
Source: https://github.com/wondelai/skills/tree/main/ddia-systems

**Install:** `npx skills add wondelai/skills/ddia-systems --global`

## What is the Data-Intensive Apps skill?

Data-Intensive Apps carries Martin Kleppmann’s Designing Data-Intensive Applications into your AI coding agent as a free, loadable skill. It supplies the internals behind storage engines, replication, partitioning and isolation levels, so when the agent proposes a database, a sharding key or a consistency model it can name the failure mode it is accepting.

## Key concepts from Designing Data-Intensive Applications your agent applies

- **Storage Engines** — Understand the trade-offs between B-trees and LSM-trees, row vs columnar storage, and when to use each.
- **Replication & Partitioning** — Choose between single-leader, multi-leader, and leaderless replication. Partition data by key range or hash for scalability.
- **Consistency & Consensus** — Navigate linearizability, causal consistency, eventual consistency, and distributed consensus algorithms like Raft and Paxos.
- **Batch & Stream Processing** — Design data pipelines using MapReduce, dataflow engines, and stream processing with exactly-once semantics.
- **Distributed System Failures** — Design for unreliable networks, clocks, and nodes. Understand the impossibility results and practical workarounds.

## When to use Data-Intensive Apps — and when not to

Reach for it when:

- Two users bought the last item in stock and the code looked correct, because the default isolation level is not serializable.
- A user updates their profile, reloads, and is served the old value from a read replica.
- You are choosing between Postgres, a document store and a key-value store, and familiarity is the only argument on the table.
- Your partition key routes most traffic to one shard, and rebalancing has become a production problem.

Reach for something else when:

- You need capacity numbers and a component sketch rather than storage-engine internals — System Design is the faster route.
- The question is how the app copes when the database is slow rather than wrong — that is Release It!.
- You are shaping domain objects and their invariants, not their persistence — Domain-Driven Design fits better.
- The bottleneck is between the browser and your first byte — see High Perf Browser.

## About Martin Kleppmann

Cambridge researcher, former LinkedIn engineer, definitive data systems author. Martin Kleppmann is a researcher at the University of Cambridge and a former software engineer at LinkedIn and Rapportive. His book has become the definitive reference for understanding the internals of modern data systems.
The book: https://openlibrary.org/books/OL26780701M/Designing_Data-Intensive_Applications_The_Big_Ideas_Behind_Reliable_Scalable_and_Maintainable_System

## Example prompts

- Recommend a storage engine and replication strategy for our write-heavy workload using ddia-systems skill _(Database design)_
- Design a data partitioning scheme that avoids hotspots using ddia-systems skill _(Scalability)_
- Evaluate consistency trade-offs for our distributed checkout system using ddia-systems skill _(System design)_
- Design a stream processing pipeline for real-time analytics using ddia-systems skill _(Data architecture)_

## Frequently asked questions

### How do I install the Data-Intensive Apps skill?

Run `npx skills add wondelai/skills/ddia-systems --global`. It takes about 30 seconds and needs no account. The Data-Intensive Apps skill then works in Claude, Claude Code, Claude Cowork, Codex, Cursor, OpenClaw and Hermes Agent — anything that reads the open agentskills.io format — and your agent loads it on its own when a task calls for it. It is free and MIT-licensed, and the source is at https://github.com/wondelai/skills.

### Which book is the Data-Intensive Apps skill based on?

It packages *Designing Data-Intensive Applications* by Martin Kleppmann — cambridge researcher, former LinkedIn engineer, definitive data systems author. The skill distils the book's method into instructions your agent follows while it works, covering storage engines, replication & partitioning and consistency & consensus. It sits in the Architecture & Systems part of the library.

### How is this different from the System Design skill?

System Design works at the level of boxes and arrows — how many servers, which cache, where the queue goes — and gets you to a defensible design quickly. Data-Intensive Apps works inside the boxes: why an LSM-tree beats a B-tree under write-heavy load, what your replication topology does to read-your-writes, which anomalies snapshot isolation still permits. Size the system with Xu, then justify what is inside it with Kleppmann.

### What does CAP actually tell me?

Much less than it is usually credited with. CAP says only that a system which must stay available to clients on both sides of a network partition cannot also be linearizable. It says nothing about latency, nothing about trade-offs while the network is healthy, and its consistency means linearizability specifically, not the C in ACID. The skill prefers the more useful framing: pick a consistency or isolation level per operation and know the anomalies it permits.

### Is eventual consistency just “the data catches up”?

That is the guarantee, and it is a weak one: it promises only that if writes stop, replicas converge, with no bound on when. What matters is which anomalies you tolerate meanwhile. Reading your own stale write, watching a value move backwards on refresh, or seeing an effect before its cause are all permitted unless you add a stronger guarantee — read-your-writes, monotonic reads or consistent prefix reads. The skill names the one each read path needs.

### Do I need a distributed system to get value from this?

No — a large share of the material is about a single node. Storage-engine trade-offs, index design, isolation levels and write skew all apply to one Postgres instance, which is where most teams’ real data bugs live. Where the skill can mislead is scope: it will happily discuss consensus, multi-leader replication and stream processing for a service with five hundred users. Ask it what you can skip, or size the system with System Design first.

## Related

- All Architecture & Systems skills: https://skills.wondel.ai/skills/architecture/
- Clean Architecture: https://skills.wondel.ai/skills/clean-architecture/
- Domain-Driven Design: https://skills.wondel.ai/skills/domain-driven-design/
- System Design: https://skills.wondel.ai/skills/system-design/
- Release It!: https://skills.wondel.ai/skills/release-it/
- High Perf Browser: https://skills.wondel.ai/skills/high-perf-browser/