Data-Intensive Apps
Designing reliable, scalable data systems
Design reliable, scalable, and maintainable data systems using Martin Kleppmann's comprehensive guide. This skill equips your AI agent with deep knowledge of storage engines, replication, partitioning, consistency models, and stream processing for building data-intensive applications.
Designing Data-Intensive Applications
npx skills add wondelai/skills/ddia-systems --global This skill is compatible with Claude, Claude Code, Claude Cowork, Codex, Cursor, OpenClaw, Hermes Agent, and other agentskills.io-compatible agents.
What is the Data-Intensive Apps skill?
Data-Intensive Apps carries Martin Kleppmann’s Designing Data-Intensive Applications into your AI coding agent as a free, loadable skill. It supplies the internals behind storage engines, replication, partitioning and isolation levels, so when the agent proposes a database, a sharding key or a consistency model it can name the failure mode it is accepting.
Key concepts from Designing Data-Intensive Applications your agent applies
Storage Engines
Understand the trade-offs between B-trees and LSM-trees, row vs columnar storage, and when to use each.
Replication & Partitioning
Choose between single-leader, multi-leader, and leaderless replication. Partition data by key range or hash for scalability.
Consistency & Consensus
Navigate linearizability, causal consistency, eventual consistency, and distributed consensus algorithms like Raft and Paxos.
Batch & Stream Processing
Design data pipelines using MapReduce, dataflow engines, and stream processing with exactly-once semantics.
Distributed System Failures
Design for unreliable networks, clocks, and nodes. Understand the impossibility results and practical workarounds.
When to use Data-Intensive Apps — and when not to
Reach for it when
- Two users bought the last item in stock and the code looked correct, because the default isolation level is not serializable.
- A user updates their profile, reloads, and is served the old value from a read replica.
- You are choosing between Postgres, a document store and a key-value store, and familiarity is the only argument on the table.
- Your partition key routes most traffic to one shard, and rebalancing has become a production problem.
Reach for something else when
- You need capacity numbers and a component sketch rather than storage-engine internals — System Design is the faster route.
- The question is how the app copes when the database is slow rather than wrong — that is Release It!.
- You are shaping domain objects and their invariants, not their persistence — Domain-Driven Design fits better.
- The bottleneck is between the browser and your first byte — see High Perf Browser.
Example prompts for the Data-Intensive Apps skill
Recommend a storage engine and replication strategy for our write-heavy workload using ddia-systems skill
Database designDesign a data partitioning scheme that avoids hotspots using ddia-systems skill
ScalabilityEvaluate consistency trade-offs for our distributed checkout system using ddia-systems skill
System designDesign a stream processing pipeline for real-time analytics using ddia-systems skill
Data architectureData-Intensive Apps vs the other Architecture & Systems skills
All 6 Architecture & Systems skills are free and install the same way — the only question is which framework fits the job in front of you.
| Skill | Based on | Best for |
|---|---|---|
| Clean Architecture | Clean Architecture — Robert C. Martin | Building maintainable, testable software architectures |
| Domain-Driven Design | Domain-Driven Design — Eric Evans | Modeling complex business domains in software |
| Data-Intensive Apps | Designing Data-Intensive Applications — Martin Kleppmann | Designing reliable, scalable data systems |
| System Design | System Design Interview — Alex Xu | Scalable system design patterns and trade-offs |
| Release It! | Release It! — Michael T. Nygard | Production-ready software patterns for stability and resilience |
| High Perf Browser | High Performance Browser Networking — Ilya Grigorik | Browser networking performance optimization |
Frequently asked questions
How do I install the Data-Intensive Apps skill?
Run npx skills add wondelai/skills/ddia-systems --global. It takes about 30 seconds and needs no account. The Data-Intensive Apps skill then works in Claude, Claude Code, Claude Cowork, Codex, Cursor, OpenClaw and Hermes Agent — anything that reads the open agentskills.io format — and your agent loads it on its own when a task calls for it. It is free and MIT-licensed, and the source is at https://github.com/wondelai/skills.
Which book is the Data-Intensive Apps skill based on?
It packages Designing Data-Intensive Applications by Martin Kleppmann — cambridge researcher, former LinkedIn engineer, definitive data systems author. The skill distils the book's method into instructions your agent follows while it works, covering storage engines, replication & partitioning and consistency & consensus. It sits in the Architecture & Systems part of the library.
How is this different from the System Design skill?
System Design works at the level of boxes and arrows — how many servers, which cache, where the queue goes — and gets you to a defensible design quickly. Data-Intensive Apps works inside the boxes: why an LSM-tree beats a B-tree under write-heavy load, what your replication topology does to read-your-writes, which anomalies snapshot isolation still permits. Size the system with Xu, then justify what is inside it with Kleppmann.
What does CAP actually tell me?
Much less than it is usually credited with. CAP says only that a system which must stay available to clients on both sides of a network partition cannot also be linearizable. It says nothing about latency, nothing about trade-offs while the network is healthy, and its consistency means linearizability specifically, not the C in ACID. The skill prefers the more useful framing: pick a consistency or isolation level per operation and know the anomalies it permits.
Is eventual consistency just “the data catches up”?
That is the guarantee, and it is a weak one: it promises only that if writes stop, replicas converge, with no bound on when. What matters is which anomalies you tolerate meanwhile. Reading your own stale write, watching a value move backwards on refresh, or seeing an effect before its cause are all permitted unless you add a stronger guarantee — read-your-writes, monotonic reads or consistent prefix reads. The skill names the one each read path needs.
Do I need a distributed system to get value from this?
No — a large share of the material is about a single node. Storage-engine trade-offs, index design, isolation levels and write skew all apply to one Postgres instance, which is where most teams’ real data bugs live. Where the skill can mislead is scope: it will happily discuss consensus, multi-leader replication and stream processing for a service with five hundred users. Ask it what you can skip, or size the system with System Design first.
Install Data-Intensive Apps
Free, open-source, and ready in 30 seconds.
npx skills add wondelai/skills/ddia-systems --global MIT Licensed · Works with Claude, Claude Code, Claude Cowork, Codex, Cursor, OpenClaw, Hermes Agent & other agentskills.io agents · No account needed
Don’t guess your AI engineering level.
Measure it.
AI Developer Scorecard
How production-grade is your AI engineering?
Twenty-five questions across the practices that separate vibe-coding from production-grade engineering. Instant score, per-section breakdown, and a 30/60/90-day playbook.
Score 0–75 · 30/60/90-day playbook Score yourself For CTOs & foundersCTO Scorecard
Is your engineering team ready for AI at scale?
Twenty-five questions on how your org adopts AI — adoption, governance, automation, and ROI. See where you sit on the path from Reactive to Strategic Leader.
Score 0–75 · Reactive → Strategic Leader Score your orgWe build the skills you already use. Now we’ll build yours.
Sprints from $3K · shipped to production, or you don’t pay the final milestone.