Skip to main content
[N|S]nosqlsummer
All papers
2008case studiespaper #23 / 29

BASE: an Acid Alternative

by Dan Pritchett (eBay)

BASE: an Acid Alternative
BASE: an Acid Alternative by Dan Pritchett In partitioned databases, trading some consistency for availability can lead to dramatic improvements in scalability.

Why this paper matters

Dan Pritchett’s 2008 paper BASE: An Acid Alternative crystallized a decade of empirical practice in web-scale systems by codifying the trade-offs between strong consistency and operational reality. At eBay, where Pritchett led architecture, the paper emerged from the operational trenches-where partitioned networks, rolling upgrades, and human-scale incident response made ACID semantics a liability rather than a virtue. The paper formalized BASE (Basically Available, Soft state, Eventually consistent) as a principled alternative to ACID, not as a compromise but as a deliberate design choice optimized for global availability and partition tolerance. Historically, it sits at the intersection of the CAP theorem’s 2000 articulation and the 2007 Amazon Dynamo paper, translating theoretical trade-offs into engineering practice for thousands of eBay services running across hundreds of data centers.

In domains where milliseconds of latency or seconds of unavailability translate directly to revenue, BASE remains the default architectural pattern. The paper’s enduring value lies in its refusal to treat consistency as a binary: it introduced a spectrum where availability, latency, and fault tolerance are tunable parameters. This framework now underpins microservices, serverless state stores, and AI agent workflows. Pritchett’s insistence that “we must design for the conditions we actually face, not the ones we wish for” has become a mantra for distributed systems engineers navigating hybrid cloud, edge computing, and multi-region deployments. The paper’s influence extends beyond databases into observability, streaming, and even LLM inference, where partial state and eventual reconciliation are not bugs but features. It also laid the groundwork for modern conflict resolution strategies, influencing how systems handle divergent states in the face of network partitions. For engineers building systems that must operate under real-world constraints, BASE offers a pragmatic path forward-one that prioritizes resilience over theoretical purity.

Key contributions

  • Introduced BASE as a formal alternative to ACID, defining its three pillars-availability, soft state, and eventual consistency-and mapping them to concrete system behaviors under network partitions.
  • Articulated the “accept temporary inconsistency to preserve availability” principle using eBay’s production telemetry, showing how user-facing features like search, recommendations, and bidding tolerate stale reads without service degradation.
  • Demonstrated how idempotency, compensating transactions, and asynchronous processing replace ACID’s isolation and atomicity in distributed workflows, with recovery strategies resilient to node failure and network jitter.
  • Proposed a design pattern for conflict resolution via application-level semantics (e.g., last-writer-wins with vector clocks or application-specific merge logic) rather than relying on database-enforced serializability.
  • Showed that operational simplicity and human-scale debugging (e.g., log-based reconciliation) scale better than distributed transactions in large organizations where teams own bounded contexts.

Impact on modern systems

Pritchett’s BASE model directly shaped the design of several modern distributed databases, where high availability and partition tolerance remain non-negotiable. Cassandra (2009), inspired by Amazon’s Dynamo, adopted BASE semantics by default: strong consistency is opt-in via QUORUM, while ONE and LOCAL_QUORUM prioritize availability and locality text. In Cassandra’s vector-clock-based conflict resolution and hinted handoff, the BASE philosophy is visible in the treatment of partitions: nodes remain writable during inter-datacenter latency spikes, accepting writes that are replayed later. Similarly, DynamoDB (2012) enforces eventual consistency by default, allowing single-digit millisecond latency for writes even under AZ failures-a design choice that enabled AWS to scale ad auctions to billions of requests per second.

ScyllaDB (2015), a C++ rewrite of Cassandra, further optimized BASE under high concurrency by reducing GC pauses and improving hint replay throughput, achieving sub-millisecond P99 latencies at 1M+ ops/sec per node. Beyond key-value stores, BASE principles permeate modern streaming platforms. Apache Kafka (2011) treats consumer offsets as soft state-consumers can lag or replay-and treats consistency as a consumer-side concern rather than a broker requirement text. This aligns with BASE’s “soft state” pillar: broker clusters remain available even when downstream consumers are slow or partitioned, and eventual consistency is achieved via consumer-group rebalancing and log compaction.

YugabyteDB (2017), a PostgreSQL-compatible distributed SQL system, offers tunable consistency via Raft-based replication, but defaults to “available” mode during network splits-accepting writes on the majority side and resolving conflicts via application-defined merge functions. This design directly echoes Pritchett’s observation that “humans can resolve conflicts faster than distributed locks can time out.” CockroachDB (2015), while built for strong consistency via Spanner-like hybrid logical clocks, still exposes BASE-like knobs: AS OF SYSTEM TIME queries allow stale reads for dashboards, and EXPEDITE mode prioritizes availability over linearizability during cluster splits. The paper’s influence is also visible in Redis (2020+) when used as a soft-state cache with eventual consistency across shards, where application logic handles staleness and conflicts via Lua scripts and CRDTs.

Even PostgreSQL’s logical replication (2018) adopts BASE semantics in async mode, where replicas lag and reconciling divergent states is delegated to the application-a pattern Pritchett documented at eBay using row-level version vectors. Modern observability systems further exemplify BASE in action. Prometheus (2016) scrapes metrics as an eventually consistent log, where temporary gaps or duplicates are acceptable during network partitions, and Grafana dashboards render stale data without blocking writes. Elasticsearch (2010) prioritizes indexing availability over immediate search consistency, using soft-state shards that reconcile via background merges-a design that underpins real-time log analytics at petabyte scale. In each case, the system chooses availability during partitions, trusting that reconciliation will eventually occur rather than blocking on distributed consensus.

The BASE model has also left its mark on financial systems where uptime is critical but absolute consistency is impractical. Stripe’s payment processing system, for instance, relies on BASE principles to maintain availability during regional outages. By accepting temporary inconsistencies in ledger states and reconciling them via idempotent operations and compensating transactions, Stripe ensures that payments are processed without blocking-even when network partitions or node failures occur. This mirrors the patterns Pritchett described at eBay, where services like bidding and search operated under similar constraints. Similarly, Shopify’s distributed order management system uses BASE to handle flash sales and high-traffic events. By allowing temporary divergence in inventory states across regions and resolving conflicts via application-level logic (e.g., prioritizing orders based on business rules), Shopify maintains availability during peak loads while minimizing the risk of overselling. Zoom’s real-time collaboration platform further illustrates BASE in practice. During the 2020 surge in remote work, Zoom’s distributed state stores accepted temporary inconsistencies in user presence and meeting metadata to preserve uptime, reconciling state via a combination of vector clocks and application-level merge logic. This enabled Zoom to scale to 300 million daily meeting participants without sacrificing availability during network partitions or datacenter failures.

AI era: how LLMs and vector databases relate to this paper

Pritchett’s BASE framework is now central to the AI stack, where latency, availability, and partial state are inherent to LLM inference, agent workflows, and semantic indexing. Vector databases like Weaviate (2021), Pinecone (2019), and Qdrant (2020) operate under BASE by design: they prioritize write availability during network splits and use eventual consistency to propagate shard states across regions. In Retrieval-Augmented Generation (RAG), embeddings are soft state-stale or partial vectors are acceptable during indexing hiccups, and conflicts in overlapping shards are resolved via application-level deduplication or CRDT merges. This mirrors eBay’s conflict resolution: when two agents write conflicting embeddings for the same entity (e.g., “Paris” as city vs. “Paris” as hotel), the application applies domain logic (geolocation + entity type) rather than waiting for a distributed lock.

LLM inference latency is directly tied to BASE trade-offs. Systems like vLLM (2023) and TensorRT-LLM prioritize availability: when a GPU node fails, the scheduler routes requests to another node accepting soft-state KV caches, allowing inference to proceed with stale or approximate context. This is BASE in action-sacrificing strict consistency in KV caches to maintain 99th-percentile latency under 100ms. RAG pipelines often use pgvector (2022) in async mode, where embeddings are written optimistically and queries accept partial results during index rebuilds-a soft state pattern Pritchett observed in eBay’s search index.

Agent state stores like LangChain’s memory backends and crewAI’s shared context use BASE principles to scale multi-agent workflows. State is soft: agents write partial plans, tool outputs, and tool calls to a distributed log without waiting for consensus. Conflicts in shared memory (e.g., two agents updating the same plan) are resolved via application-level CRDTs or last-writer-wins with vector timestamps-exactly the pattern Pritchett advocated. Semantic indexes in vector databases act as soft state stores: during a partition, new embeddings are accepted and indexed locally, then reconciled via background sync when connectivity resumes. This enables real-time semantic search at web scale, where stale or partial indexes are preferable to unavailability.

Even LLM query planning benefits from BASE. Frameworks like DSPy (2023) and LangGraph (2024) allow agents to emit partial plans under timeouts, retrying with updated context rather than blocking on distributed consistency. This reflects Pritchett’s insight: “humans don’t need locks to collaborate-they need logs and reconciliation.” The AI era has extended BASE from databases to agent orchestration, where state is soft, consistency is eventual, and availability is the primary invariant.

Vector databases serving embeddings for real-time applications exemplify this shift. Milvus (2019) and Zilliz (2021) default to eventual consistency during shard splits, prioritizing write throughput for ingestion pipelines. During peak loads, they accept approximate nearest-neighbor results to meet latency SLOs, relying on background compaction to tighten consistency windows. This design mirrors the BASE trade-offs Pritchett described: sacrificing immediate correctness for responsiveness, then using reconciliation to converge. Similarly, LlamaIndex (2023) treats document indices as soft state, where users can query partially indexed corpora during ingestion, trading precision for uptime in chatbot and search applications. The integration of BASE into AI systems goes beyond infrastructure. LLM serving platforms like Ray Serve and FastAPI-based inference servers often run in distributed mode, where model weights and KV caches are soft state. When a node restarts, the system continues serving requests from other nodes while the restarted node syncs its state-a pattern that would be unthinkable under ACID but is routine under BASE. This flexibility has enabled AI workloads to scale horizontally without sacrificing uptime, a direct legacy of Pritchett’s work.

Further reading

BASE: an Acid Alternative — architecture diagram