Interview: A Sharding Architect on Scaling Databases in 2026

About Priya Raghavan

Priya Raghavan is a principal database engineer who has spent the last 14 years architecting and scaling multi-tenant SaaS environments for high-growth infrastructure companies. She has led three major ground-up resharding initiatives at organizations serving millions of concurrent users, specializing in the transition from monolithic relational stores to distributed, horizontally scalable architectures. Her work focuses on the intersection of data consistency models and the operational realities of high-availability distributed systems.

The Interview

Q: You’ve resharded production systems three times. What’s the first decision you make?

A: The very first decision isn’t about the technology or the specific database engine; it’s about identifying the primary dimension of exhaustion. You have to be brutally honest about whether you are scaling for write throughput, storage volume, or isolation. In a multi-tenant SaaS context, these are often conflated, but they require different architectural responses. If your primary bottleneck is disk I/O on a single primary, your sharding strategy will look very different than if you are trying to solve for “noisy neighbor” performance isolation. Once that bottleneck is identified, the next critical step is defining the “unit of atomicity.” This is the smallest grain of data that must always reside on the same physical shard to avoid cross-shard joins or distributed transactions, which are the silent killers of performance in a sharded environment. In my experience, if you get this unit of atomicity wrong, no amount of clever engineering can save you from the latency tail-ends later on. You also have to decide on the degree of transparency. Do you build the sharding logic into the application layer, or do you use a middleware proxy? I almost always lean toward a layer that abstracts the physical location of data from the business logic. This allows the infrastructure team to move data between shards—whether for rebalancing or maintenance—without requiring a deployment of the application code. This separation of concerns is the bedrock of a maintainable distributed system.

Q: Hash-based vs range-based sharding — how do you choose?

A: This is the classic trade-off between even distribution and query efficiency. Hash-based sharding is your best friend when you have a high volume of point lookups and you want to ensure that data is spread as uniformly as possible across your cluster. By applying a cryptographic or non-cryptographic hash function to your shard key, you effectively randomize the placement of data, which minimizes the risk of hotspots during high-write periods. However, the cost is that range scans become incredibly expensive because the data you need is scattered across the entire cluster, necessitating a “scatter-gather” operation that hits every node — the same scatter-gather cost analyzed in PNUTS’ record-level partitioning work two decades ago. Range-based sharding, on the other hand, keeps contiguous ranges of keys together on the same shard. This is essential for time-series data or any workload where you frequently perform BETWEEN or ORDER BY queries on the shard key. The risk here is the “hot range” problem. If your shard key is a timestamp, all your new writes will hit the same shard at the end of the range, creating a massive bottleneck. You end up having to pre-split ranges or constantly rebalance.

Strategy	Distribution	Range Queries	Rebalancing Cost	Best For
Hash-based	Excellent/Uniform	Poor (Scatter-gather)	High (Requires re-hashing)	Key-value lookups, high-write volume
Range-based	Skew-prone	Excellent	Moderate	Time-series, ordered scans
Directory-based	Highly Flexible	Variable	Low	Multi-tenant SaaS with uneven tenant sizes

Q: What does a hot shard actually look like in practice?

A: In a production environment, a hot shard rarely announces itself with a clean “disk full” error. Instead, it manifests as a localized spike in latency that eventually poisons the entire system. You’ll see one node in your cluster hitting 90% CPU utilization while the others are idling at 15%. This is often accompanied by an increase in the length of the task queue and a spike in lock contention. In a multi-tenant system, this usually happens because a single “whale” tenant has grown significantly larger than the others or is running a particularly aggressive batch job. You start seeing “slow query” logs that are all hitting the same IP address. From a monitoring perspective, your aggregate metrics might still look healthy—your P50 latency might be fine—but your P99 is exploding. This is why per-shard observability is non-negotiable. If you only look at the average performance of your cluster, you are blind to the fact that 5% of your users are experiencing a complete service outage because their data happens to live on the overloaded shard.

• Hot-Shard Warning Signs:
  1. CPU utilization on a single node consistently exceeding the cluster average by >40%.
  2. Elevated P99 latency for requests hitting a specific shard range.
  3. Increased “Connection Refused” or “Timeout” errors localized to one database instance.
  4. Disk I/O wait times spiking on a single volume while others remain idle.
  5. Uneven growth in storage metadata or index size across the fleet.

Warning: Never use a monotonically increasing value, such as a sequence-based ID or a high-resolution timestamp, as your primary shard key without a prefix or a hashing step. Doing so guarantees that all new data will land on a single shard, creating an immediate write bottleneck.

Q: Walk us through a zero-downtime resharding migration.

A: A zero-downtime resharding is essentially a high-stakes heart transplant while the patient is running a marathon. The process starts with “shadow writing.” You keep your existing shard map in place, but you begin dual-writing every new insert and update to both the old shards and the new target shards. Crucially, the old shards remain the source of truth for all reads. Once dual-writing is stable, you start a background “backfill” process to migrate the historical data that existed before you started dual-writing. This backfill must be rate-limited so it doesn’t starve the production traffic of resources. After the backfill is complete, you enter the verification phase, where you run “shadow reads”—performing reads against both the old and new shards and comparing the results to ensure data integrity. Only after you have high confidence in the consistency do you flip the switch to make the new shards the source of truth.

• Resharding Runbook Checklist:
  1. Enable Dual-Writes: All incoming writes go to both the legacy and the new shard topology.
  2. Snapshot & Backfill: Move historical data using a consistent snapshot to avoid missing updates.
  3. Data Validation: Run a checksum or sample-based comparison between the old and new datasets.
  4. Read Cutover: Gradually shift read traffic (1%, 5%, 25%, 100%) to the new shards.
  5. Write Cutover: Stop writing to the legacy shards and remove the dual-write logic.
  6. Cleanup: Decommission the old hardware once the new system has survived a full peak-load cycle.

Q: How much of this traces back to the Dynamo and Cassandra papers?

A: Almost everything we do today in distributed data systems is a refinement of the concepts introduced in those foundational papers. The use of consistent hashing as pioneered in the Dynamo paper is still the gold standard for minimizing data movement during cluster expansion. Before Dynamo, adding a node often meant re-mapping every single key in the database, which was operationally catastrophic. Dynamo showed us how to use a virtual ring where nodes own “ranges” of the hash space, making scaling a local rather than a global operation. Similarly, Cassandra’s ring-based partitioning model brought the idea of “eventual consistency” and “tunable consistency” into the mainstream. It forced engineers to reckon with the CAP theorem in a practical way—deciding whether they valued availability over consistency during a partition. These papers moved us away from the idea of a single, massive, reliable machine toward a fleet of “unreliable” commodity nodes that achieve reliability through software-defined replication and partitioning. When I’m designing a sharding layer today, I’m still using the same vocabulary: gossip protocols, hint handoffs, and vector clocks. We’ve built better tooling, but the underlying physics of moving data across a network haven’t changed.

Q: What’s the biggest sharding mistake you see teams make?

A: The most common mistake is “premature sharding”—sharding before you actually have a scaling problem. Sharding introduces immense complexity: you lose cross-shard joins, you lose easy ACID transactions, and your operational overhead triples. I’ve seen teams shard a 500GB database because they read a blog post about scale, only to realize that a single large RDS instance could have handled that workload with 10% of the engineering effort. When they do eventually need to shard, the second biggest mistake is choosing a shard key with low cardinality. If you shard by “Country Code” and 80% of your users are in the US, you haven’t actually solved your scaling problem; you’ve just moved it to the “US” shard. You need a key that has enough unique values to distribute load effectively across dozens or hundreds of nodes. In these scenarios, it is critical to understand the harvest and yield trade-offs during partial outages. Sometimes it is better to serve a partial result (lower yield) from the available shards than to have the entire system wait for a single struggling shard to respond.

Q: How do you decide on shard key design upfront?

A: Shard key selection is the most consequential decision in the lifecycle of a database. You have to look at your most expensive and most frequent queries. In a multi-tenant SaaS application, the tenant_id is the most common choice, but it’s not always the right one. If your tenants vary in size by orders of magnitude—think a free-tier user versus an enterprise customer—sharding by tenant_id will lead to massive hotspots. In those cases, you might need a composite key, like tenant_id + user_id, or even a sub-sharding strategy for your largest customers. You also have to consider the “query fan-out.” If your application frequently needs to query data across all tenants (like a global analytics dashboard), a tenant_id shard key will force every query to hit every shard, which is the antithesis of scalability. You’re looking for a key that aligns with your “natural” boundaries of data access. You want the majority of your transactions to be “single-shard” transactions. If more than 10-15% of your queries require cross-shard coordination, your shard key is likely misaligned with your business logic.

Note: Always include a “version” or “epoch” field in your shard mapping metadata. This allows your application to handle transitions between different sharding configurations gracefully, preventing “split-brain” scenarios where different parts of the system think the data lives in different places.

Q: Where does directory-based sharding fit in?

A: Directory-based sharding, or “lookup-based” sharding, adds a layer of indirection between the application and the data. Instead of using a mathematical function like a hash to determine where data lives, you maintain a separate lookup table (the directory) that maps keys to specific shards. This is incredibly powerful for handling the “whale tenant” problem I mentioned earlier. If one tenant becomes too large for a standard shard, you can use the directory to move that specific tenant to its own dedicated hardware without affecting any other users. This approach was famously utilized in PNUTS’ record-level partitioning across regions, where the system needed to move individual records closer to the users who were accessing them. The downside, of course, is that the directory itself becomes a critical piece of infrastructure. It needs to be highly available and extremely low-latency, often requiring its own caching layer (like Redis) to avoid becoming the new bottleneck. For complex SaaS environments with unpredictable growth patterns, the flexibility of a directory often outweighs the overhead of the extra hop.

Q: What would you tell an engineer sharding their first database?

A: Start with observability before you write a single line of sharding logic. You cannot shard what you cannot measure. You need to know your data distribution, your query patterns, and your write throughput at a granular level. Once you have the data, choose the simplest sharding strategy that solves your immediate bottleneck. Don’t build a custom consistent hashing implementation if a simple directory-based approach or a managed service will work. Also, assume that your first shard key will be wrong. Design your system so that you can reshard in the future. This means avoiding hard-coded logic and ensuring that your application accesses the database through an abstraction layer. Finally, respect the “operational tax.” Sharding isn’t just a code change; it’s a change in how you do backups, how you perform schema migrations, and how you debug production issues. If you aren’t prepared to manage a fleet of 50 databases instead of one, you aren’t ready to shard.

Q: What’s changed about sharding since the NoSQL Summer era?

A: The biggest change is the rise of “Transparent Sharding” and NewSQL. Back in the original NoSQL Summer era, sharding was a manual, painful process that usually meant giving up SQL and joins. We’ve seen a massive evolution in the “lineage” of these systems, as detailed in our lineage piece on distributed key-value stores. Today, systems like Vitess, CockroachDB, and TiDB provide a sharding layer that feels like a single logical database to the developer. We’ve also seen a shift toward more dynamic rebalancing. Instead of a static hash ring, modern systems use a “tablet” or “region” based approach where the database automatically splits and moves small chunks of data based on load. This is very much an evolution of BigTable’s tablet-splitting approach to rebalancing, but applied to relational workloads. We are moving toward a world where sharding is an implementation detail of the storage engine rather than a burden on the application developer. This allows us to focus on the data modeling and business logic, while the infrastructure handles the physical distribution of bits.

Key Takeaways

• Identify the Bottleneck: Before sharding, determine if you are scaling for write throughput, storage, or tenant isolation.
• Shard Key Cardinality: Choose a shard key with high cardinality to ensure even data distribution and avoid “hot shards.”
• Abstraction is Critical: Use a middleware or proxy layer to decouple application logic from the physical data layout.
• Zero-Downtime is Possible: Use dual-writes and background backfills to migrate data without interrupting service.
• The “Whale” Problem: For multi-tenant SaaS, consider directory-based sharding to handle tenants of vastly different sizes.
• Observability First: Per-shard metrics are essential to identify localized hotspots that aggregate metrics will miss.
• Don’t Over-Engineer: Avoid sharding until it is strictly necessary; the operational complexity is a permanent “tax” on your team.
• Modern Context: Leverage contemporary infrastructure scaling practices in industrial software to automate the management of sharded fleets, and consult practical guidance for engineering teams building scalable web infrastructure when evaluating managed sharding services.

FAQ

Q: What is database sharding? A: Sharding is a database architecture pattern that involves splitting a large dataset across multiple independent database nodes, known as shards. Each shard acts as a standalone database holding a subset of the total data, which ensures that no single server is burdened with the storage or processing requirements of the entire dataset. This allows for horizontal scaling, where capacity is increased by adding more nodes rather than upgrading a single machine.

Q: What’s the difference between hash-based and range-based sharding? A: Hash-based sharding uses a hash function on the shard key to determine the data’s location, ensuring a uniform distribution across the cluster but making range-based queries inefficient. Range-based sharding stores contiguous ranges of keys together, which facilitates very fast range scans and ordered queries but carries the risk of creating hot shards if writes are concentrated in a specific key range.

Q: What causes a hot shard? A: A hot shard occurs when an uneven amount of traffic or data is directed to a single node in a cluster, leading to performance degradation. This is typically caused by a poorly chosen shard key that does not distribute load evenly, such as a low-cardinality key or a monotonically increasing value like a timestamp where all new writes hit the same partition.

Q: Can you reshard a database without downtime? A: Yes, resharding without downtime is achievable through a multi-phase migration process that includes dual-writing to both old and new shards. By combining this with a background backfill of historical data and a gradual cutover of read traffic, engineers can move datasets between topologies while the application remains fully operational.

Q: Is consistent hashing still relevant in 2026? A: Consistent hashing remains a foundational technique in distributed systems because it minimizes the amount of data that needs to be moved when shards are added or removed. Originally popularized by the Amazon Dynamo paper, it provides a scalable way to map keys to nodes that prevents the “global re-mapping” problem inherent in simple modulo-based sharding.