Database EngineeringAugust 16, 20269 min read

Database Scaling Strategies: Declarative Table Partitioning vs. Horizontal Sharding

Nguyen Dai Long

Nguyen Dai Long

Backend Lead & Software Engineer

When database tables grow beyond tens of millions of rows, query latency degrades as B-tree indexes exceed physical RAM capacity. Two primary architectural solutions emerge: Table Partitioning (splitting tables within a single database) and Horizontal Sharding (splitting data across multiple database instances). Here is how to evaluate and execute both strategies.

1. PostgreSQL Declarative Table Partitioning

Partitioning divides a large monolithic table into smaller physical child tables while maintaining a single logical interface. With Partition Pruning enabled, PostgreSQL query planner scans only relevant child tables matching query WHERE clauses, reducing I/O by orders of magnitude.

sql
-- Range partitioning by month in PostgreSQL
CREATE TABLE audit_logs (
    id BIGSERIAL,
    event_type VARCHAR(50) NOT NULL,
    payload JSONB,
    created_at TIMESTAMPTZ NOT NULL,
    PRIMARY KEY (id, created_at)
) PARTITION BY RANGE (created_at);

-- Child tables for individual months
CREATE TABLE audit_logs_2026_08 PARTITION OF audit_logs
    FOR VALUES FROM ('2026-08-01') TO ('2026-09-01');

CREATE TABLE audit_logs_2026_09 PARTITION OF audit_logs
    FOR VALUES FROM ('2026-09-01') TO ('2026-10-01');

Key Implementation Takeaways:

  • Use Table Partitioning before jumping to the operational complexity of sharding.
  • Range partitioning by timestamp is ideal for logs, time-series, and ledger tables.
  • Ensure queries filter on partition keys to allow the planner to prune non-matching partitions.

2. Horizontal Sharding and Consistent Hashing

When write throughput exceeds what a single beefy database instance can sustain, horizontal sharding distributes rows across independent database servers. Using Consistent Hashing algorithms minimizes key relocation when adding new database shards to your cluster.

Key Implementation Takeaways:

  • Choose your Shard Key carefully (e.g., tenant_id or user_id) to avoid cross-shard JOINs.
  • Cross-shard distributed transactions require complex 2PC (Two-Phase Commit) protocols.
  • Exhaust vertical scaling, indexing, and partitioning before introducing sharding.

Summary & Final Thoughts

Table partitioning solves table bloat on a single node; horizontal sharding scales write throughput across nodes at the expense of operational complexity.

Engineering Feedback0 likes

Was this technical breakdown helpful for your production workflow?

Nguyen Dai Long

Written by Nguyen Dai Long

Backend Engineer & Backend Lead with 4+ years of hands-on experience building production systems, RESTful APIs, and cloud infrastructure using Python (Django), Laravel, PostgreSQL, and Google Cloud Platform.

Technical Discussion0

Ask questions, challenge architectures, or share your own production insights.

Join the Technical Community Discussion

Sign in via GitHub or Google in 5 seconds to comment, exchange architecture insights, and build your engineering presence.

Loading discussion...
NDL Ecosystem

Explore Free Web Tools & Games

View All Tools & Apps