1. Capacity Estimation and Read/Write Ratio
A fundamental attribute of URL shortening services is the extreme read/write skew. For every 1 new short link generated, there are typically 50 to 100 redirect clicks. With 100M clicks per month, the system must sustain ~40 requests per second average, peaking at 300+ req/s during marketing campaigns. Writing 1M URLs/month requires compact storage: a 7-character Base62 string yields 62^7 (over 3.5 trillion) unique combinations.
Key Implementation Takeaways:
- ✓Design for a 50:1 or 100:1 read-to-write ratio.
- ✓7-character Base62 keys offer 3.5+ trillion distinct URLs, easily future-proofing storage.
- ✓Prioritize memory caching for the top 20% most popular URLs (Pareto 80/20 rule).
2. Key Generation: Base62 vs. Key Generation Service (KGS)
Hashing URLs with MD5 or SHA-256 and truncating causes frequent hash collisions that require database round-trips to resolve. Instead, we use an auto-incrementing 64-bit integer ID converted to Base62 ([a-zA-Z0-9]). To prevent sequential guessing of private links, we combine a distributed Snowflake ID generator with a Feistel cipher permutation.
BASE62_ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
def encode_base62(num: int) -> str:
if num == 0:
return BASE62_ALPHABET[0]
chars = []
base = len(BASE62_ALPHABET)
while num > 0:
num, rem = divmod(num, base)
chars.append(BASE62_ALPHABET[rem])
chars.reverse()
return "".join(chars)
def decode_base62(s: str) -> int:
base = len(BASE62_ALPHABET)
num = 0
for char in s:
num = num * base + BASE62_ALPHABET.index(char)
return numKey Implementation Takeaways:
- ✓Base62 conversion is bijective and completely eliminates hash collisions.
- ✓Apply a reversible Feistel cipher permutation to prevent predictable sequential enumeration.
- ✓Store mapping integers in 64-bit BIGINT columns in PostgreSQL with primary key B-trees.
3. Eliminating DB Hits with Bloom Filters & Redis Caching
Malicious crawlers querying non-existent short keys will bypass Redis cache (cache miss) and hit the relational database directly. We deployed a Redis Bloom Filter initialized with all existing keys. If the Bloom Filter returns false, the key definitively does not exist—allowing us to return an instant HTTP 404 without touching PostgreSQL.
Key Implementation Takeaways:
- ✓Use Bloom Filters in Redis to eliminate database load from non-existent random URL scans.
- ✓Cache active redirects with HTTP 302 or HTTP 301 headers based on analytics tracking needs.
- ✓Buffer click analytics asynchronously into Redis Streams before batch-persisting to cold storage.
Summary & Final Thoughts
By decoupling key generation, shielding databases with Redis Bloom Filters, and deferring click analytics to asynchronous message queues, ShortLink maintains ultra-low latency under massive read spikes.
Engineering Feedback0 likes
Was this technical breakdown helpful for your production workflow?
Technical Discussion0
Ask questions, challenge architectures, or share your own production insights.
Join the Technical Community Discussion
Sign in via GitHub or Google in 5 seconds to comment, exchange architecture insights, and build your engineering presence.