(Source: A royalty free photo on unsplash)
📝 tldr;
To handle the demanding uptime and scale requirements imposed by their critical business applications, Google built a widely-distributed database called Spanner - the service can span multiple machines and multiple data centers and regions around the world.
Spanner supports externally consistent reads at global scale. It does so by:
Implementing snapshot-based reads that don’t require locks.
Using a distributed clock called TrueTime to get around clock skew issues.
Timestamps for each transaction to generated shared snapshots.
🙋♀️ Credit to our Contributing Author
Bhavana wrote this amazing article to share with our 18,000+ readers! If you’re also interested in being a byte-sized design writer, consider applying here!
Bhavana Hindupur is a Principal Software Engineer at Microsoft. She brings experience from her tenure at tech giants Google and Amazon, where she designed and implemented several large-scale solutions in the cloud.
Read more from her at thepeoplessoftwareengineer
🤨 What is external consistency?
It is the guarantee that at any given point in time, any database read will see the effects of all transactions committed by that point.
🤷♀️ Why is it tricky to implement at global scale?
By applying brute force, you could achieve external consistency by blocking writes when reading. But this would really bring down the database throughput.
The problem worsens when you have to read from multiple shards - read locks would need to be acquired from every one of these shards. Remember, these shards are scattered around the world!
🤔 So how does Google Spanner do it?
Simple! Spanner reads from a logical snapshot of the database that corresponds to the timestamp at which the client made the read request.
All without blocking any writes.
When the data requested by the read spans multiple shards, it needs to come from snapshots corresponding to each of those shards. To make the read operation externally consistent, these shard snapshots need to be synchronized in time.
(In reality, this is thousands of shards scattered across thousands of machines in hundreds of data centers in multiple continents 🤯)
😎 That’s cool. But how does Spanner achieve this snapshot-ability?
Spanner does this in two steps:
Assigning every write transaction a timestamp
Guaranteeing the order of timestamps is the same as the order of transactions as seen by the client
⏰ But what about clock skew in distributed systems?
Distributed problems require distributed solutions. Spanner gets around this problem by using TrueTime - a highly available, distributed clock.
In another article, we’ll explore how exactly TrueTime works!
👨⚖️ Official Article
A link to the official article and source! Subscribing is the best way to support the newsletter!
Keep reading with a 7-day free trial
Subscribe to Byte-Sized Design to keep reading this post and get 7 days of free access to the full post archives.