Bigtable addresses a problem that, although it has been addressed many times I many different ways, is still real: how best to store data at an enormous scale. Part of why it is not a completely solved problem is that different workloads and use cases demand different requirements.
The main idea is to use an distributed and immutable storage system in which each server is responsible for a number of data tablets, each of which is comprised of a number of SSTable files which actually store the data. These are immutable, and merges across them (as well as with the running log) give the current view of the tablet. Interestingly, metadata is stored as a specific table but otherwise does not differ from data in the system.
This was different partially because of the "Google scale" problem, requiring more scalability than most systems appearing previously. Many systems attempted to be too expressive (full relational) or not expressive enough (simple key value store), and Google decided it needed a good middle ground.
As discussed above, there is a fundamental trade off in terms of expressitivity: more expressive power means you can reason about and structure your data more fully, but is also more difficulty to scale. Google chose a middle ground approach with Bigtable. There is also a trade off with their decision to use immutable data storage. This makes writes faster (since they go a log and aren't in place) and simplifies the system architecture, but adds overheads on reads (the necessity to merge) and a background GC overhead.
It has been nearly 10 years since this paper was written, and it is still influential in the sense that Bigtable is still in use. But, it does seem tailored towards Google's needs, and I don't think it has been extremely influential in other areas.
No comments:
Post a Comment