The problem is definitely real - many workloads need to access a huge amount of data quickly, and typically inter-machine communication has been an enormous bottleneck. FaRM seems to take significant steps in resolving this.
The main ideas are: use RDMA instead of TCP/IP to hugely reduce communication overhead, and reduce the amount of distributed transactions that must be carried out by using lock-free reads and local, single-machine transactions by shipping computation to the data.
This is different from previous work primarily because of new hardware; RDMA only became possible over Ethernet (which is the standard in commodity hardware cluster deployments) recently, and TCP/IP was previously fast enough to keep up with disk, but now that RAM is becoming large enough to store significant amounts of data, TCP/IP is falling behind.
One trade-off is that distributed transactions are somewhat slow; however, they manage to get around this by providing ways to avoid distributed transactions as much as possible.
While the system they have built is impressively fast, it seems that the main contribution here is really RDMA over Ethernet, with FaRM doing a very good job of tuning itself to this. While I think RDMA over Ethernet will have a significant impact in 10 years, and maybe some of the ideas from this paper will be relevant, I don't think this will be a particularly significant paper in 10 years.
No comments:
Post a Comment