The TAO of Facebook data management

Facebook reveals how its home-built distributed data store serves a billion users worldwide within milliseconds

By , IDG News Service |  

Memcache also required a fair amount of expertise from the developers who built Facebook user-facing products, Venkataramani noted. If these developers did not understand all the nuances of the Memcache, their products could have data inconsistencies, bugs and performance issues.

The TAO caching layer is run on the servers by a collection of daemons, mostly written in C and C++. They route write requests, execute read requests and maintain consistency with other caching servers. TAO cache servers are one of two types: leaders or followers. Each leader cache is assigned to a single database shard, and is responsible for maintaining the consistency of the data between itself and the shard.

The leader cache periodically sends updates to the follower caches, which are the caches that users first hit when requesting data from Facebook. Facebook works on the principle of eventual consistency, in which data written to Facebook will be made available for access, though a few seconds may lapse before the data is written to all the database and the caches. Eventual consistency has long been a behavior associated with using a distributed database.

TAO offers a number of advantages for Facebook, Venkataramani said. First, it scales easily for traffic spikes, simply by adding more follower servers. It also is easy to upgrade with because it cleanly separates the caching layer from the persistent data storage layer, allowing the company to update and scale either one without affecting the other. The API also cleanly separates the product logic from the data access. As a result, "when building products, the product engineers just use the API to store and access data," Venkataramani said.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is Joab_Jackson@idg.com

Join us:
Facebook

Twitter

Pinterest

Tumblr

LinkedIn

Google+

Answers - Powered by ITworld

ITworld Answers helps you solve problems and share expertise. Ask a question or take a crack at answering the new questions below.

Join us:
Facebook

Twitter

Pinterest

Tumblr

LinkedIn

Google+

Ask a Question
randomness