Facebook boosts Hadoop with scheduling muscle

Facebook's Corona will make better use of clusters than MapReduce does, the company claims

By , IDG News Service |  Big Data, Facebook, Hadoop

Facebook is now in the process of moving MapReduce workloads onto clusters equipped with Corona. Initially, the social networking company deployed the software on 500 nodes. Once Corona proved effective, it was then tasked with all non-mission critical workloads, including larger workloads involving 1,000 or more servers. Now, the company is deploying Corona for all Hadoop workloads.

In tests, Corona has shown itself to be more effective than MapReduce across a number of metrics, Facebook asserted. In performance tests, Corona took around 55 seconds to fill an empty workspace, whereas MapReduce took 66 seconds -- which constitutes a 17% improvement. Job are started more quickly now, as well, within 25 seconds, down from 50 seconds with MapReduce.

Corona is not the only alternative to MapReduce. Facebook also looked at Yarn, which is Apache's overhaul of MapReduce, planned for release as MapReduce 2.0. Facebook engineers were unsure Yarn could execute jobs as large as those of the social networking site, however.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is Joab_Jackson@idg.com

Join us:
Facebook

Twitter

Pinterest

Tumblr

LinkedIn

Google+

Spotlight on ...
Online Training

    Upgrade your skills and earn higher pay

    Readers to share their best tips for maximizing training dollars and getting the most out self-directed learning. Here’s what they said.

     

    Learn more

Big DataWhite Papers & Webcasts

See more White Papers | Webcasts

Answers - Powered by ITworld

ITworld Answers helps you solve problems and share expertise. Ask a question or take a crack at answering the new questions below.

Join us:
Facebook

Twitter

Pinterest

Tumblr

LinkedIn

Google+

Ask a Question
randomness