CERN attempting to tackle big data with new data center and toolchain

CERN is doubling its data center capacity and rewriting its toolchain to meet big data demands

Techworld –

CERN - the European organisation for nuclear research - is in the middle of expanding its IT infrastructure to accommodate the growing amount of data being produced by its Large Hadron Collider (LHC) in Geneva, Switzerland.

The amount of data produced by the particle accelerator is almost unfathomable. To give an idea, its 100-megapixel cameras take 40 million pictures a second of particle collisions at four Notre Dame Cathedral-sized detectors placed at various points in the 27-kilometre circular tunnel.

Up until now the LHC has been producing 35 petabytes of data a year in its search for the elusive Higgs Boson but this is set to double when it starts up again in 2015, following a temporary pause to allow for an upgrade.

The increase in data production has forced CERN to occupy 3MW in the Wigner Research Centre for Physics in Budapest, to support its 3.5MW data centre just outside Geneva.

"This is needed in order to allow us to expand out and address the increased data coming in," said Tim Bell, infrastructure manager at CERN, speaking at the Structure:Europe cloud computing event on Wednesday.

CERN is currently operating roughly 50,000 cores out of the Budapest data centre but it is targeting 300,000 cores by 2014.

CERN is also rewriting the toolsets used to process and store the petabytes of information created off the back of its experiments.

"The toolset that we wrote 10 years ago will not scale to this size and is becoming increasingly brittle and requiring large amounts of maintenance," said Bell.

The new toolsets are being written using Puppet for orchestration management and OpenStack for virtualisation.

From CIO: 8 Free Online Courses to Grow Your Tech Skills
Join the discussion
Be the first to comment on this article. Our Commenting Policies