Physicists link OpenStack, Nimbus clouds around the world to share resources

Particle physicists around the world are linking up their cloud services in order to share resources with each other. They’ve already connected nine, using a custom scheduler system to connect clouds built using OpenStack, Nimbus and even public clouds.

On stage at the OpenStack Summit this week in Portland, Oregon, Randall Sobie, a research scientist at the University of Victoria’s Institute of Particle Physics, asked for help from the community on developing a couple of OpenStack technologies that would make life easier for scientists working on this project, like common authentication, a centralized VM image store and consistent meta data.

higgs_1.jpgSource: willc2, via Flickr

The incentive for building those technologies? “The goal if you help is to understand the universe,” Sobie said. Particle physicists are responsible for the recent landmark discovery of the Higgs boson. The next step, he said, is the search for the source of dark matter, which is essentially matter that scientists detect but can’t identify.

His vision of creating an even more extensive grid of clouds would let scientists around the world make more efficient use of resources that were initially built for particular experiments. “Since this is a global collaboration, everyone has resources. We’d like to use them in a seamless way,” he said. “We’re trying to use dedicated and non-dedicated high-energy physics resources, remove the app dependence from the site and also support multiple projects.”

The kind of work that resulted in the discovery of the Higgs boson requires unique computing services. The ATLAS Detector, the instrument that observed the Higgs boson, recorded data for two years and has now stopped. It produced 40 million events per second and selected 100,000 of those that it sent to a large computer that selected 200 events per second to store. The result is that 400 MB per second of data was recorded, a total of 140 PB of data.

But now what to do with the computing power that was set up to crunch the data and is no longer needed since the data collection period has ended?

As an example, Sobie described what was done with the SLAC accelerator at Stanford. It stopped taking data in 2008. “The problem we face is once an experiment ends, the funding stops but we still want to analyze the data,” he said. However, in many cases the version of the software used in the computing infrastructure isn’t maintained by its creator. So the scientists have to build a long term data access system. They built a static cloud for SLAC data that has been in operation since 2010 and will let researchers access the data into the future.

But the 50,000 cores that were used to collect the data are now totally unused. So the scientists are starting to convert them into OpenStack clouds that will be used by researchers for other purposes, he said. This is the kind of cloud that could be linked into the grid of clouds that they’ve already begun to build.

To enable access to the grid, the scientists built a custom service they call cloud scheduler. A researcher starts out by submitting a job to the scheduler. The scheduler finds the job and looks for available VMs on any of the connected clouds. Once it finds the VM, it dispatches the job.

For now, the system can dispatch jobs to clouds running OpenStack or Nimbus, an open source implementation that was designed for the scientific community. Clouds already linked up include FutureGrid clouds at universities in Chicago, San Diego and Florida as well as OpenStack clouds NeCTAR in Melbourne, CERN, and Canada’s CANARIE East and West. The scientists are just starting to work with Oxford to link its cloud to the grid.

Public clouds from Google and Amazon can be included “in principle,” Sobie said. The researchers do use those clouds occasionally, but he sounded like they do so reluctantly and only for cases where they have exceptional demand. The challenge is that the researchers have to implement a system for ID management and they have to use public networks to shuttle data to the public clouds, which tend to be slow. The costs are higher too, he said.

So far the grid spans nine clouds over three continents and “seems to work reasonably well,” he said. “We’re hoping to scale it up further.”

Other hot topics at the OpenStack Summit have been worries that a lack of maturity of the technology is slowing down enterprise adoption, more talk of businesses leaving Amazon Web Services and tales of spies using OpenStack clouds.

Read more of Nancy Gohring's "To the Cloud" blog and follow the latest IT news at ITworld. Follow Nancy on Twitter at @ngohring. For the latest IT news, analysis and how-tos, follow ITworld on Twitter and Facebook.

ITWorld DealPost: The best in tech deals and discounts.