The art of Big Data

Can Big Data be turned into art? And can art help us to better understand Big Data?

Credit: flickr/Jer Thorp

A few years ago I was working as a technology director for a media company. The company was about to move into a shiny new building, one with several large public event spaces. Not long before the move my boss came to me and said those in charge of designing the new building wanted to create an art display out of our website’s access logs. The idea was to take live log data and display it in a way that was visually interesting, but would also change over time as people accessed the site. It would be projected on a large wall that visitors would see as they came into our new building.

For whatever reasons, it never actually happened. But I was reminded of it this week when Google released a number of stunning photographs of their data centers around the world. Fascinating stuff, to turn cables, pipes, server racks and cooling systems into art. But, boy, it worked, on a number of levels.

It got me to thinking about that abandoned attempt to create art from big data and made me wonder how many folks are out there trying to do just that. It didn’t take much Googling to find out that a number of people are using big data to create art. Or rather, that are using art to try and make more sense out of big data. 

The vast amount of data now being generated poses a problem of how to process and understand it all. New technologies like Hadoop help, but some companies are turning to artists and designers to find a new ways to try and understand what all those data can tell us.

One person who seems to be leading the way is Jer Thorp, currently serving (among things) as the New York Times’ Data Artist in Residence.  Who even know that the the New York Times had a Data Artist in Residence? I sure didn’t.

Thorp takes data from, for example, the Twitter API or the New York Times' Article Search API to try and tell stories and put data into a human context. He recently gave a TEDx talk on his work, which I highly recommend.

Thorp uses a tool calling Processing to create his work. Processing is open source software and a programming language originally conceived as a software sketchbook to promote programming literacy among visual artists. Many people are using it for a wide range of projects.

Matter of fact, I think I’m going to spend some time with Processing myself and do a little visual analysis of some data that I - and those around me - generate. Who knows? Somebody people may go to the Louvre and see a piece based on the hundreds of text messages my 12 year old sends out each month.

Anybody else out there doing this kind of work? Let us know if you or someone you know does.

ITWorld DealPost: The best in tech deals and discounts.
Shop Tech Products at Amazon