September 29, 2011, 2:08 PM —
Twitter, Facebook and other social media sites are often criticized for encouraging people to share thoughts of little consequence, though social scientists are finding these electronic missives, when assembled en masse and analyzed with big data tools, can offer a wealth of new information about how people think and act.
A pair of researchers from Cornell University are the latest to mine social networks for such academic insight. Scott Golder and Michael Macy analyzed 509 million Twitter messages emitted over a period of two years by 2.4 million users across 84 different countries. From this data, they have gleaned that people have the same daily cycle of moods, regardless of their culture or language.
A paper summarizing the work, "Diurnal and Seasonal Mood Vary with Work, Sleep, and Daylength Across Diverse Cultures," is in Thursday's issue of the journal Science.
Beyond the immediate results, the work points to a possible new path in academic research, that of mining social networks for academic insight, the researchers said.
"The paper illustrates the opportunities for doing social and behavioral science in a new way," Macy said. "The growing tendency for human beings all over the globe to interact with one another using digital devices opens opportunities for research that were unimaginable even five years ago."
"Far from seeing conversations as mundane and useless, we see value in the fact that they are real-time, time-stamped messages produced by people on the spot for sharing with their friends," Golder added.
The researchers used Twitter's API (application programming interface) to gather the messages. They set up a six-node cluster to extract the data, which arrived packaged in XML, and converted the results into flat files. They then used a 55-node Hadoop cluster, running at the Cornell University Center for Advanced Computing, to analyze the dataset.
The analysis tool they used, Linguistic Inquiry and Word Count, links specific words to various positive and negative moods. Messages that might include words such as "awesome," "fantastic" or "pretty" could indicate a positive state, whereas words like "remorse," "abandonment," "fear" or "fury" indicate a negative state of some sort.