November 12, 2012, 11:00 AM —
If you're interested in predicting what topics will be trending on Twitter, there are several options to consider:
- Call a psychic hotline
- Bribe someone at Twitter who can give you a heads up when a topic is heading in that direction
- Get a couple of MIT researchers to write a machine learning algorithm to do just that
The first two options may not really work. Most of us (I hope) wouldn’t trust the first. As for the second, even if you knew the right someone at Twitter, they still may not be able to tell you much ahead of time when a topic is about to trend.
The good news is, the third option has already been done. Devavrat Shah, a professor in the electrical engineering and computer sciences department, and Stanislav Nikolov, a graduate student, have come up with a new statistical method that can be used to predict the likelihood of events occurring over time. As a proof of concept, they applied their new method to predicting which topics will become trending on Twitter.
The results? Using real tweets, they were able to correctly predict trending topics with 95% percent accuracy. 79% of the time they predicted trending topics before they became trending on Twitter, with an average lead time of almost 90 minutes.
While these results are impressive, what’s more interesting is their methodology, which is a non-parametric approach to statistical modeling. "Awesome," you say, followed by, "What does that mean?"