Big data, metadata, and traffic analysis: What the NSA is really doing

The NSA doesn't have to intercept and read all your messages to know what you're doing -- and neither do many Internet businesses.

By , ITworld |  Big Data, big data, NSA


Traffic analysis

With traffic analysis, you're not looking at metadata so much as at communication patterns. Take that Gmail diagram Immersion created. What does it tell you? Well, even at a casual glance you can see there are clusters of people I communicate a lot with. Would it surprise you know that, as a technology journalist, there's a direct relationship between each cluster and a particular publication? It shouldn't. The square made up of blue dots at the lower left, for example, is ITworld.

It's not just who you talk to and who speaks to you that can provide hints about what's going on. The pattern of communications matters as well. For example, if you have one person in an organization who frequently sends messages to a large number of people, but who receives relatively few messages back, chances are they're a leader. Are several people in a group who haven't previously been e-mailing or IMing each other suddenly talking to each other? It's a good bet they've been assigned to a joint project or team.

Think about it. Say you know only that Steven tends to e-mail or IM Esther, Jodie, and Amy between 9 a.m. and 5 p.m. from a single IP address, Monday to Friday. What do you think the odds are that he works with them and that he's contacting them from his office? Pretty high, wouldn't you think?

And, of course, with the IP address, thanks to Internet geolocation services like IPlocation, anyone can work out, generally speaking, where Steven's office is. With just two data fields -- time of messages sent and IP address -- you can work out someone's work hours and where their office is.

This is a trivial example. Every day, as noted above when we discussed metadata, you're providing your ISP and favorite Websites -- and, oh yes, the NSA -- with far more data.

Back in the mid-2000s, the NSA was using Narus Semantic Traffic Analyzer, a Linux-based software program, to surveil American Internet traffic. With this deep packet inspection tool, the NSA was able to track who was sending what kind of traffic to whom at a rate of 10 gigabits of IP packets or 2.5 gigabits of Web traffic or email, per second.

That was eight years ago. Think about it.

Join us:
Facebook

Twitter

Pinterest

Tumblr

LinkedIn

Google+

Answers - Powered by ITworld

Join us:
Facebook

Twitter

Pinterest

Tumblr

LinkedIn

Google+

Ask a Question
randomness