June 20, 2013, 6:00 AM —
Image credit: flickr/Lordcolus
Not worried about being spied on by the NSA because, you figure, there’s no way they can afford to spy on everyone in the United States? Surely the costs of storing all of the phone calls, emails, text messages and other communications that we generate in increasingly large quantities would be prohibitive, right? Well, according to a couple of experts in storing large volumes of data, the costs would be surprisingly affordable.
Earlier this week, Brewster Kahle, who founded the Internet Archive, worked up a spreadsheet estimating the total cost of storing all phone calls in the United States for one year. Making some basic assumptions (e.g., 300 minutes of calls per person per month, 8,000 bytes per second per phone call, $100,000 in cloud costs to store one petabyte of data for a year), Kahle came up with an estimate of 272 petabytes of data generated annually that would cost $27.2 million to store for one year. For good measure, he also figured on 4,355 square feet to store the data and just under $1.8 million more in energy costs for a year.
$27 million, in the grand scheme of things, isn’t a whole lot of money. Heck, that’s the same as the total 2013 payroll of the Houston Astros - the lowest paid and one of the crummiest teams in major league baseball (sorry, 'stros fans, I feel for you). The NSA probably has that much money in loose change in between the cushions of their office sofas.
OK, you say, but what about all the other communications data that we generate which the NSA could try to get, like emails and text messages? Surely, collecting and storing all of those data would be cost prohibitive?
Maybe not. Kim Dotcom, who founded the file-hosting sites Megaupload and Mega, tweeted out some back of the envelope calculations earlier this week on how much total data is generated in the U.S. daily, including phone calls, emails, text messages and chats.
The #NSA stores daily: 10B emails 500TB + 3B sms 500GB + 5B chatmsgs 1TB + 1.8B calls 2PB + misc 1PB at a daily cost of USD 550,000. CHEAP!
— Kim Dotcom (@KimDotcom) June 16, 2013
If you add up his numbers, you get a grand total of just under 1,300 petabytes annually, which he figures costs $200 million per year to store. As Dotcom points out, the cost of storing all of our data would be less, per year, than the cost of two F-35 (A) fighter jets (cost, $153 million each). Love him or hate him, you gotta figure he knows about storing lots of data as well as he seems to know about self promotion.
If you choose to believe that either or both of these sets of estimates are relatively accurate, the NSA should have no problem storing all that data or absorbing the cost. The NSA, which reportedly has an annual budget of $10 billion, is finishing up a new data warehouse in Bluffdale, Utah (just south of Salt Lake City) that cost $1.7 billion, has 1 million square feet of space and can reportedly store one yottabyte (that’s 1 billion petabytes) of data. In other words, as Kahle put it, when it comes to the prospect of gathering and storing all these data on Americans, “money and technology would not hold back such a project– it would be held back if someone did not have the opportunity or will.”
Read more of Phil Johnson's #Tech blog and follow the latest IT news at ITworld. Follow Phil on Twitter at @itwphiljohnson. For the latest IT news, analysis and how-tos, follow ITworld on Twitter and Facebook.