Cloudyn tackles out-of-control AWS cloud storage spending

Cloudyn is adding some new features to its Amazon Web Services monitoring and optimization service and offering some potentially neat insight into AWS’ customer base.

First, the new features. Cloudyn has a biggie that could shine a spotlight on a cloud sore spot. Cloud storage is one area that is well known for being quite expensive, oftentimes more expensive than doing storage in-house. Cloudyn is promising to give customers much better insight than they have today into their AWS storage spend.

tackle.jpgSource: Schlusselbein2007 via Flickr
Cloudyn wants to help you tackle AWS storage spending.

Let’s start with the problem. “Usually people treat storage in the cloud like most devs say, as a garbage collector,” Sharon Wagner, Cloudyn’s CEO, said. “They keep all their data there without having a clue how the cloud storage is organized, the lifecycle of storage objects, what can be removed and what can be archived.”

Cloudyn decided to try to tackle this problem. When it dug into how AWS’s S3 storage service works, it found something curious with the way that AWS handles file versioning. According to Cloudyn, in customer management consoles, AWS doesn’t show the objects where old versions of files are stored. “We have identified an interesting discrepancy between what you see in your AWS management console and what you’re billed for,” Wagner said.

It’s not that AWS is overcharging users. It’s that it doesn’t count objects that represent older versions of files when displaying used objects to users. Without that visibility, users will struggle to rein in unnecessary storage spending.

The fact that customers haven’t noticed this discrepancy shows how difficult it is for users to monitor their storage spend, Wagner said.

Cloudyn now offers what it calls a shadow version detector. The feature presents to AWS customers the true storage objects they’re using in S3, allowing them to determine if some of them can be eliminated.

Wagner said that on the average AWS bill, 30 percent goes to storage, including S3, Glacier or other AWS storage products. That means AWS users have the potential to cut significant costs off their monthly bill if they can more efficiently use their AWS storage.

This isn’t the first time Cloudyn has pointed out an AWS billing gotcha.

Cloudyn is also offering more general storage optimization features. Its S3 Tracker offers users details like which are the least and most used files and when was the last time a folder was accessed. It then shows customers where there are inefficiencies, suggesting which files can be safely removed or moved to another service like Glacier.

Cloudyn is also adding new optimization features for EC2, including one that will suggest when users ought to reallocate EC2 instances between regions and availability zones. The company also added some new features for optimizing RDS.

On to the customer stats. Based on the number and size of its customers, Cloudyn estimates – and there’s no real way of knowing – that its customers are representative of AWS customers in general.

Cloudyn has 400 customers using its service with AWS. Those customers spend on average $250,000 with AWS, representing between 200 and 300 concurrently running servers, Wagner said.

Around 70 percent of Cloudyn customers – and by extension, possibly AWS customers – are in North America with 10 percent in Latin America and 20 percent split between Europe and Asia.

The most curious stat is that 20 percent of Cloudyn’s customers are public companies with the rest being private companies. That’s particularly interesting if that stat is also indicative of AWS’ customer base. Since some kinds of public companies have to comply with strict data retention and security policies, it could mean that the vast majority of public companies still can’t figure out how to use public clouds while staying in compliance.

Read more of Nancy Gohring's "To the Cloud" blog and follow the latest IT news at ITworld. Follow Nancy on Twitter at @ngohring. For the latest IT news, analysis and how-tos, follow ITworld on Twitter and Facebook.

ITWorld DealPost: The best in tech deals and discounts.
Shop Tech Products at Amazon