December 30, 2007, 9:48 PM — What seems to be the problem: Originally, IT formed around transaction processing
systems that use structured data (i.e., data that can be sorted) that typically
today involve database management systems. More recently, IT has learned how
to manage semi-structured (i.e., searchable, but not able to be sorted) data,
such as e-mail, word processing documents, and presentations. Although some
organizations have had to deal with true unstructured data (which natively are
not searchable or able to be sorted) for some time, only now is that usage increasing
dramatically. Now unstructured data may simply be photos or it could be specialized
information for a specific industry, such as medical images, but now more dynamic
information, such as video is also becoming more prominent. And with the rise
of the internet to reach customers, more and more businesses are seeking to
reach potential or actual customers using internet video in one form or another.
Can you manage internet video and rich media using your general purpose storage
systems?
What do you need to know? If you can use standard storage systems for internet
video requirements that are always relatively low relative to the capabilities
of the storage system, the answer is yes, but if you have either a high average
demand for internet video or you have a peak load demand that dramatically spikes
demand at times, then general purpose NAS and SAN systems are not designed to
meet the workload. Let's see why.
General purpose storage systems have been designed for the core use of transaction
processing (although they work well with semi-structured data as well). For
example, they have disk cache that through the use of sophisticated read-write
algorithms improves performance. This works well with database records or with
static files (even unstructured ones). However, such systems were not designed
for large files that need to be progressively downloaded or streamed. The difference
is that these files have to be presented to the user in an isochronous manner;
that is you can wait for the start of a video, but afterwards the quality of
performance has to be guaranteed. That is different from a standard file that
you have to wait for asynchronously, but you get all of it when you get it.
Blackwave is an example of a company that has optimized storage for internet
video. (Blackwave recently briefed me as an industry analyst.) Blackwave does
not use caching since it is not necessary for internet video. Blackwave has
put in place patented algorithms that focus on internet video performance, not
transaction processing performance. With its grid architecture and proprietary
software, Blackwave claims both an order of magnitude performance over traditional
systems for internet video as well as green impact where the system requires
10% of the power, cooling, and space of general purpose systems. The result
is more videos to more viewers at higher bit rates and a lower overall cost
as compared to traditional systems. Blackwave also claims the ability to automatically
reprovision content (i.e., create more copies) in real time based upon the popularity
of the content.
What can you do about it? If your internet video demands are low, you probably
don't have to worry about doing anything. However, if such demands are placing
a burden upon a traditional NAS or SAN system and the situation is likely to
get worse in the near future, then you may want to consider a standalone storage
system (such as Blackwave) that was designed specifically for managing the internet
video environment. Introducing a new element into your storage environment requires
careful research and thinking, but it may not only solve your internet video
problem, but also free up your existing NAS or SAN system for better performance
doing the tasks that they were designed to do best.
