What is Big Data?

OldHippie

I've been hearing a lot about Big Data. What makes this any different than the large amounts of data we've been generating up to now?

Tags: big data
Answer this Question

Answers

2 total
SilverHawk
Vote Up (22)

What has happened is that fast multiprocessor servers and the explosion in the growth of servers has allowed us to build larger data sets in our data warehouses than our old tools and methodologies could handle, and getting results from this mountain of data has been like finding a needle in a haystack. What makes Big Data different is that if your business is using a modern database that's optimized for it, like Sybase, there are tools and techniques that make it much easier to drill down and find the specific information you need. For example, Sybase IQ 15.3's MPP (massively parallel processing) grid architecture basically allows a sharing of processing power, memory and access to data, so that a huge number of users can all perform some of their own analytics. This is a boon to productivity as we can now consider larger data sets to recognize long-term trends.

jimlynch
Vote Up (19)

Here's a good overview of big data for you, OldHippie.

Big data
http://en.wikipedia.org/wiki/Big_data

"Big data[1] are datasets that grow so large that they become awkward to work with using on-hand database management tools. Difficulties include capture, storage,[2] search, sharing, analytics,[3] and visualizing. This trend continues because of the benefits of working with larger and larger datasets allowing analysts to "spot business trends, prevent diseases, combat crime."[4] Though a moving target, current limits are on the order of terabytes, exabytes and zettabytes of data.[5] Scientists regularly encounter this problem in meteorology, genomics[6], connectomics, complex physics simulations [7], biological and environmental research [8], Internet search, finance and business informatics. Data sets also grow in size because they are increasingly being gathered by ubiquitous information-sensing mobile devices, aerial sensory technologies (remote sensing) "software logs, cameras, microphones, Radio-frequency identification readers, wireless sensor networks and so on."[9][10] Every day, 2.5 quintillion bytes of data are created and 90% of the data in the world today was created within the past two years.[11]

One current feature of big data is the difficulty working with it using relational databases and desktop statistics/visualization packages, requiring instead "massively parallel software running on tens, hundreds, or even thousands of servers."[12] The size of "big data" varies depending on the capabilities of the organization managing the set. "For some organizations, facing hundreds of gigabytes of data for the first time may trigger a need to reconsider data management options. For others, it may take tens or hundreds of terabytes before data size becomes a significant consideration."[13]"

Ask a question

Join Now or Sign In to ask a question.
Adatao is another startup promising easier data analytics for the masses. It stands out in a few ways.
Teradata has bought the assets of Revelytix and Hadapt in a bid to grow out its capabilities for the Hadoop big-data processing framework.
Software provider Actuate is offering a free way for business units to analyze enterprise data and present the results in a format that is easy to understand.
Salesforce.com is rollling out enhancements to its Salesforce1 mobile application, with new reporting and dashboard capabilities that give users a way to dig deeper and more broadly into CRM data.
Text analytics company Luminoso, a 2010 MIT Media Lab spinoff that helps its customers make sense out of unstructured data, has raised a $6.5 million Series A round of funding. The 25-person outfit plans to use the funds for new hires in sales, product management and client services as well as to expand its product line.
Microsoft will soon offer a service aimed at making machine-learning technology more widely usable.
In the NLP (natural language processing) business for a while, Attensity sees an opportunity to get new customers with Q, a visualization tool it says can help non-technical users like marketers find insights in oceans of social media data.
Oracle CEO Larry Ellison is taking the fight to IBM, Microsoft and SAP in the burgeoning in-memory database market with a new option the company says can deliver dramatic performance boosts without requiring changes to applications.
The upcoming update of Tableau Software's flagship business-intelligence application includes the ability to add a narrative to a report, allowing authors to tell a story about the data being displayed.
Here’s a great example of how making government data open can directly benefit you
Join us:
Facebook

Twitter

Pinterest

Tumblr

LinkedIn

Google+

randomness