Companies are overwhelmed with data The data collected from

Companies are overwhelmed with data. The data collected from twitter feeds, facebook, emails, etc. in addition to the billions of bytes of data generated from internal computer applications has become nearly impossible to analyze in away that makes sense. A report from the consulting group, Forester Research, reports the majority of large companies admit to analyzing less than a 1/3 of all the data collected by their organization. The reported also stated that senior managers routinely categorized the data provided to them is being inadequate. As a result, many executives base major decisions on \"past experience\" or \"gut feeling\". The collecting and processing massive amounts of data has become known as \"Big Data\". Technology companies such as IBM are developing new database technologies to help companies address \"Big Data\". Watch the videos below as IBM\'s Chief Data Scientist Jeff Jonas (the 4th Jonas Brother) discusses \"Big Data\": Big Data Interview - Part 1 Big Data Interview - Part 2 Big Data Interview - Part 3 Discussion: Is being able to analyzing 1000 times more data the solution to the over- whelming amount of data the solution? Or is there just too much data in the world to make sense of it all?

Solution

Big Data is defined as storing large amount of data that cannot be managed by the relational database schemas. This term was introduced in late 70\'s and new methodologies have been introduced to manage the big data. Let us consider relational databases can accomodate with x amount of data . But in the current world, every day more than x data has been generated hence there will more number of databases to be needed for a single day hence big data has been introduced which is 1000 times better in accomodating the data.

Analyzing the over-whelming amount of data is not possible by the relational databases but in present world large amounts of data is generated each and every day which will be overwhleming data for relational databases. Unstructured data is also generated in large amount which can not be handled by relational databases. All the tweets, feeds etc.. comes under unstructured data. So to manage these type of unstructured data methodologies of big data has been used. New methodologies of big data are able to handle the data by using Map-Reduce technique.