Big Data Overview

Big Data is very popular today because of its large data storage & processing capabilities which are not handled by traditional relational Database’s.

But Big Data is more than the Biggness of Data.It has few more challenges. We call them - 

• 3 defining characteristics of Big Data
• 3 Challenges of Big Data
• 3 V's of Big Data

Volume, Velocity & Variety

More details on 3Vs of Big Data.

In today's world we have almost 80% unstructured data in the form of books, journals, documents, metadata, health records,audio, video, analog data, files, and unstructured text such as the body of an e-mail message, Web page, or word-processor document etc. And every years this data is increasing with the percentage of around 40% which doubles this data in every 2 years.

Today we have around 40 Billion web pages, which is huge. Loads of data is getting generated everyday on social networking websites e.g. Facebook,Twitter etc.

And this data is in Terabytes or Petabytes or even more.

BIG Data is Unstructured, Semi Structured or Structured data which is in Terabytes or Petabytes or even more in size.

This flood of data is coming from many sources. Consider the following:

• The New York Stock Exchange generates about one terabyte of new trade data per day.

• Facebook hosts approximately 10 billion photos, taking up one petabyte of storage.

• Ancestry.com, the genealogy site, stores around 2.5 petabytes of data.

• The Internet Archive stores around 2 petabytes of data, and is growing at a rate of 20 terabytes per month.

• The Large Hadron Collider near Geneva, Switzerland, will produce about 15 petabytes of data per year. 

We can see different Products to process BIG DATA in market are:
1.  Apache Hadoop   (Open source and very popular today)
2.  SAP - HANA
3.  IBM Netezza
etc.