Understanding Big Data
Some years ago, I recall going for one of those enterprise data warehouse seminars where the term big data was mentioned and I asked the speaker to explain what it was. The speaker did, but I just couldn’t grasp or fully comprehend what it was back then. Fast forward to the present, and I can say I now have a better understanding although I am still learning more every day about the wonderful world of big data and its applied uses.
Big data is a common term these days and chances are if you perform a web search you are bound to come up with a plethora of information on the subject. When it first came on the scene, some people thought it wouldn’t stay, and it was a fad. Apparently, the reverse is the case as it keeps evolving, and the need to analyse big data has even given room to data science. Although currently, some schools of thought think that it has reached its peak and is becoming outdated, I think not, as we have only just scratched the surface. A lot of people are still in the place I was many years ago about big data and have not been able to make sense of it all.
Uncovering Patterns
What is the big deal? They might ask. Well, the big deal is about data being very precious and referred to as the new oil or the new gold. Data holds all the information that organisations need to succeed in today’s competitive world. Whoever harnesses it correctly can get insight, be competitive and have an edge in the business community. The truth is, we are in an age of the individual; everybody wants to sell products and services, and companies are targeting the needs of these individuals. One of the best ways to meet your customers’ needs is to consider their data, what they do, where they go, what they buy and try to anticipate their requirements or aspirations.
Big data deals with large defined data sets that cannot be stored in a format that enables analysis the way you perform traditional data analysis. Big data technology ensures that data can now be stored, retrieved and analysed. In the process, uncovering hidden patterns, understanding market trends, making connections and linkages, prioritising customer needs and gathering other useful information, help businesses in their decision making. The more you uncover this information and understand the base level, the more you can make predictions about these patterns.
The Vs of Big Data
There is a principle that governs Big Data, which is called the 4Vs (Volume, Variety, Velocity and Veracity). Volume deals with a large amount of increasing data. Variety refers to the diverse and range of data types that make up big data e.g. text files, images, emails, social media updates, audio files, video files, etc. Velocity is about the speed at which the data is being produced and refreshed, while Veracity relates to the reliability and genuineness of the data, how it can be trusted. Depending on who you speak to, there is an argument about including other principles of Big Data characteristics such as validity, visualisation, and value. Validity like veracity refers to how clean and fit for purpose the data is. Visualisation helps you use your data to tell a story, giving it illustrations and visuals that make sense as well as meaning. Big Data must make sense and bring value in all that Volume, Variety, Voracity and Velocity. Data is the new gold, and Big Data must uncover the gem.
Data Everywhere
The data types for Big Data are mainly of the unstructured and semi-structured variety. In my article To Cloud, Now What?, I mentioned the 80/20 distribution of data globally, with 80% of data being unstructured and 20% being structured. With the 20% of structured data already analysed, we haven’t scratched the surface of 80% of unstructured and semi-structured data. Data is useless if it does not have business value and Big Data offers a new direction in providing significant business value. Big Data is an enabler for you going digital with your business, driving digital transformation, machine learning, Internet of Things (IoT) and artificial intelligence. Moreover, for business models, the possibilities and new use cases keep growing by the minute and are made possible by Big Data technology. We leave our digital footprints everywhere daily, with all our digital transactions; also, mobile devices are generating location and GPS information consistently. We have social media interactions, with pictures, videos being uploaded continuously and emails sent daily. New devices and machines have sensors that keep generating data. We are a connected society, and the more connections are made, the more data grows exponentially and the more we understand this data, the more we make predictions that are useful.
Whether by creating the best customer experience or getting data fast so you can act quickly, data-driven organisations can tell what follow-up product or service an individual might need and offer tailor made promotions to these customers, that meet their specific preferences. Web sites can be aligned based on the visiting pattern of customers, making them feel welcome on each visit. Law Enforcement Agencies can also use Big Data in crime prevention, with the intelligence and data they collect. Organisations can drive profit with the models they choose to use. With the growing amount of data, data types and data models, the possibilities of Big Data are simply endless.
