Introduction to Big Data - Megadata opportunities, storage and analysis

Over the past twenty years, the amount of data generated has only increased. We currently produce a very large mass of data annually, estimated at almost 3 trillion (3.10 ¹⁸ ) bytes of data. It is thus estimated that in 2016 90% of the world's data was created in the previous two years . According to the IDC (International Data Corporation) report, the total mass of data created and copied worldwide for 2011 was 1.8 zettabytes, or 10 ²¹ bytes, and is growing by a factor of 9 every 5 years . This increase in data affects all sectors, both scientific and economic, as well as the development of Web applications and social networks .

In this context, the term Big Data was coined. The origin of this Anglo-Saxon term, literally "big data", is controversial, and its recommended official French translation is mégadonnées, although it is sometimes referred to as données massives.

This megadata is now at the heart of the concerns of players in all fields of activity. The worldwide average annual growth rate of the Big Data technology and services market over the period 2011-2016 is estimated at over 30%. According to a 2013 IDC study, this market should reach $23.8 billion in 2016. In Europe, megadata activity is expected to account for around 8% of European GDP by 2020 (AFDEL February 2013). According to Markess International, the French market for analytics, big data and data management solutions and services reached 1.9 billion euros in 2015. Its average annual growth rate between now and 2018 is expected to exceed 12% (according to Le monde informatique, March 15, 2016).

The aim of this article is to define the term Big Data or megadata, clarify the associated economic and societal issues, and introduce the various methods and techniques involved. This article focuses on two major issues associated with megadata: on the one hand, its storage, as traditional storage techniques such as relational databases are unable to store such quantities of data, and...