“Big data” is a term widely used to describe our data-rich world, in which virtually every activity generates a digital data footprint that can be collected and analyzed. While data and data analysis are not necessarily new, the effective use of the extremely large - and rapidly-growing - datasets of today require new approaches to data management.

In order to harness big data for important applications like machine learning and artificial intelligence, you need more than an Excel spreadsheet or a traditional relational database and SQL. Instead, an entire data infrastructure is necessary to collect and process this data at scale, including data pipelines, data lakes, and data warehouses.

To make this possible, data engineers rely on new approaches to data processing such as MapReduce, developed by Google, the open-source Apache Hadoop ecosystem including Apache Spark and Apache Hive, and, increasingly, cloud computing and cloud database platforms like Cloudera.‎

With companies in practically every industry eager to discover ways to harness the power of big data in their operations, having a background in this area can open doors to a wide range of careers. Operations managers at manufacturing or logistics companies may harness data to improve their demand forecasting, inventory planning, and process efficiency; digital marketers use marketing analytics to better understand their customers and the effectiveness of their messaging; and “quants” at hedge funds rely on data-based financial engineering approaches to move millions of dollars in milliseconds.

Understanding how big data applications are built and what they are capable of can thus be incredibly valuable even if you aren’t a data engineer or data scientist yourself. However, if you have the expertise and desire to work directly with big data yourself, data engineers are responsible for building the data infrastructure capable of reliably and efficiently delivering big data at scale, and data scientists are responsible for using a wide range of analytic and programming approaches to uncover insights from it.

These two roles are in extremely high demand, and command salaries to match. According to Glassdoor, data engineers earn an average annual salary of $102,864, and data scientists earn an average annual salary of $113,309.‎

Yes! In fact, Coursera is one of the best places to learn about big data. You can take individual courses and Specializations spanning multiple courses on big data, data science, and related topics from top-ranked universities from all over the world, from the University California San Diego to Universitat Autònoma de Barcelona. Coursera also offers the opportunity to learn from industry leaders in the field like Google, Cloudera, and IBM, including options to get professional certificates.‎