4.3
stars
189 ratings

Karthik Muthuraman +1 more instructor

23,877 already enrolled

Offered By

IBM Skills Network

About this Course

190,258 recent views

Bernard Marr defines Big Data as the digital trace that we are generating in this digital era. In this course, you will learn about the characteristics of Big Data and its application in Big Data Analytics. You will gain an understanding about the features, benefits, limitations, and applications of some of the Big Data processing tools. You’ll explore how Hadoop and Hive help leverage the benefits of Big Data while overcoming some of the challenges it poses. 
Hadoop is an open-source framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. Hive, a data warehouse software, provides an SQL-like interface to efficiently query and manipulate large data sets residing in various databases and file systems that integrate with Hadoop.

Apache Spark is an open-source processing engine that provides users new ways to store and make use of big data. It is an open-source processing engine built around speed, ease of use, and analytics. In this course, you will discover how to leverage Spark to deliver reliable insights. The course provides an overview of the platform, going into the different components that make up Apache Spark. 

In this course, you will also learn about Resilient Distributed Datasets, or RDDs, that enable parallel processing across the nodes of a Spark cluster.

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Coursera Labs

Includes hands on learning projects.

Learn more about Coursera Labs

Beginner Level

7 weeks of study, 1-2 hours / week

Approx. 12 hours to complete

English

Could your company benefit from training employees on in-demand skills?

Try Coursera for Business

What you will learn

Explainthe impact of Big Data including use cases, tools, and processing methods.
ExplainApache Hadoop architecture, ecosystem, and practices, and userelatedapplications including HDFS, HBase, Spark, and MapReduce.
Apply Spark programming basics, including parallel programming basics forDataFrames, data sets, and Spark SQL.
UseSpark’s RDDsanddata sets, optimizingSparkSQLusing Catalyst and Tungsten, anduseSpark’s development and runtime environment options.

Skills you will gain

Apache Hadoop
SparkSQL
SparkML
Big Data
Apache Spark

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Coursera Labs

Includes hands on learning projects.

Learn more about Coursera Labs

Beginner Level

7 weeks of study, 1-2 hours / week

Approx. 12 hours to complete

English

Could your company benefit from training employees on in-demand skills?

Try Coursera for Business

Instructors

Instructor rating

4.42/5 (43 Ratings)

Karthik Muthuraman

Software Engineer (Machine Learning)

IBM Center for Open-Source Data and AI Technologies

25,703 Learners

2 Courses

Aije Egwaikhide

Senior Data Scientist

IBM

431,611 Learners

6 Courses

Offered by

IBM Skills Network

IBM is the global leader in business transformation through an open hybrid cloud platform and AI, serving clients in more than 170 countries around the world. Today 47 of the Fortune 50 Companies rely on the IBM Cloud to run their business, and IBM Watson enterprise AI is hard at work in more than 30,000 engagements. IBM is also one of the world’s most vital corporate research organizations, with 28 consecutive years of patent leadership. Above all, guided by principles for trust and transparency and support for a more inclusive society, IBM is committed to being a responsible technology innovator and a force for good in the world.

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

Syllabus - What you will learn from this course

Content Rating87%(1,006 ratings)

Week1

Week 1

1 hour to complete

What is Big Data?

1 hour to complete

7 videos (Total 42 min), 2 readings, 2 quizzes

Week2

Week 2

2 hours to complete

Introduction to the Hadoop Ecosystem

2 hours to complete

6 videos (Total 37 min), 1 reading, 4 quizzes

Week3

Week 3

1 hour to complete

Apache Spark

Build your skills when you turn your attention to the popular Apache Spark platform. Explore attribute and benefits of Apache Spark and distributed computing. You'll gain key insights about functional programming and Lambda functions. Explore Resilient Distributed Datasets (RDDs), Parallel Programming, resilience in Apache Spark and relate RDDs and Parallel Programming with Apache Spark. Dive into additional Apache Spark components and learn how Apache Spark scales with Big Data. Working with Big Data signals the need for working with queries, including structured queries using SQL. Learn about the functions, parts and benefits of Spark SQL and DataFrame queries, and discover how DataFrames work with SparkSQL.

1 hour to complete

5 videos (Total 24 min), 1 reading, 3 quizzes

Week4

Week 4

2 hours to complete

DataFrames and SparkSQL

Learn about Resilient Distributed Datasets (RDDs), their uses in Apache Spark, and RDD transformations and actions. You'll compare the use of datasets with Spark's latest data abstraction, DataFrames. You'll learn to identify and apply basic DataFrame operations. Explore Apache Spark SQL optimization. Learn how Spark SQL and memory optimization benefit from using Catalyst and Tungsten. Learn how to create a table view and apply data aggregation techniques. Fortify your skills guided via the hands-on lab.

2 hours to complete

5 videos (Total 26 min), 1 reading, 4 quizzes

Week5

Week 5

4 hours to complete

Development and Runtime Environment Options

Explore how Spark processes the requests that your application submits. Learn how you can track work using the Spark Application UI. Because Spark application work happens on the cluster, you need be able to identify Apache Cluster Managers, their components, benefits, and know how to connect with each cluster manager and how and when you might want to set up a local, standalone Spark instance. Next, learn about Apache Spark application submission, including use of Spark’s unified interface, ‘spark-submit’ and learn about options and dependencies. Developers now have the option of AIOps. Discover how to use Spark within AIOps and with Apache Spark application submission, including use of Spark’s unified interface, ‘spark-submit’, describe and apply options for submitting applications, identify external application dependency management techniques and list Spark Shell benefits. View and see recommended practices for Spark's static and dynamic configuration options. Round out your development knowledge with insights about Spark on Kubernetes. This module features hands-on Spark labs using IBM Cloud and Kubernetes.

4 hours to complete

6 videos (Total 33 min), 2 readings, 7 quizzes

6 videos

Apache Spark Architecture5mOverview of Apache Spark Cluster Modes6mHow to Run an Apache Spark Application6mUsing Apache Spark on IBM Cloud4mSetting Apache Spark Configuration5mRunning Spark on Kubernetes 4m

2 readings

Summary & Highlights2mSummary & Highlights2m

4 practice exercises

Practice Quiz: Spark Architecture6mGraded Quiz: Spark Architecture9mPractice Quiz: Spark Runtime Environments6mGraded Quiz: Spark Runtime Environments9m

Week6

Week 6

3 hours to complete

Monitoring & Tuning

Platforms and applications require monitoring and tuning to manage issues that inevitably happen. In this module you'll learn about connecting the Apache Spark user interface web server and using the same UI web server to manage application processes. Identify common Apache Spark application issues. Learn about debugging issues using the application UI and locating related log files. Discover and gain real-world knowledge about how Spark manages memory and processor resources via videos and in the available hands-on lab.

3 hours to complete

5 videos (Total 30 min), 4 readings, 4 quizzes

5 videos

The Apache Spark User Interface5mMonitoring Application Progress7mDebugging Apache Spark Application Issues5mUnderstanding Memory Resources5mUnderstanding Processor Resources5m

4 readings

Summary & Highlights2mInstructions for the Final Exam2mCongrats & Next Steps2m Team & Acknowledgements2m

3 practice exercises

Practice Quiz: Introduction to Monitoring & Tuning10mGraded Quiz: Introduction to Monitoring & Tuning15mFinal Exam 1h 40m

Reviews

4.3

37 reviews

5 stars
62.82%
4 stars
20.41%
3 stars
9.94%
2 stars
2.61%
1 star
4.18%

TOP REVIEWS FROM INTRODUCTION TO BIG DATA WITH SPARK AND HADOOP

by JHDec 31, 2022

Muy util curso, sin embargo fue complicado es muy teórico

by APMar 7, 2022

the lecture was clearly understandible and I feel very gratefull to have this lecture

thank you it was phenomenal😊

by NDNov 8, 2022

All the thinks I need to know about Big Data, Spark, Hadoop and Hive and explained in details

by RRMay 1, 2022

hands on lab and quizzes at the end of each session was very helpful

View all reviews

Frequently Asked Questions

When will I have access to the lectures and assignments?
Access to lectures and assignments depends on your type of enrollment. If you take a course in audit mode, you will be able to see most course materials for free. To access graded assignments and to earn a Certificate, you will need to purchase the Certificate experience, during or after your audit. If you don't see the audit option:
The course may not offer an audit option. You can try a Free Trial instead, or apply for Financial Aid.
The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
What will I get if I subscribe to this Certificate?
When you enroll in the course, you get access to all of the courses in the Certificate, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.

More questions? Visit the Learner Help Center.

Build employee skills, drive business results

Discover Coursera for Business

Introduction to Big Data with Spark and Hadoop

About this Course

What you will learn

Skills you will gain