École Polytechnique Fédérale de Lausanne
Big Data Analysis with Scala and Spark
École Polytechnique Fédérale de Lausanne

Big Data Analysis with Scala and Spark

This course is part of Functional Programming in Scala Specialization

Taught in English

Some content may not be translated

99,469 already enrolled

Included with Coursera Plus

Course

Gain insight into a topic and learn the fundamentals

4.6

(2,584 reviews)

|

92%

Intermediate level
Some related experience required
27 hours (approximately)
Flexible schedule
Learn at your own pace

Details to know

Shareable certificate

Add to your LinkedIn profile

Course

Gain insight into a topic and learn the fundamentals

4.6

(2,584 reviews)

|

92%

Intermediate level
Some related experience required
27 hours (approximately)
Flexible schedule
Learn at your own pace

See how employees at top companies are mastering in-demand skills

Placeholder

Build your subject-matter expertise

This course is part of the Functional Programming in Scala Specialization
When you enroll in this course, you'll also be enrolled in this Specialization.
  • Learn new concepts from industry experts
  • Gain a foundational understanding of a subject or tool
  • Develop job-relevant skills with hands-on projects
  • Earn a shareable career certificate
Placeholder
Placeholder

Earn a career certificate

Add this credential to your LinkedIn profile, resume, or CV

Share it on social media and in your performance review

Placeholder

There are 4 modules in this course

Get up and running with Scala on your computer. Complete an example assignment to familiarize yourself with our unique way of submitting assignments. In this week, we'll bridge the gap between data parallelism in the shared memory scenario (learned in the Parallel Programming course, prerequisite) and the distributed scenario. We'll look at important concerns that arise in distributed systems, like latency and failure. We'll go on to cover the basics of Spark, a functionally-oriented framework for big data processing in Scala. We'll end the first week by exercising what we learned about Spark by immediately getting our hands dirty analyzing a real-world data set.

What's included

7 videos7 readings3 programming assignments

This week, we'll look at a special kind of RDD called pair RDDs. With this specialized kind of RDD in hand, we'll cover essential operations on large data sets, such as reductions and joins.

What's included

4 videos2 programming assignments

This week we'll look at some of the performance implications of using operations like joins. Is it possible to get the same result without having to pay for the overhead of moving data over the network? We'll answer this question by delving into how we can partition our data to achieve better data locality, in turn optimizing some of our Spark jobs.

What's included

4 videos

With our newfound understanding of the cost of data movement in a Spark job, and some experience optimizing jobs for data locality last week, this week we'll focus on how we can more easily achieve similar optimizations. Can structured data help us? We'll look at Spark SQL and its powerful optimizer which uses structure to apply impressive optimizations. We'll move on to cover DataFrames and Datasets, which give us a way to mix RDDs with the powerful automatic optimizations behind Spark SQL.

What's included

5 videos2 programming assignments

Instructor

Instructor ratings
4.7 (147 ratings)
Prof. Heather Miller
École Polytechnique Fédérale de Lausanne
2 Courses100,771 learners

Recommended if you're interested in Algorithms

Why people choose Coursera for their career

Felipe M.
Learner since 2018
"To be able to take courses at my own pace and rhythm has been an amazing experience. I can learn whenever it fits my schedule and mood."
Jennifer J.
Learner since 2020
"I directly applied the concepts and skills I learned from my courses to an exciting new project at work."
Larry W.
Learner since 2021
"When I need courses on topics that my university doesn't offer, Coursera is one of the best places to go."
Chaitanya A.
"Learning isn't just about being better at your job: it's so much more than that. Coursera allows me to learn without limits."

Learner reviews

Showing 3 of 2584

4.6

2,584 reviews

  • 5 stars

    72.98%

  • 4 stars

    21.05%

  • 3 stars

    4.45%

  • 2 stars

    0.65%

  • 1 star

    0.85%

MP
5

Reviewed on Apr 8, 2017

SS
4

Reviewed on Oct 23, 2018

CC
5

Reviewed on Jun 7, 2017

New to Algorithms? Start here.

Placeholder

Open new doors with Coursera Plus

Unlimited access to 7,000+ world-class courses, hands-on projects, and job-ready certificate programs - all included in your subscription

Advance your career with an online degree

Earn a degree from world-class universities - 100% online

Join over 3,400 global companies that choose Coursera for Business

Upskill your employees to excel in the digital economy

Frequently asked questions