This course is part of the Data Science at Scale Specialization

4.3
stars
759 ratings

Bill Howe

59,365 already enrolled

Offered By

Data Science at Scale SpecializationUniversity of Washington

About this Course

6,887 recent views

Data analysis has replaced data acquisition as the bottleneck to evidence-based decision making --- we are drowning in it.  Extracting knowledge from large, heterogeneous, and noisy datasets requires not only powerful computing resources, but the programming abstractions to use them effectively.  The abstractions that emerged in the last decade blend ideas from parallel databases, distributed systems, and programming languages to create a new class of scalable data analytics platforms that form the foundation for data science at realistic scales.
In this course, you will learn the landscape of relevant systems, the principles on which they rely, their tradeoffs, and how to evaluate their utility against your requirements. You will learn how practical systems were derived from the frontier of research in computer science and what systems are coming on the horizon.   Cloud computing, SQL and NoSQL databases, MapReduce and the ecosystem it spawned, Spark and its contemporaries, and specialized systems for graphs and arrays will be covered.

You will also learn the history and context of data science, the skills, challenges, and methodologies the term implies, and how to structure a data science project.  At the end of this course, you will be able to:

Learning Goals: 
1. Describe common patterns, challenges, and approaches associated with data science projects, and what makes them different from projects in related fields.
2. Identify and use the programming models associated with scalable data manipulation, including relational algebra, mapreduce, and other data flow models.
3. Use database technology adapted for large-scale analytics, including the concepts driving parallel databases, parallel query processing, and in-database analytics
4. Evaluate key-value stores and NoSQL systems, describe their tradeoffs with comparable systems, the details of important examples in the space, and future trends.
5. “Think” in MapReduce to effectively write algorithms for systems including Hadoop and Spark.  You will understand their limitations, design details, their relationship to databases, and their associated ecosystem of algorithms, extensions, and languages.
write programs in Spark
6. Describe the landscape of specialized Big Data systems for graphs, arrays, and streams

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Course 1 of 4 in the

Data Science at Scale Specialization

Approx. 20 hours to complete

English

Could your company benefit from training employees on in-demand skills?

Try Coursera for Business

Skills you will gain

Relational Algebra
Python Programming
Mapreduce
SQL

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Course 1 of 4 in the

Data Science at Scale Specialization

Approx. 20 hours to complete

English

Could your company benefit from training employees on in-demand skills?

Try Coursera for Business

Instructor

Instructor rating

4.07/5 (14 Ratings)

Bill Howe

Director of Research

Scalable Data Analytics

84,767 Learners

4 Courses

Offered by

University of Washington

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

Syllabus - What you will learn from this course

Content Rating89%(3,016 ratings)

Week1

Week 1

6 hours to complete

Data Science Context and Concepts

Understand the terminology and recurring principles associated with data science, and understand the structure of data science projects and emerging methodologies to approach them. Why does this emerging field exist? How does it relate to other fields? How does this course distinguish itself? What do data science projects look like, and how should they be approached? What are some examples of data science projects?

6 hours to complete

22 videos (Total 125 min), 4 readings, 1 quiz

22 videos

Appetite Whetting: Politics5mAppetite Whetting: Extreme Weather2mAppetite Whetting: Digital Humanities8mAppetite Whetting: Bibliometrics4mAppetite Whetting: Food, Music, Public Health5mAppetite Whetting: Public Health cont'd, Earthquakes, Legal4mCharacterizing Data Science5mCharacterizing Data Science, cont'd5mDistinguishing Data Science from Related Topics4mFour Dimensions of Data Science6mTools vs. Abstractions7mDesktop Scale vs. Cloud Scale5mHackers vs. Analysts2mStructs vs. Stats5mStructs vs. Stats cont'd5mA Fourth Paradigm of Science3mData-Intensive Science Examples6mBig Data and the 3 Vs5mBig Data Definitions4mBig Data Sources6mCourse Logistics7mTwitter Assignment: Getting Started14m

4 readings

Supplementary: Three-Course Reading List10mSupplementary: Resources for Learning Python10mSupplementary: Class Virtual Machine10mSupplementary: Github Instructions10m

Week2

Week 2

5 hours to complete

Relational Databases and the Relational Algebra

Relational Databases are the workhouse of large-scale data management. Although originally motivated by problems in enterprise operations, they have proven remarkably capable for analytics as well. But most importantly, the principles underlying relational databases are universal in managing, manipulating, and analyzing data at scale. Even as the landscape of large-scale data systems has expanded dramatically in the last decade, relational models and languages have remained a unifying concept. For working with large-scale data, there is no more important programming model to learn.

5 hours to complete

24 videos (Total 122 min)

24 videos

Data Models, Terminology5mFrom Data Models to Databases4mPre-Relational Databases5mMotivating Relational Databases3mRelational Databases: Key Ideas4mAlgebraic Optimization Overview6mRelational Algebra Overview4mRelational Algebra Operators: Union, Difference, Selection6mRelational Algebra Operators: Projection, Cross Product4mRelational Algebra Operators: Cross Product cont'd, Join6mRelational Algebra Operators: Outer Join4mRelational Algebra Operators: Theta-Join4mFrom SQL to RA6mThinking in RA: Logical Query Plans4mPractical SQL: Binning Timeseries5mPractical SQL: Genomic Intervals6mUser-Defined Functions3mSupport for User-Defined Functions4mOptimization: Physical Query Plans5mOptimization: Choosing Physical Plans4mDeclarative Languages5mDeclarative Languages: More Examples4mViews: Logical Data Independence5mIndexes6m

Week3

Week 3

5 hours to complete

MapReduce and Parallel Dataflow Programming

5 hours to complete

26 videos (Total 122 min)

26 videos

What Does Scalable Mean?5mA Sketch of Algorithmic Complexity5mA Sketch of Data-Parallel Algorithms5m"Pleasingly Parallel" Algorithms4mMore General Distributed Algorithms4mMapReduce Abstraction4mMapReduce Data Model3mMap and Reduce Functions2mMapReduce Simple Example3mMapReduce Simple Example cont'd3mMapReduce Example: Word Length Histogram2mMapReduce Examples: Inverted Index, Join6mRelational Join: Map Phase4mRelational Join: Reduce Phase4mSimple Social Network Analysis: Counting Friends3mMatrix Multiply Overview5mMatrix Multiply Illustrated4mShared Nothing Computing4mMapReduce Implementation5mMapReduce Phases6mA Design Space for Large-Scale Data Systems4mParallel and Distributed Query Processing5mTeradata Example, MR Extensions5mRDBMS vs. MapReduce: Features6mRDBMS vs. Hadoop: Grep5mRDBMS vs. Hadoop: Select, Aggregate, Join3m

Week4

Week 4

3 hours to complete

NoSQL: Systems and Concepts

3 hours to complete

36 videos (Total 166 min)

36 videos

NoSQL Context and Roadmap4mNoSQL Roundup4mRelaxing Consistency Guarantees3mTwo-Phase Commit and Consensus Protocols5mEventual Consistency4mCAP Theorem4mTypes of NoSQL Systems4mACID, Major Impact Systems4mMemcached: Consistent Hashing2mConsistent Hashing, cont'd4mDynamoDB: Vector Clocks5mVector Clocks, cont'd5mCouchDB Overview4mCouchB Views3mBigTable Overview5mBigTable Implementation5mHBase, Megastore3mSpanner5mSpanner cont'd, Google Systems6mMapReduce-based Systems5mBringing Back Joins4mNoSQL Rebuttal4mAlmost SQL: Pig4mPig Architecture and Performance3mData Model3mLoad, Filter, Group5mGroup, Distinct, Foreach, Flatten5mCoGroup, Join3mJoin Algorithms3mSkew5mOther Commands3mEvaluation Walkthrough3mReview6mContext3mSpark Examples5mRDDs, Benefits6m

2 hours to complete

Graph Analytics

2 hours to complete

21 videos (Total 91 min)

21 videos

Graph Overview6mStructural Analysis4mDegree Histograms, Structure of the Web4mConnectivity and Centrality4mPageRank3mPageRank in more Detail3mTraversal Tasks: Spanning Trees and Circuits5mTraversal Tasks: Maximum Flow1mPattern Matching6mQuerying Edge Tables4mRelational Algebra and Datalog for Graphs4mQuerying Hybrid Graph/Relational Data3mGraph Query Example: NSA6mGraph Query Example: Recursion4mEvaluation of Recursive Programs3mRecursive Queries in MapReduce4mThe End-Game Problem3mRepresentation: Edge Table, Adjacency List4mRepresentation: Adjacency Matrix2mPageRank in MapReduce5mPageRank in Pregel5m

Reviews

4.3

169 reviews

5 stars
57.23%
4 stars
25.39%
3 stars
9.07%
2 stars
4.73%
1 star
3.55%

TOP REVIEWS FROM DATA MANIPULATION AT SCALE: SYSTEMS AND ALGORITHMS

by KODec 21, 2019

Engaging problemset makes sure that you will get your hands dirty with data. And that is great! Definitely worth your time.

by WLMay 27, 2016

I like the breadth of coverage of this class. Each of the exercise is a gem in that I get to learn something new also. I would highly recommend this even to experience practitioner also.

by BTFeb 24, 2016

- great and very useful overview of concepts important in big data that does not get bogged down in random details

- interesting and sufficiently challenging assignments

by WEOct 3, 2016

Definitely need some background in R or Python and the lectures are a bit old. Seem to be from around 2013 when this first came out but most of the info is still relevant.

View all reviews

About the Data Science at Scale Specialization

Learn scalable data management, evaluate big data technologies, and design effective visualizations.

This Specialization covers intermediate topics in data science. You will gain hands-on experience with scalable SQL and NoSQL data management solutions, data mining algorithms, and practical statistical and machine learning concepts. You will also learn to visualize data and communicate results, and you’ll explore legal and ethical issues that arise in working with big data. In the final Capstone Project, developed in partnership with the digital internship platform Coursolve, you’ll apply your new skills to a real-world data science project.

Frequently Asked Questions

When will I have access to the lectures and assignments?
Access to lectures and assignments depends on your type of enrollment. If you take a course in audit mode, you will be able to see most course materials for free. To access graded assignments and to earn a Certificate, you will need to purchase the Certificate experience, during or after your audit. If you don't see the audit option:
The course may not offer an audit option. You can try a Free Trial instead, or apply for Financial Aid.
The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
What will I get if I subscribe to this Specialization?
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
Is financial aid available?
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.

More questions? Visit the Learner Help Center.

Build employee skills, drive business results

Discover Coursera for Business

Data Manipulation at Scale: Systems and Algorithms

About this Course

Skills you will gain