This course is part of the Building Cloud Computing Solutions at Scale Specialization

4.1
stars
57 ratings

Noah Gift

6,095 already enrolled

Offered By

Building Cloud Computing Solutions at Scale SpecializationDuke University

About this Course

14,484 recent views

Welcome to the third course in the Building Cloud Computing Solutions at Scale Specialization! In this course, you will learn how to apply Data Engineering to real-world projects using the Cloud computing concepts introduced in the first two courses of this series. By the end of this course, you will be able to develop Data Engineering applications and use software development best practices to create data engineering applications. These will include continuous deployment, code quality tools, logging, instrumentation and monitoring. Finally, you will use Cloud-native technologies to tackle complex data engineering solutions. 
This course is ideal for beginners as well as intermediate students interested in applying Cloud computing to data science, machine learning and data engineering. Students should have beginner level Linux and intermediate level Python skills. For your project in this course, you will build a serverless data engineering pipeline in a Cloud platform: Amazon Web Services (AWS), Azure or Google Cloud Platform (GCP).

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Course 3 of 4 in the

Building Cloud Computing Solutions at Scale Specialization

Intermediate Level

Beginner level Linux and Python skills

Approx. 17 hours to complete

English

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Course 3 of 4 in the

Building Cloud Computing Solutions at Scale Specialization

Intermediate Level

Beginner level Linux and Python skills

Approx. 17 hours to complete

English

Instructor

Instructor rating

4.5/5 (18 Ratings)

Noah Gift

Executive in Residence and Founder of Pragmatic AI Labs

Interdisciplinary Data Science (MIDS)

37,170 Learners

11 Courses

Offered by

Duke University

Syllabus - What you will learn from this course

Week1

Week 1

4 hours to complete

Getting Started with Cloud Data Engineering

This week, you will learn about the methodologies involved in Data Engineering. You will also learn to evaluate best practices for dealing with the end of Moore’s Law, develop distributed systems that apply software engineering best practices and evaluate best practices for implementing solutions with Big Data. You will apply these practices to build a GPU programming project using Numba and the CUDA SDK.

4 hours to complete

24 videos (Total 101 min), 6 readings, 1 quiz

24 videos

Instructor Introduction1mCourse Introduction1mLab Onboarding1mCourse 3 Project Overview1mIntroduction to the End of Moore's Law2mThe Problem with Concurrency in Python2mExploring the End of Moore's Law3mUsing CUDA and Numba6mWhat is an ASIC?2mTaking Advantage of Colab Pro1mIntroduction to Distributed Systems1mLogging and Instrumentation Distributed Systems1mCAP Theorem2mAmdahl's Law2mElasticity3mHighly Available Nine Nine's1mDebugging Python Code25mIntroduction to Big Data1mWhat is Big Data?2mThe Three V's of Big Data2mData Lakes2mBig Data Processing3mData Feedback Loop1mUsing EMR Spark to Run Parallel Jobs25m

6 readings

Specialization Project Roadmap: Course 3 10mCourse Structure and Discussion Etiquette10mThe End of Moore's Law10mChallenges and Opportunities in Distributed Systems10mChallenges in Big Data10mCreating high-performance code10m

1 practice exercise

Week 1 Quiz30m

Week2

Week 2

6 hours to complete

Examining Principles of Data Engineering

6 hours to complete

9 videos (Total 156 min), 1 reading, 1 quiz

9 videos

Introduction to Data Engineering1mWhat is Data Engineering?48sBatch vs. Streaming vs. Events1mBuilding CLI Tools with Click33mBuilding Containerized Command-line Tools12mAdvanced Testing with Amazon CodeGuru15mAdvanced Testing with AWS CodeBuild29mMapping Functions to CLI: Part 131mMapping Functions to CLI: Part 227m

1 reading

Extending Command-Line Data Processing Tool10m

1 practice exercise

Week 2 Quiz30m

Week3

Week 3

4 hours to complete

Building Data Engineering Pipelines

4 hours to complete

22 videos (Total 192 min), 2 readings, 1 quiz

22 videos

Introduction to Serverless Data Engineering54sWhat is Serverless?51sServerless Concepts: Service Model6mServerless Concepts: Functions4mServerless Concepts: Ecosystem7mAWS Lambda: Overview37sIntroduction to AWS Cloud97mAWS Lambda: Hello World12mAWS Lambda: Deploy Testing4mAWS Lambda: Event Handling5mAWS Lambda: Wikipedia Example14mBuild a Serverless Data Pipeline37mServerless Cookbook with AWS32mServerless Cookbook with GCP17mIntroduction to Data Governance48sWhat is Data Governance?52sThe Principle of Least Privilege1mCloud Security with IAM on AWS6mAWS Shared Security Model7mAWS IAM Service12mAWS Cloud Security Operations5mEncrypt at Rest and Transit3m

2 readings

Using Amazon EFS for AWS Lambda10mBuilding Computer Vision Label Trigger for S310m

1 practice exercise

Week 3 Quiz30m

Week4

Week 4

3 hours to complete

Applying Key Data Engineering Tasks

3 hours to complete

14 videos (Total 116 min), 3 readings, 1 quiz

14 videos

Introduction to Extract, Transform, Load (ETL)44sWhat is ETL?51sUsing Amazon Athena with AWS Glue22mReal-World Problems in ETL13mIntroduction to Cloud Databases6mOne Size Does Not Fit All in the Cloud?1mUsing Google BigQuery15mUsing AWS Aurora Serverless2mUsing AWS DynamoDB2mUsing AWS Redshift30mIntroduction to Cloud Storage46sWhy Cloud Storage?1mCloud Storage Deep Dive13mUsing Amazon S34m

3 readings

Cloud Databases10mCloud Storage Solutions10mCreate a serverless Data Engineering Pipeline10m

1 practice exercise

Week 4 Quiz30m

Reviews

4.1

16 reviews

5 stars
47.36%
4 stars
31.57%
3 stars
8.77%
2 stars
10.52%
1 star
1.75%

TOP REVIEWS FROM CLOUD DATA ENGINEERING

by RADec 30, 2021

Very important information and concepts was shared.

View all reviews

About the Building Cloud Computing Solutions at Scale Specialization

With more companies leveraging software that runs on the Cloud, there is a growing need to find and hire individuals with the skills needed to build solutions on a variety of Cloud platforms. Employers agree: Cloud talent is hard to find. This Specialization is designed to address the Cloud talent gap by providing training to anyone interested in developing the job-ready, pragmatic skills needed for careers that leverage Cloud-native technologies.

In the first course, you will learn how to build foundational Cloud computing infrastructure, including websites involving serverless technology and virtual machines, using the best practices of DevOps. The second course will teach you how to build effective Microservices using technologies like Flask and Kubernetes that are continuously deployed to a Cloud platform: Amazon Web Services (AWS), Azure or Google Cloud Platform (GCP). The third course begins to put together all of the Cloud concepts introduced in the first two courses to tackle more complex data engineering solutions. And finally, in the fourth course you will apply Machine Learning Engineering to build a Flask web application that serves out Machine Learning predictions.

Building Cloud Computing Solutions at Scale

Frequently Asked Questions

When will I have access to the lectures and assignments?
Access to lectures and assignments depends on your type of enrollment. If you take a course in audit mode, you will be able to see most course materials for free. To access graded assignments and to earn a Certificate, you will need to purchase the Certificate experience, during or after your audit. If you don't see the audit option:
The course may not offer an audit option. You can try a Free Trial instead, or apply for Financial Aid.
The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
What will I get if I subscribe to this Specialization?
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
Is financial aid available?
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.

More questions? Visit the Learner Help Center.

Cloud Data Engineering

About this Course