Build & Transform Data Pipelines

This course is part of Modern Data Architecture & Lakehouse Engineering Specialization

Instructor: Hurix Digital

Access provided by CelcomDigi Berhad

2 modules

Gain insight into a topic and learn the fundamentals.

Beginner level

Recommended experience

2 hours to complete

Flexible schedule

Learn at your own pace

2 modules

Gain insight into a topic and learn the fundamentals.

Beginner level

Recommended experience

2 hours to complete

Flexible schedule

Learn at your own pace

What you'll learn

Modular pipeline design enables maintainable, scalable data systems that can adapt to changing business requirements.
Integration of complementary tools (Spark, dbt, Airflow) creates more robust and efficient data processing workflows than single-tool approaches.
Proper separation of concerns between ingestion, transformation, and loading stages reduces complexity and improves debugging capabilities.
Automation and orchestration are essential for reliable, production-grade data systems that minimize manual intervention and human error.

Details to know

Shareable certificate

Add to your LinkedIn profile

Assessments

4 assignments¹

AI Graded see disclaimer

Taught in English

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

logos of Petrobras, TATA, Danone, Capgemini, P&G and L'Oreal

Build your subject-matter expertise

This course is part of the Modern Data Architecture & Lakehouse Engineering Specialization

When you enroll in this course, you'll also be enrolled in this Specialization.

Learn new concepts from industry experts
Gain a foundational understanding of a subject or tool
Develop job-relevant skills with hands-on projects
Earn a shareable career certificate

There are 2 modules in this course

Ready to build data pipelines that power modern analytics? This course transforms you from someone who processes data manually into a data engineer who creates automated, modular pipeline systems.

This Short Course was created to help Data Management and Engineering professionals accomplish scalable, maintainable data processing workflows. By completing this course, you'll be able to design and implement production-ready pipelines that seamlessly move data from raw sources to analytics-ready destinations using industry-standard tools. By the end of this course, you will be able to: • Create modular pipeline stages for data ingestion, cleansing, transformation, and loading • Implement automated workflows using Python, dbt, and Airflow • Deploy scalable solutions on cloud platforms like AWS and Snowflake This course is unique because it focuses on hands-on implementation with real-world scenarios using popular open-source tools that drive today's data infrastructure. To be successful in this project, you should have a background in basic SQL, Python programming, and familiarity with data concepts.

Learners will establish the foundational understanding and core skills for creating modular data pipeline stages, focusing on the principles of separation of concerns and tool integration fundamentals.

What's included

1 video1 reading1 assignment1 ungraded lab

1 video Total 7 minutes

Open Source Tool Ecosystem: Spark, dbt, and Airflow Integration 7 minutes

1 reading Total 12 minutes

Fundamentals of Modular Data Pipeline Architecture 12 minutes

1 assignment Total 3 minutes

Modular Pipeline Design Fundamentals Assessment 3 minutes

1 ungraded lab Total 20 minutes

Create Your First Modular Data Pipeline with API Integration 20 minutes

Learners will implement complete end-to-end data pipelines by integrating modular components with industry-standard tools, culminating in comprehensive assessment of their pipeline development capabilities.