Design and implement production ready Lakehouse architectures using Delta Lake and Databricks. By the end of this course, you will be able to build multi layer Medallion pipelines including Bronze, Silver, and Gold layers, manage ACID transactions, enforce and evolve schemas, implement Change Data Capture, and optimize Delta tables for performance using data skipping, compaction, and Liquid Clustering. You will also learn to unify batch and streaming workloads while ensuring reliability, scalability, and recoverability in enterprise environments.
This course stands out by going beyond Delta Lake syntax and focusing on end to end Lakehouse engineering, from architectural design patterns to production optimization and concurrency control. Through structured modules and hands on implementation, you will gain practical experience designing scalable data platforms that support both BI analytics and machine learning workloads.
If you are a data engineer, analytics engineer, or platform architect looking to modernize legacy data warehouses or data lakes, this course provides the applied skills required to build efficient, cost effective, and future ready data infrastructure on Databricks. This is primarily aimed at first- and second-year undergraduates interested in engineering or science, along with high school students and professionals with an interest in programming.
This module introduces the evolution of modern data platforms, from traditional warehouses and data lakes to the unified Lakehouse architecture. Learners explore foundational concepts of Databricks, Apache Spark, and Delta Lake that enable scalable, reliable, and governed data processing.
Das ist alles enthalten
15 Videos5 Lektüren4 Aufgaben
Infos zu Modulinhalt anzeigen
15 Videos•Insgesamt 79 Minuten
Course Introduction•5 Minuten
Warehouses to Lakehouse Data Architecture•6 Minuten
Limitations of Traditional Data Warehouses and Data Lakes•6 Minuten
What is Lakehouse Architecture and Why It Matters•6 Minuten
Principles of Lakehouse Architecture•5 Minuten
Data Lake vs. Data Warehouse vs. Lakehouse•5 Minuten
Demonstration: Databricks Workspace and Lakehouse Components Overview•4 Minuten
Introduction to Delta Lake on Databricks•6 Minuten
Delta Lake Architecture and Core Components•6 Minuten
ACID Transactions in Delta Lake•6 Minuten
Demonstration: Apache Spark Overview•6 Minuten
Demonstration: Creating and Querying First Delta Table on Databricks•4 Minuten
Demonstration: Understanding the Delta Transaction Log•5 Minuten
5 Lektüren•Insgesamt 75 Minuten
Course Syllabus•15 Minuten
Evolution of Analytics Workloads and Data Consumption Patterns•15 Minuten
Organizational Transition from Data Lakes to Lakehouse Architectures•15 Minuten
Open Table Formats: Delta Lake, Iceberg, and Hudi•15 Minuten
Module Summary: Modern Data Architecture Fundamentals•15 Minuten
4 Aufgaben•Insgesamt 33 Minuten
Practice Knowledge Check: Modern Data Architecture Fundamentals•6 Minuten
Practice Knowledge Check: Lakehouse Architecture Fundamentals•6 Minuten
Practice Knowledge Check: Apache Spark, Delta Lake, and Environment Setup•6 Minuten
Knowledge Check: Modern Data Architecture Fundamentals•15 Minuten
Core Delta Lake Concepts and Operations
Modul 2•3 Stunden abzuschließen
Moduldetails
This module focuses on the core operational capabilities of Delta Lake, including storage architecture, metadata management, transactional processing, and schema control. Learners gain hands-on experience with CRUD operations, incremental data pipelines, time travel, and streaming to build reliable, production-ready data workflows.
Das ist alles enthalten
12 Videos4 Lektüren4 Aufgaben
Infos zu Modulinhalt anzeigen
12 Videos•Insgesamt 55 Minuten
Introduction to Metadata in Delta Lake•5 Minuten
Schema Enforcement vs. Schema Evolution•6 Minuten
Demonstration: Inspecting Delta Table Versions and History•4 Minuten
Demonstration: Enforcing and Evolving Schema in Delta Tables•4 Minuten
Transactional Data Operations in Delta Lake•7 Minuten
Demonstration: Creating Managed and External Delta Tables•7 Minuten
Demonstration: Performing CRUD Operations on Delta Tables•3 Minuten
Demonstration: Implementing MERGE for Incremental Loads•3 Minuten
Time Travel and Versioned Data Access•5 Minuten
Streaming and Batch Unification in Delta Lake•5 Minuten
Demonstration: Data Auditing and Recovery Using Time Travel•3 Minuten
Demonstration: Streaming Reads and Writes with Delta Lake•3 Minuten
4 Lektüren•Insgesamt 60 Minuten
Reading: Practical Usage of Delta Lake in Production Environments•15 Minuten
Designing Incremental Data Pipelines for Large-Scale Systems•15 Minuten
Data Lineage, Auditing, and Observability•15 Minuten
Module Summary: Core Delta Lake Concepts and Operations•15 Minuten
4 Aufgaben•Insgesamt 33 Minuten
Practice Assignment: Delta Lake Storage, Metadata, and Schema Management•6 Minuten
Practice Knowlede Check: Delta Table Types and Data Modification•6 Minuten
Time Travel and Streaming with Delta Lake•6 Minuten
Knowledge Check: Core Delta Lake Concepts and Operations•15 Minuten
Lakehouse Architecture and Performance Optimization
Modul 3•3 Stunden abzuschließen
Moduldetails
This module focuses on designing scalable Lakehouse architectures using Medallion patterns and optimizing Delta Lake for performance and cost efficiency. Learners build multi-layer data pipelines and apply advanced optimization techniques to support BI and machine learning workloads.
Das ist alles enthalten
13 Videos5 Lektüren4 Aufgaben
Infos zu Modulinhalt anzeigen
13 Videos•Insgesamt 70 Minuten
Introduction to Medallion Architecture in the Lakehouse•7 Minuten
Layered Data Refinement Using Medallion Architecture•6 Minuten
Lakehouse Pipeline Architecture Design•5 Minuten
Demonstration: Building Bronze to Silver Pipelines using Medallion Architecture•6 Minuten
Demonstration: Transforming Data into Gold Tables•5 Minuten
Data Skipping, Statistics, and File Pruning•6 Minuten
Delta Table Performance Optimization•7 Minuten
Demonstration: File Compaction and Storage Optimization using OPTIMIZE•4 Minuten
Liquid Clustering Overview•5 Minuten
Demonstration: Validating the impact of Delta Lake optimizations•4 Minuten
Delta Lake Interoperability with BI Tools•6 Minuten
Using Delta Lake for Machine Learning Workloads•5 Minuten
Demonstration: Diagnosing the Small File Problem•4 Minuten
5 Lektüren•Insgesamt 75 Minuten
Pipeline Orchestration and Dependency Management•15 Minuten
Lakeflow Connect on Databricks•15 Minuten
Balancing Cost, Latency, and Scalability•15 Minuten
Choosing Storage Layouts for Analytics, BI, and ML•15 Minuten
Module Summary: Lakehouse Architecture and Performance Optimization•15 Minuten
4 Aufgaben•Insgesamt 33 Minuten
Practice Assignment: Lakehouse Design Patterns•6 Minuten
Practice Knowledge Check: Delta Lake Performance Optimization Techniques•6 Minuten
Practice Knowledge Check: Storage Efficiency and Interoperability•6 Minuten
Knowledge Check: Lakehouse Architecture and Performance Optimization•15 Minuten
Advanced Delta Lake and Production Readiness
Modul 4•3 Stunden abzuschließen
Moduldetails
This module focuses on designing scalable Lakehouse architectures using Medallion patterns and optimizing Delta Lake for performance and cost efficiency. Learners build multi-layer data pipelines and apply advanced optimization techniques to support BI and machine learning workloads.
Das ist alles enthalten
9 Videos3 Lektüren4 Aufgaben
Infos zu Modulinhalt anzeigen
9 Videos•Insgesamt 50 Minuten
Concurrency Control and Optimistic Transactions•7 Minuten
Isolation Levels in Delta Lake•6 Minuten
Data Quality, Constraints, and Expectations•6 Minuten
Demonstration: Simulating Concurrent Writes in Delta Lake•7 Minuten
Demonstration: Applying Constraints and Validating Data in Delta Lake•5 Minuten
Introduction to Change Data Capture•6 Minuten
Demonstration: Using Delta Change Data Feed in Delta Lake•5 Minuten
Demonstration: Building Incremental CDC Pipelines using Delta Lake•7 Minuten
Course Summary•3 Minuten
3 Lektüren•Insgesamt 60 Minuten
Failure Modes and Recovery in Distributed Data Systems•15 Minuten
Event-Driven Architectures and Incremental Processing•15 Minuten
Practice Project: Building an End-to-End Lakehouse on Databricks with Delta Lake•30 Minuten
4 Aufgaben•Insgesamt 72 Minuten
Practice Knowledge Check: Advanced Delta Lake Internals and Reliability•6 Minuten
Practice Knowledge Check: Change Data Capture with Delta Lake•6 Minuten
End Course Knowledge Check: Lakehouse Architecture and Delta Lake with Databricks•30 Minuten
Designing a Production-Ready Enterprise Lakehouse Modernization Strategy•30 Minuten
Edureka is an online education platform focused on delivering high-quality learning to working professionals. We have the
highest course completion rate in the industry and we strive to create an online ecosystem for our global learners to equip
themselves with industry-relevant skills in today’s cutting edge technologies.
What is Delta Lake and why is it used with Databricks?
Delta Lake is an open-source storage layer that brings ACID transactions, schema enforcement, and time travel to data lakes. This course teaches you to use Delta Lake natively on Databricks to build reliable, scalable data pipelines.
Do I need prior Databricks experience to enroll?
No. The course starts from the Databricks environment and workspace setup. Basic SQL and Python knowledge is recommended, but no prior Databricks or Delta Lake experience is required.
What is Lakehouse architecture and how is it different from a data warehouse?
A Lakehouse combines the low-cost storage of a data lake with the reliability and performance of a data warehouse. You'll learn how Databricks implements this unified architecture using Delta Lake as its core.
What is the Medallion Architecture covered in this course?
Medallion Architecture is a layered data design pattern using Bronze (raw), Silver (cleaned), and Gold (analytics-ready) layers. The course includes hands-on pipeline building across all three layers using Delta Lake on Databricks.
When will I have access to the lectures and assignments?
To access the course materials, assignments and to earn a Certificate, you will need to purchase the Certificate experience when you enroll in a course. You can try a Free Trial instead, or apply for Financial Aid. The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
What will I get if I purchase the Certificate?
When you purchase a Certificate you get access to all course materials, including graded assignments. Upon completing the course, your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.
Is financial aid available?
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.
Finanzielle Unterstützung verfügbar, weitere Informationen
¹ Einige Aufgaben in diesem Kurs werden mit AI bewertet. Für diese Aufgaben werden Ihre Daten in Übereinstimmung mit Datenschutzhinweis von Courseraverwendet.