Are programming skills required?

Basic programming knowledge is helpful but not mandatory. The course introduces PySpark concepts in a beginner-friendly manner.

What tools and technologies are covered?

The course covers Databricks, Apache Spark, PySpark, Unity Catalog, Data Lakes, Data Warehouses, and Lakehouse architecture.

Will I get hands-on practice?

Yes. The course includes multiple demonstrations that allow learners to work directly with Databricks notebooks, data processing, and ingestion workflows.

Does the course cover real-world data ingestion techniques?

Yes. The course includes batch ingestion, real-time ingestion, API-based ingestion, and modern data source integration concepts.

Will I learn Apache Spark in this course?

Yes. You will learn Spark fundamentals, DataFrames, transformations, schemas, and common data processing operations using PySpark.

Will I learn about modern data architectures?

Yes. The course covers Data Lakes, Data Warehouses, and Lakehouse architecture, along with their use cases and differences.

Is this course focused on theory or practical learning?

The course combines conceptual understanding with hands-on demonstrations to help learners apply what they learn.

Is this course part of a Specialization?

Yes. It is the first course in the Data Engineering for AI and ML Pipelines Specialization. You can also take it on its own.

What will I get if I subscribe to this Specialization?

When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.

Is financial aid available?

Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.

Data Engineering and Spark Foundations for AI and ML

La Fête du Travail commence avec plus de 70 $ d'économies sur Coursera Plus. Bénéficiez de 40 % de réduction pendant 3 mois.

Ce cours n'est pas disponible en Français (France)

Nous sommes actuellement en train de le traduire dans plus de langues.

Data Engineering and Spark Foundations for AI and ML

Ce cours fait partie de Spécialisation "Data Engineering for AI and ML Pipelines"

Instructeur : Edureka

Inclus avec En savoir plus

Demander à Coursera

3 modules

Obtenez un aperçu d'un sujet et apprenez les principes fondamentaux.

niveau Débutant

Expérience recommandée

6 heures à compléter

Planning flexible

Apprenez à votre propre rythme

3 modules

Obtenez un aperçu d'un sujet et apprenez les principes fondamentaux.

niveau Débutant

Expérience recommandée

6 heures à compléter

Planning flexible

Apprenez à votre propre rythme

Ce que vous apprendrez

Explain data engineering concepts, Lakehouse architecture, and the role of Databricks in modern data platforms.
Apply Apache Spark and PySpark to process, transform, and manage structured data efficiently.
Implement batch, API-based, and streaming data ingestion workflows using Databricks.
Organize data assets using Unity Catalog, catalogs, schemas, and volumes.

Compétences que vous acquerrez

Catégorie : Applied Machine Learning
Catégorie : Data Processing
Catégorie : Artificial Intelligence
Catégorie : Machine Learning
Catégorie : Data Import/Export
Catégorie : Data Collection
Catégorie : Data Pipelines
Catégorie : SQL
Catégorie : Database Architecture and Administration
Catégorie : Data Warehousing
Catégorie : Data Architecture
Catégorie : Data Management
Catégorie : Data Entry
Catégorie : Distributed Computing

Outils que vous découvrirez

Catégorie : JSON
Catégorie : Python Programming
Catégorie : Data Lakes
Catégorie : Databricks
Catégorie : PySpark
Catégorie : Apache Spark

Détails à connaître

Certificat partageable

Ajouter à votre profil LinkedIn

Récemment mis à jour !

juillet 2026

Évaluations

6 devoirs

Enseigné en Anglais

Découvrez comment les employés des entreprises prestigieuses maîtrisent des compétences recherchées

En savoir plus sur Coursera pour les affaires

logos de Petrobras, TATA, Danone, Capgemini, P&G et L'Oreal

Élaborez votre expertise du sujet

Ce cours fait partie de la Spécialisation "Data Engineering for AI and ML Pipelines"

Lorsque vous vous inscrivez à ce cours, vous êtes également inscrit(e) à cette Spécialisation.

Apprenez de nouveaux concepts auprès d'experts du secteur
Acquérez une compréhension de base d'un sujet ou d'un outil
Développez des compétences professionnelles avec des projets pratiques
Obtenez un certificat professionnel partageable

Il y a 3 modules dans ce cours

This course builds a strong foundation in modern data engineering using Databricks, Apache Spark, and Lakehouse architecture. You will learn how data engineers design, process, organise, and ingest data to support analytics, AI, and machine learning workflows.

You will begin by exploring the role of data engineering in modern organisations, including how it differs from data science and why reliable data pipelines are essential for AI/ML success. You will then get introduced to Databricks, its workspace environment, notebooks, and core platform capabilities. Next, you will work with Apache Spark and PySpark to process data at scale. You will explore DataFrames, transformations, schemas, supported file formats, and practical workflows for reading, transforming, and writing data in formats such as CSV, JSON, and Parquet. You will also examine modern storage concepts, including data lakes, data warehouses, and Lakehouse architecture. Through Databricks, you will learn how to organise data using catalogs, schemas, and volumes, and understand how structured data management supports scalable engineering workflows. By the end of this course, you will be able to: - Explain core data engineering concepts and the role of Databricks. - Describe Lakehouse architecture and modern data storage patterns. - Process and transform data using Apache Spark and PySpark. - Organise project data using catalogs, schemas, and volumes. - Ingest data through batch, API-based, and real-time workflows. Designed for aspiring data engineers, data analysts, software developers, and students entering the data field, this course prepares you to build a strong technical foundation for advanced Databricks, Delta Lake, and AI/ML pipeline development.

Build a strong foundation in data engineering by exploring its role in modern analytics and AI/ML ecosystems. This module introduces the data engineering lifecycle, key architectural concepts such as data lakes, warehouses, and lakehouses, and provides hands-on experience with the Databricks platform. Learners will gain the knowledge needed to navigate Databricks, create notebooks, and understand how modern data platforms support scalable data and AI workloads.

Inclus

12 vidéos4 lectures2 devoirs

12 vidéosTotal 58 minutes

Specialization Introduction6 minutes
Course Introduction 5 minutes
Introduction to Data Engineering 5 minutes
Role of Data Engineering in AI/ML Pipelines4 minutes
Exploring Data Engineering Lifecycle7 minutes
Undercurrents of the Data Engineering Lifecycle7 minutes
Introduction to Databricks 5 minutes
Exploring the Databricks Platform4 minutes
Building First Notebook in Databricks4 minutes
Data Lake vs Data Warehouse: Foundational Concepts5 minutes
Lakehouse Architecture in Modern Data Systems4 minutes
Exploring the Lakehouse Structure in Databricks3 minutes

4 lecturesTotal 40 minutes

Course Syllabus: Data Engineering and Spark Foundations for AI and ML10 minutes
Data Engineering vs. Data Science: Understanding the Distinctions10 minutes
Industry Use Cases of Databricks for AI/ML10 minutes
Databricks Free Edition Guide10 minutes

2 devoirsTotal 21 minutes

Introduction to Data Engineering and Databricks15 minutes
Revisiting the Fundamentals of Data Engineering and Databricks6 minutes

Learn how modern data platforms process large-scale datasets using Apache Spark and PySpark. This module covers Spark’s core architecture, execution concepts, and DataFrame-based transformations while providing hands-on experience reading, processing, and storing data in common formats. Learners will build foundational skills for scalable data processing and analytics within the Databricks environment.

Inclus

7 vidéos2 devoirs

7 vidéosTotal 36 minutes

Apache Spark: Engine Behind Big Data Processing5 minutes
Spark Core APIs and Execution Concepts 6 minutes
PySpark DataFrames and Core Operations5 minutes
Supported Data Formats in Spark6 minutes
Using Built-in Datasets in Databricks4 minutes
Reading CSV and JSON in Spark6 minutes
Writing Transformed Data in Parquet Format6 minutes

2 devoirsTotal 21 minutes

Data Processing with Apache Spark and PySpark15 minutes
Core Concepts Check: Apache Spark and PySpark6 minutes

Develop the foundational skills required to organize and ingest data in modern Databricks environments. This module explores Unity Catalog structures, including catalogs, schemas, and volumes, while introducing key data ingestion concepts, source systems, and ingestion patterns. Through hands-on exercises, learners will work with batch, streaming, and API-based ingestion workflows to build reliable and scalable data pipelines.

Inclus

10 vidéos3 lectures2 devoirs

10 vidéosTotal 57 minutes

Evolution of Storage in Databricks5 minutes
Working with Catalogs, Schemas, and Volumes6 minutes
Organizing Project Files in Unity Catalog Volumes6 minutes
Introduction to Data Ingestion 5 minutes
Data Source Systems in Modern Pipelines7 minutes
Exploring Data Ingestion Options in Databricks5 minutes
Ingesting Data from a Public REST API7 minutes
Simulating Batch Ingestion in Databricks5 minutes
Real-Time Ingestion in Databricks8 minutes
Course summary3 minutes

3 lecturesTotal 50 minutes

Data Ingestion Patterns in Data Engineering10 minutes
Enterprise Data Ingestion vs. Databricks Free Edition10 minutes
Practice project: Retail Order Pipeline Analytics for UrbanCart Retail30 minutes

2 devoirsTotal 36 minutes

End Course Knowledge Check: Data Engineering and Spark Foundations for AI and ML30 minutes
Data Organization and Ingestion Fundamentals6 minutes

Obtenez un certificat professionnel

Ajoutez ce titre à votre profil LinkedIn, à votre curriculum vitae ou à votre CV. Partagez-le sur les médias sociaux et dans votre évaluation des performances.

Instructeur

Edureka

234 Cours215 480 apprenants

Offert par

Edureka

En savoir plus sur Data Analysis

Packt
Data Engineering with Databricks Cookbook
Cours
Statut : Essai gratuit
Duke University
Spark, Hadoop, and Snowflake for Data Engineering
Cours
Statut : Essai gratuit
Edureka
Data Engineering for AI and ML Pipelines
Spécialisation
Packt
Data Engineering with Scala and Spark
Cours

Pour quelles raisons les étudiants sur Coursera nous choisissent-ils pour leur carrière ?

Felipe M.

Étudiant(e) depuis 2018

’Pouvoir suivre des cours à mon rythme à été une expérience extraordinaire. Je peux apprendre chaque fois que mon emploi du temps me le permet et en fonction de mon humeur.’

Jennifer J.

Étudiant(e) depuis 2020

’J'ai directement appliqué les concepts et les compétences que j'ai appris de mes cours à un nouveau projet passionnant au travail.’

Larry W.

Étudiant(e) depuis 2021

’Lorsque j'ai besoin de cours sur des sujets que mon université ne propose pas, Coursera est l'un des meilleurs endroits où se rendre.’

Chaitanya A.

’Apprendre, ce n'est pas seulement s'améliorer dans son travail : c'est bien plus que cela. Coursera me permet d'apprendre sans limites.’

Profitez de plus de 10 000 programmes grâce à notre offre spéciale pour la Fête du Travail
Get Coursera Plus
Commencez par réaliser des économies simples pour vos équipes qui travaillent dur
Save 30% today

Foire Aux Questions

This course introduces the fundamentals of Data Engineering and provides hands-on experience with Databricks, Apache Spark, and modern data architectures.

This course is suitable for aspiring Data Engineers, Data Analysts, AI/ML practitioners, software professionals, and anyone interested in modern data platforms.

No. The course starts with foundational concepts and gradually progresses to practical data processing and ingestion workflows.

To access course materials, assignments, and earn a Certificate, you'll need to purchase the Certificate experience when you enroll in a course. Eligible learners may also have the option to start with a Free Trial. Some courses may also offer a Full Course, No Certificate option. This lets you access course materials, submit required assessments, and receive a final grade, but you won't be able to earn or purchase a Certificate.