Data Analytics and Machine Learning for Big Data

This course is part of Microsoft Big Data Management and Analytics Professional Certificate

Instructor: Microsoft

Access provided by PiggyVest

5 modules

Gain insight into a topic and learn the fundamentals.

3 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

5 modules

Gain insight into a topic and learn the fundamentals.

3 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

Skills you'll gain

Tools you'll learn

Details to know

Shareable certificate

Add to your LinkedIn profile

Assessments

46 assignments¹

AI Graded see disclaimer

Taught in English

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

logos of Petrobras, TATA, Danone, Capgemini, P&G and L'Oreal

Build your Data Analysis expertise

This course is part of the Microsoft Big Data Management and Analytics Professional Certificate

When you enroll in this course, you'll also be enrolled in this Professional Certificate.

Learn new concepts from industry experts
Gain a foundational understanding of a subject or tool
Develop job-relevant skills with hands-on projects
Earn a shareable career certificate from Microsoft

There are 5 modules in this course

This advanced course teaches machine learning and AI techniques for big data systems. Learners will build end-to-end ML pipelines with PySpark ML, implement supervised and unsupervised models, and apply NLP techniques at scale. The course also explores deep learning, distributed training, and integrating Generative AI into big data workflows.

By the end of this course, you will be able to: Implement ML pipelines using PySpark ML Build supervised, unsupervised, and recommendation models Apply NLP and text analytics to large datasets Integrate Generative AI and LLMs with big data systems Tools & Software: PySpark ML, PyTorch, TensorFlow, Azure Machine Learning, Azure OpenAI Service Skills: Machine learning, NLP, Deep learning, Generative AI, Model evaluation

Machine learning appears quite different when data exceeds the capacity of a single system. In this section, learners explore the foundational ideas behind machine learning in big data environments and how familiar approaches change at scale. You will examine supervised and unsupervised learning, regression and classification problems, and the practical challenges that arise with massive datasets—such as scalability, distributed computing, and the need to adapt algorithms for large-scale processing.

What's included

6 videos3 readings7 assignments

6 videos Total 29 minutes

Machine Learning Transforms Big Data into Business Intelligence 4 minutes
ML Problem Classification and Business Mapping 7 minutes
Data Quality Drives ML Success at Scale 4 minutes
Distributed Data Preparation Workflows 6 minutes
Rigorous Evaluation Prevents ML Disasters at Scale 4 minutes
Implementing Scalable Model Evaluation 5 minutes

3 readings Total 90 minutes

Machine Learning Fundamentals for Big Data Environments 30 minutes
Big Data ML Preparation Techniques 30 minutes
ML Model Evaluation for Big Data Systems 30 minutes

7 assignments Total 210 minutes

Machine Learning Problem Analysis 30 minutes
ML Fundamentals for Big Data Assessment 30 minutes
ML Data Preparation Pipeline 30 minutes
Data Preparation for ML at Scale Assessment 30 minutes
Scalable Model Evaluation 30 minutes
Model Evaluation at Scale Assessment 30 minutes
ML Fundamentals for Big Data Mastery 30 minutes

A practical foundation for building scalable machine learning solutions using PySpark ML in big data environments. The content focuses on designing and implementing end-to-end machine learning pipelines with transformers and estimators, while developing regression, classification, and clustering models that scale across distributed systems. Emphasis is placed on real-world implementation and informed platform selection for enterprise deployments using Azure Databricks, Microsoft Fabric, and Azure HDInsight, ensuring solutions are both technically robust and operationally viable at scale.

What's included

6 videos3 readings10 assignments

6 videos Total 36 minutes

Democratizing Machine Learning at Enterprise Scale 4 minutes
PySpark ML Pipeline Development Across Platforms 10 minutes
Supervised Learning Success Stories in Enterprise Big Data 5 minutes
Supervised Learning Model Development 6 minutes
Recommendation Systems Drive Business Growth 4 minutes
Building Scalable Recommendation Systems 8 minutes

3 readings Total 90 minutes

PySpark ML Architecture and Platform Comparison 30 minutes
Supervised Learning Algorithms for Big Data 30 minutes
Unsupervised Learning and Recommendation Systems 30 minutes

10 assignments Total 300 minutes

ML Pipeline Component Development 30 minutes
ML Platform Comparison and Pipeline Creation 30 minutes
PySpark ML Platform Fundamentals Assessment 30 minutes
Supervised Learning Implementation 30 minutes
Supervised Learning Model Development 30 minutes
Supervised Learning at Scale Assessment 30 minutes
Recommendation System Implementation 30 minutes
Recommendation System Development 30 minutes
Unsupervised Learning and Recommendations Assessment 30 minutes
PySpark ML Implementation Mastery 30 minutes

Large-scale text analytics introduces the challenges and techniques required to process and analyze unstructured text at enterprise scale using distributed computing frameworks. The focus is on applying natural language processing (NLP) techniques in scalable architectures to support text classification, sentiment analysis, and entity and relationship extraction across massive text corpora. Emphasis is placed on practical, production-oriented approaches for handling high-volume text data, with integration of Azure Cognitive Services to enhance accuracy, scalability, and operational efficiency in real-world analytics solutions.

What's included

6 videos3 readings10 assignments

6 videos Total 39 minutes

Unlocking Value from Unstructured Text at Scale 5 minutes
Building Scalable Text Processing Pipelines 9 minutes
Advanced NLP Drives Business Intelligence 5 minutes
Implementing Advanced NLP at Scale 7 minutes
Production-Scale Text Classification Transforms Business Operations 4 minutes
Building Production Text Classification Systems 8 minutes

3 readings Total 90 minutes

Distributed Text Processing Techniques 30 minutes
Advanced NLP Techniques for Big Data 30 minutes
Scalable Text Classification Architectures 30 minutes

10 assignments Total 300 minutes

Text Preprocessing Pipeline Development 30 minutes
Scalable Text Preprocessing Design 30 minutes
Text Processing at Scale Assessment 30 minutes
Advanced NLP Implementation and Monitoring 30 minutes
NLP System Architecture Design 30 minutes
Advanced NLP Techniques Assessment 30 minutes
Text Classification System Development 30 minutes
Text Classification System Implementation 30 minutes
Text Classification at Scale Assessment 30 minutes
Text Analytics and NLP Mastery 30 minutes

Deep Learning for Big Data introduces the fundamentals of deep learning and advanced architectures specifically adapted for big data environments. Students will learn to implement neural networks for big data applications, apply transfer learning techniques with pre-trained models, and scale deep learning training across distributed clusters using modern frameworks and optimization techniques.

What's included

6 videos3 readings10 assignments

6 videos Total 31 minutes

Deep Learning Revolutionizes Big Data Analytics 5 minutes
Neural Network Implementation in Big Data Frameworks 5 minutes
Advanced Architectures Transform Complex Data Analysis 6 minutes
CNN and RNN Implementation at Scale 5 minutes
Distributed Deep Learning Enables Breakthrough Scale 4 minutes
Implementing Distributed Deep Learning Training 5 minutes

3 readings Total 90 minutes

Deep Learning Architectures for Big Data 30 minutes
Advanced Deep Learning Architectures for Scale 30 minutes
Distributed Deep Learning Training Strategies 30 minutes

10 assignments Total 300 minutes

Neural Network Implementation 30 minutes
Neural Network for Big Data Classification 30 minutes
Deep Learning Fundamentals Assessment 30 minutes
Advanced Architecture Implementation 30 minutes
Deep Learning Architecture Design 30 minutes
Advanced Deep Learning Architectures Assessment 30 minutes
Distributed Training Implementation and Management 30 minutes
Distributed Deep Learning Training 30 minutes
Distributed Deep Learning Training Assessment 30 minutes
Deep Learning for Big Data Mastery 30 minutes

Generative AI and Big Data Integration explores how generative AI transforms big data analytics by enabling intelligent, natural language–driven workflows at scale. You will learn how foundation models and large language models integrate with distributed data pipelines to automate insights, enhance analytics, and power modern data applications. Through hands-on labs, you will implement LLM integration, apply fine-tuning for domain-specific use cases, and design production-ready GenAI solutions for real-world big data scenarios.