Microsoft

Distributed Training & Advanced Application

Microsoft

Distributed Training & Advanced Application

 Microsoft

Instructor: Microsoft

What you'll learn

  • Build PyTorch DDP and FSDP pipelines for multi-node distributed training, profiling communication overhead, and scaling GPU throughput.

  • Configure DeepSpeed ZeRO optimization stages and apply Microsoft Olive and Azure Container for PyTorch to accelerate large model training.

  • Build computer vision and NLP pipelines for object detection, NER, classification, and QA using Florence-2, CLIP, and Hugging Face Transformers.

  • Design multimodal fusion architectures combining vision, text, and audio using Microsoft Phi-4 and OpenAI Whisper on Azure ML.

Details to know

Shareable certificate

Add to your LinkedIn profile

Assessments

1 assignment

Taught in English

See how employees at top companies are mastering in-demand skills

 logos of Petrobras, TATA, Danone, Capgemini, P&G and L'Oreal

Build your Machine Learning expertise

This course is part of the Microsoft Deep Learning Engineering with Azure Professional Certificate
When you enroll in this course, you'll also be enrolled in this Professional Certificate.
  • Learn new concepts from industry experts
  • Gain a foundational understanding of a subject or tool
  • Develop job-relevant skills with hands-on projects
  • Earn a shareable career certificate from Microsoft

There is 1 module in this course

Synthesize your distributed training and model acceleration skills to scale a massive transformer model across a multi-GPU, multi-node cloud cluster. You will write a Python training script that implements Microsoft DeepSpeed ZeRO-3 parameter sharding and activation checkpointing, paired with a DeepSpeed configuration file implementing ZeRO-3 CPU offloading. You will then write the Azure ML SDK v2 code to submit this job to a remote compute cluster using the curated ACP environment and optimized NCCL environment variables.

What's included

2 readings1 assignment

Earn a career certificate

Add this credential to your LinkedIn profile, resume, or CV. Share it on social media and in your performance review.

Instructor

 Microsoft
445 Courses2,915,380 learners

Offered by

Microsoft

Why people choose Coursera for their career

Felipe M.

Learner since 2018
"To be able to take courses at my own pace and rhythm has been an amazing experience. I can learn whenever it fits my schedule and mood."

Jennifer J.

Learner since 2020
"I directly applied the concepts and skills I learned from my courses to an exciting new project at work."

Larry W.

Learner since 2021
"When I need courses on topics that my university doesn't offer, Coursera is one of the best places to go."

Chaitanya A.

"Learning isn't just about being better at your job: it's so much more than that. Coursera allows me to learn without limits."