This section teaches you to establish governance processes for production prompt management, including authoring standards, change control workflows, A/B testing methodology, and rollback procedures.

This module teaches you to design comprehensive RAG experiment matrices that systematically vary chunking strategies, embedding models, and retrieval configurations while maintaining reproducibility.

This module teaches you to evaluate RAG pipeline experiments using the Azure OpenAI Evaluation SDK, interpret results across quality metrics, and make data-driven configuration decisions.

This module teaches learners to select and combine evaluation metrics by GenAI use case. Learners distinguish reference-based metrics, RAG retrieval/response metrics, classification metrics, rubric/custom evaluators and safety/security evaluators so they can design evaluation frameworks that avoid false confidence and production risk.

This module teaches you to build automated evaluation gates that integrate into CI/CD pipelines and design human-in-the-loop processes to handle cases where automation is insufficient.

This module teaches you to understand the fundamental limitations of automated evaluation gates and design mitigations that prevent false confidence in quality controls.

This section teaches you to design the complete operational lifecycle for fine-tuning generative AI models, from Azure ML data asset preparation through Azure OpenAI Foundry model catalog registration, with emphasis on reproducibility via seed/hyperparameter manifesting, Azure Monitor governance logging, and Azure Service Health base model retirement planning, with emphasis on reproducibility and governance.

This section teaches you to evaluate different fine-tuning approaches against use case requirements and design model versioning and promotion strategies that enable safe progression from development to production.

This module teaches you to use AI assistance to draft, vary, review, and document prompts while preserving human ownership, safety review, version control, and evaluation-before-release discipline.

Learners design a comprehensive Prompt & RAG Experimentation Framework for an enterprise scenario. Learners will create a PromptOps governance policy, design a RAG experiment matrix, build an evaluation framework with automated gates and human-in-the-loop processes, and document the complete experimentation workflow, producing portfolio-ready artifacts that demonstrate your ability to establish systematic optimization practices.