This course covers part of the model development phase of the machine learning (ML) lifecycle. In this course, you will learn how to analyze and evaluate model performance for traditional ML and generative AI. This course covers establishing performance baselines, detecting drift, interpreting metrics such as bias and variance, and diagnosing convergence issues. You will track experiments with MLflow on Amazon SageMaker AI, compare shadow and production variants, and explain predictions with SHAP. For generative AI, you will use Amazon Bedrock evaluations, natural language processing metrics, the large language model as a judge (LLM-as-a-judge) pattern, and RAG evaluation.