Learn to optimize LLM observability with Datadog. This course covers setting up Datadog, instrumenting AI workflows, debugging multi-agent systems, and monitoring performance, security, and costs in enterprise environments. Gain hands-on experience in ensuring production-grade AI applications.
This course provides an in-depth look at LLM observability using Datadog, equipping you with essential skills for monitoring and tracing AI applications. You will begin with Datadog setup, understanding span types, SDK integrations, and the process of tracing LLM calls in local environments. As the course progresses, you’ll dive into instrumenting multi-step AI workflows, using annotations and tags to track performance, and debugging complex agentic AI systems. You’ll also explore LangChain integration, learning how to instrument RAG pipelines and use Datadog for full observability. The course goes further by focusing on evaluations, quality monitoring, and A/B testing to optimize LLM performance. You’ll learn how to create custom evaluations and integrate them with your code for automated monitoring. In addition to performance and debugging, you’ll tackle cost optimization, ensuring your LLM applications run efficiently and affordably. You’ll also gain knowledge in security and compliance practices, such as PII redaction and deployment best practices. By the end of the course, you will have a comprehensive understanding of deploying and maintaining observability in production-grade LLM applications, ensuring they are secure, efficient, and cost-effective. This course is designed for AI engineers, DevOps professionals, data scientists, and anyone working with AI systems, especially LLM applications. It is ideal for those focused on AI observability, monitoring, and optimization in production environments. A basic understanding of cloud-based systems and software development is recommended. Familiarity with Datadog or LLM technologies is beneficial but not required. This course takes a hands-on approach, offering practical exercises to set up observability tools, debug complex AI workflows, and optimize performance. Participants will learn through interactive sessions and real-world scenarios, ensuring they gain valuable skills to deploy and monitor LLM applications effectively in production. This course is based on LLM Monitoring & Tracing: AI Observability with Datadog, by Paulo Dichone. This course is licensed and distributed by Packt. All rights reserved. Packt is one of the world's most prolific publishers of cutting-edge technical content. For over two decades we've made it our mission to curate and publish the knowledge of only the very best technical experts. We focus on real-world courses that help our customers get the job done, with coverage that extends across a wide range of established and cutting-edge technical topics. If you're an individual or an organisation that embraces learning by doing, Packt is the perfect fit for you.
