- [Morgan] Monitoring is a really important part of running containers on AWS. There are multiple monitoring services that you can use with your container workloads on AWS that are easily integrated and managed. You can, of course, always use third-party monitoring services as well, but it can add operational complexity. The first service I want to cover is applicable to both ECS and EKS based workloads, and it's called Amazon CloudWatch Container Insights. CloudWatch Container Insights makes it easy to collect metrics like CPU, memory, disk, and network utilization, as well as log information, in one centralized location. Let's see how this all works. To work with CloudWatch Container Insights, the first thing you need to do is opt in to Container Insights enabled clusters. This can be done by, in the ECS cluster, going to Account Settings, and then you can scroll down to CloudWatch Container Insights, and enable it for your IAM user. You can see it's already enabled for this account, but this is where you would do it for yours, and then make sure that you click Save afterwards. Keep in mind that the IAM user performing this action needs the appropriate IAM permission, the ecs:PutAccountSetting permission, to perform this action. Once you've enabled this, all the new ECS clusters that you create will automatically be monitored by CloudWatch Container Insights. If I scroll back up and then click back on Clusters, if I take a look at my two ECS clusters, you'll see that I already have CloudWatch Container Insights enabled for both of them. If I click on the demo-cluster, and then I click on the Metrics tab, you can see that we have some monitoring information built in directly into the ECS console. But to explore further, I can click on the View Container Insights button, and then it will take me to the CloudWatch dashboard, and we can dig into it a little more from here. We are now in the CloudWatch Container Insights dashboard. You can explore this page and see what types of metrics that it can show you. I want to check to see how many running tasks there are, and we can see here that there is a graph on this dashboard that shows me the task count, and we can see that I had one task, and then it bumped up to two and then back down to one, and then back up to two. So we can view that information here, and we can also see that I had two services running, and then that went up to three services. You can also see the amount of CPU or memory that is being consumed from your cluster at this page. You can also drill down to the service or task level for ECS. So if I click on this dropdown here, you can then change if you want to look at ECS instances, or services, or tasks. So if I click on Services, we can see I have new information. This is specifically for the services on this demo-cluster. I could change this to look at a different cluster, so you can see how you can explore this data here. I'm going to go back to the demo-cluster. And from here, what we want to do is see how this data would change if I add more tasks to the ECS cluster. So I'm going to go back to the ECS dashboard, click on ECS, and then I'm going to go to my demo-cluster, scroll down, and then I'm going to click on one of my services. I'm going to just go ahead and select this example-linux-service. And then I want to Update this service, and then I can decide how many services, I mean, how many tasks, I want for the service. I'm going to change this to be 4, and then I'm going to scroll down and click Next Step. Next Step again. Next Step again. And then you can review your configurations, and then click Update Service. And now I'm going to click View Service, and then we can scroll down, and you can watch this deploy. So, you can see we have the desired count, and the pending count, and then the running count. If you expand these, you can read those a little bit better. So desired count is 4, pending count is 3 running count is 1, so we already had one running. We launched three more, and those are in our pending state. So we'll give this a few minutes, and then we will go back into the CloudWatch dashboard and see how that data changed. Now, we can go back into CloudWatch Insights and see how this affected the metrics for our cluster. So back into the CloudWatch dashboard, we can now select ECS Services. And if I scroll down and then I open up this one here, for Number of Desired Tasks, and then change the time to be 1 hour, we can see that we had one task running for example-linux-service, and then that spiked, and now we have four. So we can see that this is collecting this data. You can also search and analyze your ECS log data, as well as using CloudWatch Logs Insights. So to do this, I will exit out of this, and then we're going to scroll up, open up the side panel, and then click on Logs Insights. From here, you can choose the log group that you want to work with, so I'm going to open up this dropdown, and I'm going to select this demo-cluster log group, here. If you enable CloudWatch Container Insights in the ECS dashboard as I did earlier in this video, it will auto-populate ECS logs. With CloudWatch Logs Insights, you can visualize log data and perform queries to help you troubleshoot and identify potential issues. It uses a query language that has a few simple commands, such as stats. You can use stats to calculate statistics based off of the values of log fields. For example, I could run this query, which states stats avg(CpuUtilized), avg(MemoryUtilized) by bin (30m). Using this, I can see the average CPU utilization and memory utilization for log events for each 30-minute period. If you wanted to, you could use this information to improve performance for your tasks over time. So I'm going to go ahead and click Run query, and we can then scroll down, and see the data. The results here show the number of log events in the log group that were received by CloudWatch Logs for each 30-second period. To visualize this, we can click on the Visualization tab, and the results are shown as a line graph. You can also view this as a stacked area, or a bar, or pie chart. Depending on the data, you'll choose whatever is the best fit. You can also add commonly used queries to your dashboard, so you can get a quick glance at important information without needing to run this query manually all the time. So, if you're leveraging ECS or EKS, using CloudWatch Container Insights can provide diagnostic information, metrics, and help you with log investigation. Beyond CloudWatch Container Insights, there is another managed monitoring service for EKS and self-managed Kubernetes clusters that is called Amazon Managed Service for Prometheus. Amazon Managed Service for Prometheus is a serverless, Prometheus-compatible monitoring service for container metrics that makes it easier to securely monitor your containers. What's nice about this service is that a lot of people are already using Prometheus for monitoring, and then another tool called Grafana for visualization. So you can use the same open-source Prometheus data model and query language that you use today to monitor the performance of your containerized workloads, while also enjoying improved scalability, availability, and security without having to manage all of the underlying infrastructure for yourself. I'll include resources for you to learn more about these monitoring services in the notes following this video.
