Let's start to talk about general ideas of log service. Before we get started, the first thing we should know is, what exactly is log management? In short, log management comprises an approach to dealing with large volumes of computer-generated log messages. It generally covers these four main parts : log collection; the way to collect log data from server, and log data query; the way to search log data. The third one is log data analysis, it includes log data virtualization and another method of data analysis. Last but not least, log storage; store the collected log data. Now let's see the structure of log service. It's a one-stop service for log data. Log service binds the four parts of log management together. This is the structure of log service. Log service can basically be separated to three parts: LogHub, LogSearch, and LogShipper. We can firstly collect log data by using a LogHub from Elastic Computer Service, containers, mobile terminals, open-source software and JS to access real-time log data service. Then use LogHub to consume the log data by the provided real-time consumption interface. We can analyze the collected log data by using LogSearch, it provides PB scale log analysis. We can index query and analyze log data in real-time. In addition, we can use log shipper which is a stable and reliable lock shipper function to ship log data to storage services like OSS, MaxCompute and Tablestore. In the final part of this course, I will show you how to use log service. Firstly, using LogHub, collect bank user log data from ECS. Then query and analyze the log data by LogSearch. Finally, ship the log data to OSS. This is a basic structure of Log Service. Why should we choose Log Service? Because it has significant improvement compared to traditional log data analyzed approach. In traditional approach, we firstly need to collect log data from instrumentation and store it. Then we need to do the computation, like data cleaning and grouping. After that, we need to store it again in a warehouse. Finally, we can visualize it. The traditional approach actually requires high maintenance costs in terms of data volume, machine, operation, environment compatibility. In Log Service, we don't need such process. Log Service provides all the functions like log data collection, storage, consumption, computation, analysis and visualization. We just need to focus on analytics and data inside. We're free from dirty work. The other reason is Log Service is one component of Alibaba Cloud ecosystem. It connects to services in Alibaba Cloud like ECS and OSS, MaxCompute, etc. They basically improve the operation and maintenance efficiency, cut the cost of building servers, higher operation and maintenance and build the processing capabilities to handle massive logs in the data technology era. It's a one-stop service for our log data. Log Service has many advantages. LogHub is a more cost-effective choice for users in 98 percent of scenarios compared to building Kafka with purchased Cloud hosts plus Cloud risks at less than 30 percent of the Kafka cost for small websites, and it provides restful APIs and supports data collection on mobile devices, saving you the cost of the gateway servers for log collection. In addition, LogHub is O&M-Free and we can autoscaling anytime and anywhere. For a LogShipper, LogShipper have the following five advantages : it requires no code or machine resources, and provides flexible configuration, rich monitoring data, PB grade scalability and it is now free. LogSearch, the part responsible for log data query and analysis. Have at less than 15 percent of the cost of purchasing cloud hosts plus self-building ELK, and offers powerful query capability, massive data processing scale, a better price than the above-mentioned log management software for its ability to seamlessly integrate with various popular stream computing plus offline computing frameworks to allow for unobstructed flow of log data. With the advantages of Log Service, Log Service can be applied in many scenarios. The first scenario is, of course, log data analysis. The first step of log data analysis is to collect clips with it through agents or API. Then those log data can be analyzed through the LogHub. For example, we can use LogHub to analyze customers' favorite TV series, the most popular channel, the subscribe rate for each cities, and so on. Business executives can make proper strategies by this. Log data analysis can be applied to many areas like streaming media, e-commerce, mobile analytics, and game operations. The other scenario is log audit. We first collect log data to Log Service. In this process, we don't need to worry about we mistakenly delete some data or data deleted by hackers. Log Service provide high-level security. After the log data being collected, we can apply LogSearch to analyze the data quickly. For example, we can query operation recurrence of a specific account. Then we can use LogShipper to ship the log data to OSS or MaxCompute saves the log data at a long time to satisfy audit requirements. This scenario is applicable to areas like e-commerce, government platform and websites. The third scenario is problem diagnosis. We can use LogHub to collect log data from clients, mobile devices, and servers. We can use functions like cloud monitor and stream compute to obtain the real-time with a record. When we request or others [inaudible] we don't need log on the server. Instead of that, we can't directly use LogSearch to query key words, request time, and find the problem before it cause big effect. This scenario is applicable to areas like trading systems, other systems and mobile network. The first one is operation and maintenance. We can collect log data from handled and [inaudible] machines and use Log Service to manage this log data. We can process real-time supervision to visit log indexing and query operation log and store important log offline. Users who manage multiple servers may need this, and there are many other scenarios of Log Service. Metering and billing business system monitoring, vulnerability detection, operation analysis, and mobile client analysis. Log Service is up critters in Alibaba Cloud. Almost all Cloud products are using Log Service to process and analyze logs. That is a brief intro of Log Service. In the next part, I will talk about query and visualization methods in Log Service.
