Welcome to the section on Selecting Windowing Functions. In this section we'll cover the following, Windowing Functions in Azure, How Stream Analytics Relate to Windowing Functions, Stream Processing Jobs, How Stream Analytics Support Native Windowing Functions, Temporal Windows, Using Window Functions in the GROUP BY clause of the Query Syntax, and Aggregating Events over Multiple Windows using WindowsQ. In applications that process real-time events, that is common to perform some set-based computation, or other operations over subsets of events that fall within a period of time. It is important to have a simple way to work with the time component of query logic in the system. Because the concept of time is a fundamental necessity to Complex Event Processing Systems. Azure Stream Analytics, these events subsets are defined through windows to represent groupings of time. But window contains event data along a timeline, enables you to perform various operations against the events in that window. Here we were counting the tweets per time zone every 10 seconds. But window function assigns events in your stream to Windows. To be precise it is more at window relation than a function because it theoretically does not have to assign all events to Windows. That is, it is not total, and they can assign an event to multiple windows. In addition, it is neither surjective. Not all windows have to contain events, nor injective, the window can contain multiple events. Nevertheless, we are going to stick to the mathematically incorrect term window function. Given a window function and a stream of data, we can compute aggregates on events inside each window. There are four types of windows, Tumbling, Hopping, Sliding and Session, we'll look at these types in a moment. Stream Analytics has native support for windowing Functions, enabling developers to author complex stream processing jobs with minimal effort. You can choose from the four types of windows. Then you use the window functions in the GROUP BY clause of the query syntax and your Stream Analytics jobs. You can also aggregate events over multiple windows using the Windows function. Tumbling windows are another way of selecting data from an Azure stream to drive an Azure ML experiment. Tumbling window functions are used to segment a data stream into distinct time segments and perform a function against them. The key differentiators of a tumbling window, are that they repeat, do not overlap in an event can't belong to more than one tumbling window. A Hopping window is very similar to the Tumbling window, but here the windows have an overlap. It aggregates data within a time window and does that for a specified duration. The Sliding window is the most difficult to explain, it aggregates the values and the time window every time a new event occurs or an existing event falls out of the time window. So when using the sliding window, you are interested in aggregating values whenever an event occurs. This is in contrast to the hopping and tumbling windows, which have a fixed interval. Session window functions group events that arrive at similar times, filtering out periods of time where there is no data. It has three main parameters, a timeout, maximum duration, and to partitioning key. Azure Stream Analytics is a real-time analytics and complex event processing engine that is designed to analyze and process high volumes of fast streaming data from multiple sources simultaneously. The following scenarios are examples of when you can use Azure Stream Analytics. You can analyze real-time telemetry streams from IoT devices. You could analyze weblogs and clickstream analytics. You could also analyze geo-spatial Analytics for fleet management and driverless vehicles. You could consider remote monitoring and predictive maintenance of high-value assets. Finally, real-time analytics on point of sale data for inventory control and anomaly detection. Because every service can have access to the raw events, it no longer must wait for ETL processes to finish. The image below illustrates individual events happening over time. By definition, streams are unbound. So if we're looking for an aggregated view, we need to introduce some boundary. In the batch world, this boundary was heavily influenced by the ETL schedule. But in stream analytics, we are free to discrete eyes this dream any way we want in order to perform aggregations on top. But which timestamp are you using to assign an event to a window when the event was generated or when the event arrived at the processor. If you're using the creation time, you need to be aware of that event producers may not have properly synchronized their clocks. This concludes this lesson. Next, we will look at how Stream Analytics support Native Windowing Function.
