Data Processing with Azure

This course is part of Microsoft Azure: AI, Infrastructure, and Data Solutions Specialization

Instructor: LearnQuest Network

Access provided by Masterflex LLC, Part of Avantor

7,849 already enrolled

8 modules

Gain insight into a topic and learn the fundamentals.

68 reviews

Intermediate level

Some related experience required

1 week to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

8 modules

Gain insight into a topic and learn the fundamentals.

68 reviews

Intermediate level

Some related experience required

1 week to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

What you'll learn

Configure batch processing with Databricks and Data Factory on Azure
Use ETL and ELT to load and transform data
Create linked services and identify pipelines for data stored within Data Factory
Explain Data Virtualization in PolyBase

Skills you'll gain

Tools you'll learn

Details to know

Shareable certificate

Add to your LinkedIn profile

Assessments

7 assignments

Taught in English

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

logos of Petrobras, TATA, Danone, Capgemini, P&G and L'Oreal

Build your subject-matter expertise

This course is part of the Microsoft Azure: AI, Infrastructure, and Data Solutions Specialization

When you enroll in this course, you'll also be enrolled in this Specialization.

Learn new concepts from industry experts
Gain a foundational understanding of a subject or tool
Develop job-relevant skills with hands-on projects
Earn a shareable career certificate

There are 8 modules in this course

This Azure training course is designed to equip students with the knowledge need to process, store and analyze data for making informed business decisions. Through this Azure course, the student will understand what big data is along with the importance of big data analytics, which will improve the students mathematical and programming skills. Students will learn the most effective method of using essential analytical tools such as Python, R, and Apache Spark.

This Azure training course is designed to equip the students with the knowledge need to process, store and analyze data for making informed business decisions. Through this Azure course, the student will understand what big data is along with the importance of big data analytics, which will improve the students mathematical and programming skills. Students will learn the most effective method of using essential analytical tools such as R, and Apache Spark.

What's included

1 video

One of the primary benefits of Azure Databricks is its ability to integrate with many other data environments to pull data through an ETL or ELT process. In module course, we examine each of the E, L, and T to learn how Azure Databricks can help ease us into a cloud solution.

What's included

5 videos3 readings1 assignment1 plugin

5 videosTotal 40 minutes

1.1 Batch Processing with Databricks and Data Factory in Azure8 minutes
1.2 - ELT Processing using Azure9 minutes
1.3 - Databricks and Azure Spark7 minutes
1.4 Transform Data using Databricks in ADF8 minutes
1.5 Use Case: ADF and Spark7 minutes

3 readingsTotal 50 minutes

Azure Databricks and Apache Spark10 minutes
Exercise 1 - Use Batch Processing with Databricks and Data Factory on Azure10 minutes
Exercise 2 - Intro to Databricks and Data Factory Page30 minutes

1 assignmentTotal 30 minutes

Module 1 Quiz30 minutes

1 pluginTotal 15 minutes

Microsoft Video: Cloud Computing Overview15 minutes

Processing big data in real-time is now an operational necessity for many businesses. Azure Stream Analytics is Microsoft’s serverless real-time analytics offering for complex event processing. In this section we examine how customers unlock valuable insights and gain competitive advantage by harnessing the power of big data.

What's included

4 videos3 readings1 assignment2 plugins

4 videosTotal 46 minutes

Pipelines and Activities - Introduction21 minutes
Processing using a Pipeline7 minutes
Analyzing Logs for an HDInsight Cluster10 minutes
Using Azure Blob Storage within HDInsight8 minutes

3 readingsTotal 35 minutes

Using Azure Blob Storage with HDInsight5 minutes
Exercise 1 - Pipeline Activities & Usage in Azure Data Factory15 minutes
Exercise 2 - Examine Logs within the HDInsight/Blob Storage15 minutes

1 assignmentTotal 30 minutes

Module 2 Quiz30 minutes

2 pluginsTotal 25 minutes

Microsoft Video: Azure SQL Data Warehouse15 minutes
2.2 Activity10 minutes

A data factory can have one or more pipelines. A pipeline is a logical grouping of activities that together perform a task. The activities in a pipeline define actions to perform on your data. Before you create a dataset, you must create a linked service to link your data store to the data factory. This section deals with linked services and data sets within Azure Blob Storage.

What's included

5 videos1 reading1 assignment2 plugins

5 videosTotal 38 minutes

Link Services and Datasets - Introduction14 minutes
Identifying Pipelines for a Data Factory10 minutes
Data Stores and Azure Blob Storage5 minutes
Linked Service and Connecting Data Factory to External Resources4 minutes
Processing Input Blobs with Azure Data Factory6 minutes

1 readingTotal 10 minutes

Exercise 1 - Link Data within Datasets in Azure Storage10 minutes

1 assignmentTotal 30 minutes

Module 3 Quiz30 minutes

2 pluginsTotal 25 minutes

3.2 Activity10 minutes
Video: Processing Input Blobs with Azure Data Factory15 minutes

Azure Data Factory is a fully managed, cloud-based data orchestration service that enables data movement and transformation. In this section, we explore scheduling triggers for Azure Data Factory to automate your pipeline execution.

What's included

5 videos1 reading1 assignment2 plugins

5 videosTotal 17 minutes

Schedules and Triggers - Introduction5 minutes
Creating a Trigger that Runs a Pipeline on a Schedule2 minutes
Scheduling a Trigger in Azure Data Factory4 minutes
Pipeline Execution and Triggers in ADF3 minutes
Use Case: Azure Schedule, Trigger, and Events3 minutes

1 readingTotal 5 minutes

Pipeline Execution and Triggers in ADF5 minutes

1 assignmentTotal 30 minutes

Module 4 Quiz30 minutes

2 pluginsTotal 30 minutes

Video: Azure Data Factory Event Triggers15 minutes
Module 4 Exercise15 minutes

In time-streaming scenarios, performing operations on the data contained in temporal windows is a common pattern. Stream Analytics has native support for windowing functions, enabling developers to author complex stream processing jobs with minimal effort. In this section, we study windowing functions related to in-stream analytics.

What's included

5 videos1 reading1 assignment3 plugins

5 videosTotal 30 minutes

Selecting Windowing Functions - Introduction5 minutes
How Stream Analytics Support Native Windowing Functions 10 minutes
Temporal Windows6 minutes
Using Window Functions in the GROUP BY Clause3 minutes
Aggregating Events over Multiple Windows using WindowsQ6 minutes

1 readingTotal 10 minutes

Understanding Stream Analytics Windowing Functions10 minutes

1 assignmentTotal 30 minutes

Module 5 Quiz30 minutes

3 pluginsTotal 45 minutes

Video: Window Functions for Analysis15 minutes
5.5 Activity10 minutes
Module 5 Exercise20 minutes

This section teaches how to analyze phone call data using Azure Stream Analytics. The phone call data, generated by a client application, contains some fraudulent calls, which will be filtered by the Stream Analytics job.

What's included

7 videos1 reading1 assignment3 plugins

7 videosTotal 33 minutes

How Stream Analytics Relate to Data Solutions3 minutes
Generate Sample Call Data and Send it to Event Hubs10 minutes
Creating a Stream Analytics Job 7 minutes
Configuring Job Input and Output 4 minutes
Define a Query to Filter Fraudulent Calls 2 minutes
Test and Start the Job4 minutes
Visualize Results in Power BI3 minutes

1 readingTotal 10 minutes

Output Real-Time Stream Analytics Data to a Power BI Dashboard10 minutes

1 assignmentTotal 30 minutes

Module 6 Quiz30 minutes

3 pluginsTotal 35 minutes

Video: Create a Stream Analytics Job15 minutes
6.3 Activity10 minutes
6.7 Activity10 minutes

Traditional SMP data warehouses use an Extract, Transform and Load (ETL) process for loading data. Azure SQL Data Warehouse is a massively parallel processing (MPP) architecture that takes advantage of the scalability and flexibility of compute and storage resources. Utilizing an Extract, Load, and Transform (ELT) process can take advantage of MPP and eliminate resources needed to transform the data prior to loading.