Linux and Bash for Data Engineering

Linux and Bash for Data Engineering

This course is part of Python, Bash and SQL Essentials for Data Engineering Specialization

Instructors: Noah Gift

Access provided by UNext Manipal

12,032 already enrolled

4 modules

Gain insight into a topic and learn the fundamentals.

133 reviews

Intermediate level

Recommended experience

7 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

4 modules

Gain insight into a topic and learn the fundamentals.

133 reviews

Intermediate level

Recommended experience

7 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

What you'll learn

Use Linux tools build data engineering solutions.
Develop Bash syntax to configure and control Linux.

Skills you'll gain

Tools you'll learn

Details to know

Shareable certificate

Add to your LinkedIn profile

Assessments

16 assignments

Taught in English

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

logos of Petrobras, TATA, Danone, Capgemini, P&G and L'Oreal

Build your subject-matter expertise

This course is part of the Python, Bash and SQL Essentials for Data Engineering Specialization

When you enroll in this course, you'll also be enrolled in this Specialization.

Learn new concepts from industry experts
Gain a foundational understanding of a subject or tool
Develop job-relevant skills with hands-on projects
Earn a shareable career certificate

There are 4 modules in this course

In this second course of the Python, Bash and SQL Essentials for Data Engineering Specialization, you will learn the fundamentals of Linux necessary to perform data engineering tasks. Additionally, you will explore how to use both Bash and zsh configurations, and develop the syntax needed to interact and control Linux. These skills will allow you to manage and manipulate databases in a Bash environment.

In this module, you will learn to use Linux to solve technical problems common in software engineering, data engineering and machine learning. You will explore the key parts of the Linux shell, and learn how to implement your knowledge in common scenarios.

What's included

18 videos14 readings4 assignments1 discussion prompt5 ungraded labs

18 videosTotal 99 minutes

Welcome to Linux and Bash for Data Engineering1 minute
Meet your Course Instructor: Noah Gift1 minute
Overview of Key Concepts4 minutes
Introduction to Working with Linux1 minute
Introduction to the Linux Shell2 minutes
Installing and Accessing Linux11 minutes
Getting Started with Github Codespaces9 minutes
Comparing Linux Shell Environments10 minutes
Writing 'Hello World' Bash Script6 minutes
Popular Terminal Commands in Linux15 minutes
Introduction to Shell Piping1 minute
What is Shell Piping?4 minutes
Walkthrough of Shell Pipeline Examples12 minutes
Introduction to SSH2 minutes
What is SSH?5 minutes
Creating SSH Keys and using with GitHub5 minutes
Using SSH to Connect to Remote Server in AWS Cloud7 minutes
Walkthrough of SSH Tunneling Example4 minutes

14 readingsTotal 135 minutes

Key Terms10 minutes
Meet your Supporting Instructors: Kennedy Behrman and Alfredo Deza10 minutes
Course Structure and Discussion Etiquette10 minutes
Report a problem with the course5 minutes
Key Terms10 minutes
Amazon CodeWhisperer CLI10 minutes
Lesson Reflection10 minutes
Key Terms10 minutes
Shell Pipelines10 minutes
Lesson Reflection10 minutes
Key Terms10 minutes
Securely Connecting to a Linux Instance with SSH10 minutes
Secure Remote Development with VSCode and SSH10 minutes
Lesson Reflection10 minutes

4 assignmentsTotal 570 minutes

Using Linux30 minutes
Quiz-Using Linux180 minutes
Quiz-How Shell Piping Works180 minutes
Quiz-Using SSH180 minutes

1 discussion promptTotal 10 minutes

Meet and Greet (optional)10 minutes

5 ungraded labsTotal 300 minutes

Practicing Common Linux Shell Commands60 minutes
Using Linux Lab Challenge60 minutes
Practicing Common Shell Piping Operations60 minutes
Creating SSH Keys Locally60 minutes
Creating a Shell Pipeline60 minutes

In this module, you will learn to configure Bash Scripts to enhance and control your Linux development environment and production systems. You will also learn about Shell variables, and how to effectively use Standard In and Standard Out.

What's included

14 videos6 readings4 assignments3 ungraded labs

14 videosTotal 65 minutes

Introduction to Working with Bash1 minute
Introduction to Configuring your Bash Shell Environment2 minutes
What are Shell Configuration Files?6 minutes
Configuring .bashrc12 minutes
Configuring .zshrc and Third-Party Tool oh-my-zsh10 minutes
Introduction to Working with Shell Variables1 minute
What are Shell Variables?4 minutes
Exporting Shell Variables in an Interactive Prompt7 minutes
Using Shell Variables in a Bash Script7 minutes
Introduction to Standard Streams1 minute
What are Standard Streams?2 minutes
Using Standard Out6 minutes
Using Standard In2 minutes
Using Standard Error3 minutes

6 readingsTotal 60 minutes

Key Terms10 minutes
Lesson Reflection10 minutes
Key Terms10 minutes
Lesson Reflection10 minutes
Key Terms10 minutes
Lesson Reflection10 minutes

4 assignmentsTotal 570 minutes

Using Bash30 minutes
Quiz-Configuring the Shell Environment180 minutes
Quiz-Working with Shell Variables180 minutes
Quiz-Standard Streams: Using Standard In, Standard Out, Standard Error180 minutes

3 ungraded labsTotal 180 minutes

Configuring the Bash Shell60 minutes
Sourcing Shell Variables from a Script60 minutes
Practicing with Bash Streams60 minutes

In this module, you will learn to build Bash Scripts to solve common problems in Data Engineering. You will also learn about Shell logic, dealing with data in the shell, and how to build command-line tools in Bash. You will then apply this knowledge to create command-line tools in Bash.

What's included

18 videos6 readings4 assignments4 ungraded labs

18 videosTotal 63 minutes

Introduction to Building Bash Scripts1 minute
Introduction to Shell Logic and Control Flow1 minute
What is Shell Control Flow?3 minutes
Using Shell Control Flow to Solve Problems in Bash2 minutes
Using Shell Loops in Bash7 minutes
Evaluating Conditions in Bash5 minutes
Introduction to Manipulating Data in the Bash Shell1 minute
Shell Techniques for Data in the Bash Shell3 minutes
Truncating Data in Bash3 minutes
Filtering Data in Bash4 minutes
Searching Data in Bash4 minutes
Introduction to Writing Scripts and Command-Line Tools in Bash1 minute
What are Scripts and Command-Line Tools?3 minutes
Building a Bash Script4 minutes
Building a Bash Function6 minutes
Building a Bash Command-Line Tool3 minutes
Bash-Adjacent Tools: Makefiles and Dockerfiles8 minutes
Data Structures in Bash4 minutes

6 readingsTotal 60 minutes

Key Terms10 minutes
Lesson Reflection10 minutes
Key Terms10 minutes
Lesson Reflection10 minutes
Key Terms10 minutes
Lesson Reflection10 minutes

4 assignmentsTotal 570 minutes

Building Bash Scripts30 minutes
Quiz-Using Shell Logic and Control Flow180 minutes
Quiz-Manipulating Data with the Shell180 minutes
Quiz-Writing Bash Scripts and Command-Line Tools180 minutes

4 ungraded labsTotal 240 minutes

Writing Loops in Bash60 minutes
Truncating Large Files with Bash60 minutes
Building a Command-Line Tool for Data Processing60 minutes
Building a Bash CLI with Options60 minutes

In this module, you will learn to use Linux to compose file and data management solutions. You will also learn to search the filesystem, modify files, directories and control permissions. You will then process text in Bash and apply this knowledge to create a search solution in Bash.

What's included

17 videos8 readings4 assignments5 ungraded labs

17 videosTotal 62 minutes

Introduction to Composing File and Data Management Solutions in Linux0 minutes
Introduction to Searching a Filesystem in Linux1 minute
Methods for Searching a Filesystem in Linux4 minutes
Using the Locate Command to Find Files in Linux3 minutes
Using the Find Command to Find Files in Linux3 minutes
Using xargs to Extend Search Capabilities in Linux3 minutes
Using mdfind on OS X6 minutes
Introduction to Modifying Files, Directories, Permissions and Archiving in Linux1 minute
Overview of Modifying the Filesystem in Linux3 minutes
Moving Files and Directories in Linux7 minutes
Setting Permissions on Files and Directories in Linux5 minutes
Archiving Data in Linux3 minutes
Introduction to Processing Text in Linux1 minute
Benefits of Processing Text in Linux2 minutes
Using grep, cut, sort and unique in Linux8 minutes
Editing with Truncation, awk and sed in Linux7 minutes
Using Regular Expressions (regex) in Linux6 minutes