Datastream MySQL to BigQuery

In this Project, you will:

Prepare a MySQL Cloud SQL instance using the Google Cloud Console

Create a GCS bucket to be used in replication

Create a Pub/Sub topic, subscription, and a GCS Pub/Sub notification policy

1 hour 30 minutes
Beginner
No download needed
Shareable certificate
English
Desktop only

This is a self-paced lab that takes place in the Google Cloud console. Learn to migrate MySQL Databases to BigQuery using Datastream and Dataflow. Datastream is a serverless and easy-to-use Change Data Capture (CDC) and replication service that allows you to synchronize data across heterogeneous databases, storage systems, and applications reliably and with minimal latency. In this lab you'll learn how to replicate data from your OLTP workloads into BigQuery, in real time. You will begin by deploying MySQL on Cloud SQL and import a dataset using the gcloud command line. Then, in the Cloud Console UI, you will create and start a Datastream stream and a Dataflow job for replication. The replication uses a Dataflow template to enable continuous replication of data, along with Cloud Storage and Pub/Sub for buffering data.

Skills you will develop

  • Dataflow

  • Google Cloud Platform

  • Bigquery

  • CloudSQL

  • MySQL

How Projects work

Learn a new tool or skill in an interactive, hands-on environment

You'll gain access to software and tools in a cloud workspace - no download required

Offered by

Placeholder

Google Cloud

Frequently Asked Questions

In Projects, you'll complete an activity or scenario by following a set of instructions in an interactive hands-on environment. Projects are completed in a real cloud environment and within real instances of various products as opposed to a simulation or demo environment.

By purchasing a Project, you'll get everything you need to complete the Project including temporary access to any product required to complete the Project.

Even though Projects are technically available on mobile devices, we highly recommend that you complete Projects on a laptop or desktop only.

Yes, you can download and keep any of your created files from the Project. To do so, please make sure you save any files and work to your device before exiting the product environment.

Projects are not eligible for refunds. See our full refund policy.

Financial aid is not available for Projects.

In rare instances, Projects may be taken down for maintenance or other reasons. If you are experiencing any issues, please contact us.

Auditing is not available for Projects.

At the top of the page, you can view the experience level recommended for this Project.

Yes, everything you need to complete your Project will be available in your browser.