Explore
For Enterprise
Join for Free
Log In

Browse
Search
For Enterprise
Log In
Join for Free

Looking into the details

Loading...

Natural Language Processing in TensorFlow

deeplearning.ai

4.6 (819 ratings) | 13K Students Enrolled

Course 3 of 4 in the TensorFlow in Practice Specialization

If you are a software developer who wants to build scalable AI-powered algorithms, you need to understand how to use the tools to build them. This Specialization will teach you best practices for using TensorFlow, a popular open-source framework for machine learning. In Course 3 of the deeplearning.ai TensorFlow Specialization, you will build natural language processing systems using TensorFlow. You will learn to process text, including tokenizing and representing sentences as vectors, so that they can be input to a neural network. You’ll also learn to apply RNNs, GRUs, and LSTMs in TensorFlow. Finally, you’ll get to train an LSTM on existing text to create original poetry! The Machine Learning course and Deep Learning Specialization from Andrew Ng teach the most important and foundational principles of Machine Learning and Deep Learning. This new deeplearning.ai TensorFlow Specialization teaches you how to use TensorFlow to implement those principles so that you can start building and applying scalable models to real-world problems. To develop a deeper understanding of how neural networks work, we recommend that you take the Deep Learning Specialization.

Skills You'll Learn

Natural Language Processing, Tokenization, Machine Learning, Tensorflow, RNNs

Reviews

4.6 (819 ratings)

5 stars
595 ratings
4 stars
166 ratings
3 stars
40 ratings
2 stars
12 ratings
1 star
6 ratings

SS

Aug 06, 2019

This course covers the overview of NLP without going into much mathematical detail. In short time span, many things can be learnt from this course, helpful for the beginners.

GI

Jun 22, 2019

Amazing course by Laurence Moroney. But only after finishing Sequence Models by Andrew NG, I was able to understand the concepts taught here.

From the lesson

Word Embeddings

Last week you saw how to use the Tokenizer to prepare your text to be used by a neural network by converting words into numeric tokens, and sequencing sentences from these tokens. This week you'll learn about Embeddings, where these tokens are mapped as vectors in a high dimension space. With Embeddings and labelled examples, these vectors can then be tuned so that words with similar meaning will have a similar direction in the vector space. This will begin the process of training a neural network to udnerstand sentiment in text -- and you'll begin by looking at movie reviews, training a neural network on texts that are labelled 'positive' or 'negative' and determining which words in a sentence drive those meanings.

A conversation with Andrew Ng2:08

Introduction2:14

The IMBD dataset1:03

Looking into the details4:01

How can we use vectors?2:30

More into the details2:47

Notebook for lesson 110:09

Remember the sarcasm dataset?1:07

Building a classifier for the sarcasm dataset1:55

Let’s talk about the loss function1:26

Pre-tokenized datasets0:43

Diving into the code (part 1)1:25

Diving into the code (part 2)2:30

Notebook for lesson 35:10

Taught By

Laurence Moroney
AI Advocate

Transcript

So let's start looking at it. There are a couple of things that you need to take into account before you start working with this week's code in TensorFlow. The first is the version of TensorFlow you're using. Use this code to determine it. Also, do note that all the code I'm using here is in Python 3. There are some differences if you use Python 2. So if you're using a Colab, you can set the environment to three. If you're doing this in your own environment, you may need to make some changes. If the previous code gave you TensorFlow 1.x, you'll need this line of code before you can go any further. If it gave you 2.x, then you won't need anything because eager execution is enabled by default in TensorFlow 2.0. If you're using Google Colab, then you should have TensorFlow datasets already installed. Should you not have them, they're easily installed with this line of code. Now, you can import TensorFlow datasets, and in this case I call them tfds. With imdb reviews, I can now call tfds.load, pass it the string imdb reviews, and it will return the data from imdb, and metadata about it with this code. The data is split into 25,000 samples for training and 25,000 samples for testing. I can split them out like this. Each of these are iterables containing the 25,000 respective sentences and labels as tensors. Up to this point, we've been using the Cara's tokenizers and padding tools on arrays of sentences, so we need to do a little converting. We'll do it like this. First of all, let's define the lists containing the sentences and labels for both training and testing data. Now, I can iterate over training data extracting the sentences and the labels. The values for S and I are tensors, so by calling their NumPy method, I'll actually extract their value. Then I'll do the same for the test set. Here's an example of a review. I've truncated it to fit it on this slide, but you can see how it is stored as a tf.tensor. Similarly, here's a bunch of labels also stored as tensors. The value 1 indicates a positive review and zero a negative one. When training, my labels are expected to be NumPy arrays. So I'll turn the list of labels that I've just created into NumPy arrays with this code. Next up, we'll tokenize our sentences. Here's the code. I've put the hyperparameters at the top like this for the reason that it makes it easier to change and edit them, instead of phishing through function sequences for the literals and then changing those. Now, as before, we import the tokenizer and the pad sequences. We'll create an instance of tokenizer, giving it our vocab size and our desired out of vocabulary token. We'll now fit the tokenizer on our training set of data. Once we have our word index, we can now replace the strings containing the words with the token value we created for them. This will be the list called sequences. As before, the sentences will have variant length. So we'll pad and or truncate the sequenced sentences until they're all the same length, determined by the maxlength parameter. Then we'll do the same for the testing sequences. Note that the word index is words that are derived from the training set, so you should expect to see a lot more out of vocabulary tokens in the test exam. Now it's time to define our neural network. This should look very familiar by now, except for maybe this line, the embedding. This is the key to text sentiment analysis in TensorFlow, and this is where the magic really happens.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

Coursera

About
Leadership
Careers
Catalog
Certificates
MasterTrack™ Certificates
Degrees
For Enterprise
For Government

Community

Learners
Partners
Developers
Beta Testers
Translators
Blog
Tech Blog

More

Terms
Privacy
Help
Accessibility
Press
Contact
Directory
Affiliates

© 2019 Coursera Inc. All rights reserved.