There are a lot of AI and machine learning techniques today. And while supervised learning, that is learning A to B mappings, is the most valuable one, at least economically today, there are many other techniques that are worth knowing about. Let's take a look. The best-known example of unsupervised learning is clustering, here's an example. Let's say you run a grocery store that specializes in selling potato chips. And you collect data on different customers, and keep track of how many different packets of potato chips a single customer buys, as well as what's the average price per package that person paid for their potato chips. So you sell some low end, cheaper potato chips, as well as some high end, more expensive packets of potato chips. And different people may buy different numbers of potato chip packets, in a typical trip to your grocery store. Given data like this, a clustering algorithm will say that it looks like you have two clusters in your data. Some of your customers tend to buy relatively inexpensive potato chips, but buy a lot of packets. If your grocery store is near a college campus, for example, you may find a lot of college students that are buying cheaper potato chips packets, but they sure buy a lot of them. And there's a second cluster in this data of a different group of shoppers that buy fewer packets of potato chips, but buy more expensive packets. A clustering algorithm looks at data like this, and automatically groups the data into two clusters or more clusters, and is commonly used for market segmentation. And will help you discover things like that if you have a college student audience that buys a certain type of potato chips, and a working professional audience that buys fewer potato chips but is willing to pay more. And this can help you market differently to these market segments. The reason this is called unsupervised learning is the following. Whereas supervised learning algorithms run an A to B mapping, and you have to tell the algorithm what is the output B that you want, an unsupervised learning algorithm doesn't tell the AI system exactly what it wants. Instead it gives the AI system a bunch of data such as this customer data, and it tells the AI to find something interesting in the data, find something meaningful in the data. And in this case, a clustering algorithm doesn't know in advance that there's a college student demographic and a working professional demographic. Instead, it just tries to find what are the different market segments without being told in advance what they are. So unsupervised learning algorithms, given data without any specific design output labels, without the target label B, can automatically find something interesting about the data. One example of unsupervised learning that I have worked on was the slightly infamous Google cat. In this project, I had my team run a unsupervised learning algorithm on a very large set of YouTube videos, and we asked the algorithm, tell us what you find in YouTube videos. And one of the many things that are found in YouTube videos was cats, because somewhat stereotypically, YouTube apparently has a lot of cat videos. But it was a remarkable result that without telling it in advance that it should find cats, the AI system, the unsupervised learning algorithm, was able to discover the concept of a cat all by itself. Just by watching a lot of YouTube videos and discovering that, boy, there are a lot of cats in YouTube videos. It's hard to visualize exactly what an AI algorithm is thinking sometimes, but this picture on the right is a visualization of the concept of the cat that the system had learned. Even though supervised learning is an incredibly valuable and powerful technique, one of the criticisms of supervised learning is that it just needs so much labeled data. For example, if you're trying to use supervise learning to get the AI system to recognized coffee mugs, then you may give it 1000 pictures of coffee mugs, or 10,000 pictures of coffee mugs. And that's just a lot of picture of coffee mugs we will be giving our AI systems. For those of you that are parents, I can almost guarantee to you that no parent on this planet, no matter how loving and caring, has ever pointed out 10,000 unique coffee mugs to their children, to try to teach the children what is a coffee mug. So, AI systems today require much more labeled data to learn than when a human child or than would most animals. Which is why AI researchers hold a lot of hope out for unsupervised learning as way, maybe in the future, for AI to learn much more effectively in a more human like way, and more biological-like way for much less labeled data. Now, we have pretty much no idea how the biological brain works, and so to realize this vision, we'll take major breakthroughs in AI than none of us know yet today how to realize. But many of us hold a lot of hope for the future of unsupervised learning. Having said that, unsupervised learning is valuable today. There are some specific applications and natural language processing, for example, where unsupervised learning helps the quality of web search quite a bit, for example. But the value today of unsupervised learning is so a lot smaller than the value created through supervised learning. Another important AI technique is transfer learning. Let's look an example. Let's say you bought a self-driving car, and you've trained your AI system to detect cars. But you didn't deploy your vehicle to a new city and somehow this new city has a lot of golf carts travelling round, and so you need to also build a golf cart detection system. You may have trained your car detection system with a lot of images, say 100,000 images, but in this new city where you just start operating, you may have a much smaller number of images of golf carts. Transfer learning is the technology that lets you from a task A, such as car detection, and use the knowledge to help you on a different task B, such as golf cart detection. Where transfer learning really shines is if having learn from a very large dataset of car detection, task A, you can now do pretty well on golf cart detection, even though you have a much smaller golf cart dataset. Because some of the knowledge it has learned from the first task, of what the vehicles look like, what the wheels look like, how the vehicles move. Maybe that will be useful also for golf cart detection. Transfer learning doesn't get a lot of press, but it is one of the very valuable techniques in AI today. And for example, many computer vision systems are built using transfer learning, and this makes a big difference to their performance. You may also have heard of a technique called reinforcement learning. So, what is reinforcement learning? Let me illustrate with another example. This is a picture of the Stanford autonomous helicopter. So it's instrumented with GPS, accelerometers, and a compass, so it always knows where it is. And let's say you want to write a program to make it fly by itself. It's hard to used supervised learning, input/output, A to B mliings, because it's very difficult to specify what is the optimal way. What is the best way to fly the helicopter when it is in a certain, given position. Reinforcement learning offers a different solution. I think of reinforcement learning as similar to how you might train a pet dog to behave. My family, when I was growing up, had a pet dog. So, how do you train a dog to behave itself? Well, we let the dog do whatever it wanted to do, and then whenever it behaved well we would praise it. You go, good dog, and whenever it does something bad you would go, bad dog. And over time it learns to do more of the 'good dog' things, and fear of the bad dog things. Reinforcement learning takes the same principle and applies it to a helicopter or two other things. So, we would have the helicopter flying around in a simulator so it could crash without hurting anyone. But we would let the AI fly the helicopter however it wants, and whenever it flew the helicopter well, we will go, good helicopter. And when if it crashes, we go bad helicopter. And then there was the AI's job to learn how to fly the helicopter to get more of the good helicopter rewards, and fewer of the bad helicopter negative rewards. More formally, a reinforcement learning algorithm uses a reward signal, to tell the AI when it's doing well or poorly. This means that whenever it's doing well, you give it a large positive number to give it a large positive reward. And whenever it's doing really bad job, you send it a negative number to give it a negative reward. And it's the AI's job to then automatically learn to behave so as to maximize the rewards. So, good dog responds to giving a positive number, and bad dog or bad helicopter, corresponds to you giving a negative number. And that the AI will learn to get more of the behaviors that results in large positive numbers, or in large positive rewards. Let me show you a video of the Stanford autonomous helicopter after we did this. This is a video of the helicopter flying under reinforcements learning control. I was the cameraman that day, and when you zoom out the camera, you see the trees pointing to the sky. So, we actually gave it a reward signal which rewarded the helicopter flying upside down. And using reinforcement learning, we built one of the most capable autonomous helicopters in the world. In addition to robotic control, reinforcement learning has also had a lot of traction in playing games, games such as Othello or checkers or chess or Go. You might have heard of AlphaGo, which did a very good job of playing Go using reinforcement learning. And reinforcement learning has also been very effective at playing video games. One of the weaknesses of reinforcement learning algorithms is that they can require a huge amount of data. So, if you are playing a video game, a reinforcement learning algorithm can play, essentially, an infinite number of video games. Because it's just a computer playing a computer game, and get a huge amount of data to learn how to behave better. Or for playing games like checkers, or other games. It can play a lot of games against itself, and for free, get a huge amount of data to feed in its reinforcement learning algorithm. In the case of the autonomous helicopter, we had simulator for the helicopter, so it could fly in simulation for a long time to figure out what works and what doesn't work for flying a helicopter. There so a lot of exciting research work being done to make reinforcement learning work, even for settings where you may not have an accurate simulator. Whereas harder to get this huge amount of data. Despite the huge amount of media attention on reinforcement learning, at least today it is creating significantly less economic value than supervised learning. But there may be breakthroughs in the future that could change that. And AI is advancing so rapidly that all of us certainly hope that there will be breakthroughs in all of these areas that we're talking about. GANs, or generative adversarial networks, are another exciting new AI technique. They were created by my former student Ian Goodfellow. GANs are very good at synthesizing new images from scratch. Let me show you a video generated by a team from NVIDIA, that used GANs to synthesize pictures of celebrities. And these are all pictures of people that had never existed before. But by learning what's celebrities look like from a databases celebrity images, it's able to synthesize all these brand-new pictures. There's exciting work by different things right now on applying GANs to the entertainment industry. Everything ranging from computer graphics to computer games, to media, and to just making up new contents like this from scratch. Finally, the knowledge graph is another important AI technique that I think it's very underrated. If you do a search on Google for Leonardo da Vinci, you might find this set of results with this panel on the right of information about da Vinci. If you do a search on Ada Lovelace, you'll also similarly find a panel of additional information on the right. This information is drawn from a Knowledge Graph, which basically means a database that lists people and key information about these people. Such as their birthday, when they pass away, their bio, and other properties of these individuals. Today different companies have built knowledge graphs of many different types of things not just people. But also they built these databases of movies, of celebrities, of hotels, of airports, of scenic attractions, and on and on and on. For example, a Knowledge Graph with hotel information may have a big database of hotels as well as key information about these hotels. So that if you look them up on the map you can find the right information relatively quickly. The term Knowledge Graph was initially popularized by Google, but this concept has spread to many other companies. Interestingly, even though Knowledge Graphs are creating a lot of economic value for multiple large companies at this point, this is one subject that is relatively little studied in academia. And so, the number of research papers you see on Knowledge Graphs seems to be disproportionately small, relative to the actual economic impact today. But, depending on what industry vertical you work in, perhaps some of the techniques for building Knowledge Graphs will also be useful for building a big database of information about something relevant to your company. In this video, you learned about unsupervised learning, transfer learning, reinforcement learning, GANs, and knowledge graphs. It seems like a lot, doesn't it? I hope that some of these ideas will also be useful to your projects, and that knowing what these algorithms are, will make it easier for you to fruitful discussions with AI engineers. In this week we've talk a lot about how AI can affect companies. Maybe how you could use AI to affect your company. AI is also having a huge impact on society. So, how can we understand the impact that AI is having on society, as well as make sure that we do ethical things? And that we use AI only to help people and make people better off? And next week we'll talk about AI and society. Thanks for sticking with me up to this point, I look forward to seeing you in the final week of videos for this course.
