In the previous video you heard about how GAN have improved over the past few years. In this video you'll learn about StyleGAN relatively new architecture that's considered to not only be the state of the art GAN right now, but an inflection point improvement in GAN, particularly in its ability to generate extremely realistic human faces. I'll start by going over StyleGAN, primary goals and show you how it's become the embodiment of the improvements in gains you just heard about. Then I'll talk about what the style in StyleGAN means, and finally, you'll get an introduction to its architecture in individual components. So one of the first goals of StyleGAN is to produce high quality, high resolution images that are capable of pulling a casual onlooker. So you and me, perhaps. [LAUGH] The 2nd is a greater diversity of images in the output. So if you're still thinking about the cute little dog pupper examples in the earlier videos, the output would probably range from a golden tree bird to a golden doodle to a French poodle instead of just giving you a range of slightly different golden retrievers with the same background. One of the really cool things about StyleGAN is also the increased control over image features. And this can be adding features like hats or sunglasses or mixing styles from two different generated images together. So in a sense you could really see what Brad Pitt and Angelina Jolie would make in Brangelina. Getting high resolution fidelity that fools the untrained eye is a huge feat that up until recently, was much more difficult to achieve. And this was mainly due to smaller model capacity, lower resolution datasets, and also the challenge of high resolution just really wasn't tackled until ProGAN, which is StyleGAN predecessor that was launched in 2018. So here you see these generated faces from 2014. Look for the most part like bad sketches there, greenie an unlikely to convince anyone of their authenticity. Now look at this more recent high resolution face generated by StyleGAN, if you saw this image without any context, would you be able to guess that she wasn't a real person? It would seem pretty apparent that's StyleGAN has achieved its goal of greater fidelity, but on an interesting side note, StyleGAN has tried both W GAN. Last from week three that you learned about, and the original GAN laws in week one that you learned about and found that each work better on a different data set. And both datasets were high resolution faces, so the jury is still out, but really both probably work, and so the takeaway is that it's all about experimenting. Finally, StyleGAN also wants to increase the control you have over your image feature. And this can be mixing the style of 1 image into another like you can see here where this face here is a mix from the below and to the right where you can see hair color and styles coming from the picture from the right and various facial features coming from the face on the bottom. And then you see here, on the bottom right hand corner this woman is a mixing of styles from the woman above, as well as styles from the boy on the left. Control could also mean adding accessories such as glasses and style. StyleGAN accomplishes this by disentangling the latent space which you'll be learning about in more detail here shortly. So as you can see, all these dudes got some sunglasses. The word style in the context of image generation is pretty much any variation in the image. And you can think of these variations as generally representing the look and feel of different levels in the image. And these different levels could mean larger, coarser styles such as general face shape or facial structure to finer, more detailed styles which are like color their hair or placement of certain wisps of hair. The StyleGAN generator, Interestingly enough, is made up of blocks where earlier blocks roughly aligned with coarser features, like facial structure or pose. So the earlier layers here and nearer to the output, finer details styles such as hair color, eyebrow shape, are controlled. So you can imagine this woman here could be composited or change in some ways based on core styles from the woman on top medium styles from the woman in the middle and finer grain styles from the woman on the bottom here. Now that you're more familiar with the concept of styles, let's take a gander at how StyleGAN generator differs from a traditional GAN generator that you might be more familiar with. In a traditional GAN generator, you taken this noise vector into the generator and the generator then outputs an image. Now in StyleGAN, the noise vector is handled a little bit differently, so instead of feeding in the noise vector directly into the generator, it goes through a mapping network to get an intermediate noise vector W, say. That then gets injected into the StyleGAN generator, actually multiple times to produce this image. And also there's this extra random noise that's passed in to add various stochastic variation onto this image. And that could just be moving this wisp of hair in different ways. So smaller variation that is random noise could then help produce and for this random noise there's no learning aspect of It is largely just uncorrelated random gaussian noise. Where you perturb the values coming out of convolutions, various layers a little bit. However, this mapping network, which you'll see shortly, is very important and consists of learnable parameters as well. So backdrop will go all the way from the discriminator, back here through the generator and through this mapping network. And something that's important to know is that W is not just imputed easily into the StyleGAN generator. Instead, styles are extracted from the W value. This intermediate noise value and then added to various points in the StyleGAN generator and an earlier points of the StyleGAN, generator W will inform course or styles that you saw previously. Such as general face shape and injecting W into layer layers will control styles of finer grained things such as hair color. And this injection of this intermediate noise into all of these layers of StyleGAN is done through an operation called at a in or adaptive instance normalization. And this is the type of normalization similar to batch normalization, except that after it normalizes in some way, it tries to apply some kind of style which is just statistics of an image based on this W coming in. The 3rd and final important component of StyleGAN is progressive growing. Which slowly grows the image resolution being generated by the generator and evaluated by the discriminator over the process of training. And progressive growing originated with program organ with progressive growing. And it's not unique to StyleGAN, but it is its predecessor and the others didn't know that it did help with training on higher resolution images. So during the training process, both the generator and discriminator start with a small low resolution image. And the goal servers trained the generator to be able to generate something easier than a high resolution face, as in this small blurry image that's in the right ballpark. You know there's still some face pixels going on. And when the models are stable, they can then beat sanded to double the height and width. So it's slightly harder task now, slightly less blurry, needs to look a little more face like, but still easier than a super high res face. And this continues until the desired image resolution is reached. In this doubling occurs at scheduled times during the training process, but let's tricky is not. The doubling can't be too abrupt, but must be more gradual in order to ease the generator into generating those larger images. So if you're feeling confused, don't worry at all. This too will be covered in more detail a bit later in the course. The main takeaway is that you gotta go slow if you want to grow. So now you're acquainted with StyleGAN, and some of its amazing achievements. which include greater fidelity on higher resolution images, increase diversity in their outputs, and more control over image features like hair color accessories. You saw a bit about what style means in the context of image generation, which is to say the general texture or look and feel of different levels of the image. And these vary depending on their location in the generator. From large core styles like the shape of a face to finer styles like hair color. Finally you were introduced to the StyleGAN architecture, and you've seen how its main components that noise mapping network, adaptive instance normalization set it apart from a more traditional GAN. Progressive growing isn't limited to use in StyleGAN, but it does help with more stable training of higher resolution images. And this was just a high level introduction you're about to get a deeper dive into each of these components and how they work next.
