Previously you learned about how GANs work, mainly focusing on the positives, like their amazing empirical results. Now you'll see some of the shortcomings of GANs as well, because that's equally important when you learn about any new technique. Remember that a significant advantage with GANs is that they can produce high-quality, realistic results. To the human eye like yours and mine, you could be fooled into believing these people actually exist, but these are all generated. Another pro is that once you have a trained model, you can generate objects fairly quickly. You might recall seeing this in your assignment. All you need to do is load the weights of the model and then pass in some noise. However, GANs also have their disadvantages. First, they lack concrete theoretically grounded intrinsic evaluation metrics. How do you measure their performance? You can't just look at the model weights or outputs and easily say, "This is the best model. This model is better than that one." In order to evaluate your GAN, you might remember that you usually need to inspect the features across many generated samples and compare them to those of the real images, and even that technique isn't that reliable. It's an approximate estimate of what you would ideally want for your evaluation. Another downside is that, during training, the model can be unstable and take considerable amount of time to train. Sometimes it feels like more of an art than a science because gradient descent doesn't always get you a generator you need. For example, like mode collapse, you see here of producing all sevens when the generator gets stuck, you can't just keep training and expect that your GAN will converge. You need to babysit it and check in a lot to see when to stop training, and you need to visually inspect those samples qualitatively. At the same time, you've also seen this problem being remedied with W loss a bit and one Lipschitz continuity. While this is an issue, it's not necessarily a huge one anymore, so maybe we can cross it out like that, though it definitely was a disadvantage of GANs in their early days. Depending on what you want to use your generative model for, GANs might not be the right type of model if you want to explicitly get the probability density over your modeled features, and what that means is, the likelihood of say, a particular image here. How likely are these features to present themselves? This might be useful if you want to say do anomaly detection, by seeing what an unlikely dog would look like versus a likely dog. Perhaps this is not likely, or it could detect cat dogs that are very unlikely out there. This is known as density estimation because it's estimating this probability density of all these features. Density estimation is useful to know how often this golden fur or floppy ears, for example, typically make up a dog, and that can then feed into downstream tasks like finding anomalies out where there's low probability for certain features. Over lots of samples, you could of course get some approximation for your GAN. Lastly, the generator is not trained to be invertible. What that means is that you can take an image such as this one, and be able to figure out what noise vector it maps onto, so the opposite task. Instead of the usual task of inputting a noise factor and then outputting an image. Now you want to feed in an image to figure out what its associated noise vector is. There have been new methods that have emerged to remedy this problem of invertibility, typically with another model that does the opposite of the GAN, and there are also GANs that are designed to learn both directions at once. One GAN going in one direction and the other one going in the other. You might be wondering why inversion can be useful, and inversion can be particularly convenient for image editing because that means you can apply your controllable generation skills to that noise vector that you find for any image, and this could be a real image. It doesn't have to be generated already to find that noise vector. Then you tweak that noise vector using those controllable generation skills that you have now, so that this image could be, for example, younger, older, or have blue hair. Wouldn't that be nice? In summary, GANs have incredibly high quality results and relatively fast generation from a trained model. However, they lack intrinsic evaluation metrics, have unstable training, though that's been fairly remedied. No formal density estimation that's inherent to the model, and it can be challenging to invert an image to its latent space representation, especially if the model is very large and it's hard to find where that latent might be. It is important, I think, to emphasize the significance of having high-fidelity results. GANs are arguably the best and arguably the first AI model to achieve such realistic outputs, and very consistently too. GANs are often used elsewhere just to enhance the output's realism. That's really critical to know, and that's where GANs can be applied in so many different areas. In the next video, you'll see how some of these disadvantages are remedied with other approaches.
