So we've looked at the sunspot data using a standard DNN like way back at the beginning of this course. We've also been working a lot with RNNs and a little with convolutional neural networks. So what would happen if we put them all together to see if we can predict the sunspot activity? This is a difficult dataset because like we've seen already, while it's seasonal the period is really long, around 11 years, and it's not perfectly seasonable during that period. So let's take a look at using all the tools we have to see if we can build a decent prediction using machine learning. So here's the first piece of code we can try. I've gone a little crazy here, so let's break it down piece by piece. First of all I'm setting the batch size to 64 and the window size to 60. Then we'll start with a 1D convolution that we'll learn 32 filters. This will output to a couple of LSTMs with 32 cells each before feeding into a DNN similar to what we saw earlier, 30 neurons, then 10, and one. Finally, as our numbers are in the 1-400 range, there is a Lambda layer that multiplies out our X by 400. With the first test run to establish the best learning, rate we get this chart. This suggests the best learning rate for this network will be around 10 to the minus 5. So when I trained for 500 epochs with this setup, here's my results. It's pretty good with a nice low MAE. But when I look at my loss function during training, I can see that there's a lot of noise which tells me that I can certainly optimize it a bit, and as we saw from earlier videos, one of the best things to look at in these circumstances is the batch size. So I'll increase it to 256 and retrain. After 500 epochs, my predictions have improved a little which is a step in the right direction. But look at my training noise. Particularly towards the end of the training is really noisy but it's a very regular looking wave. This suggests that my larger batch size was good, but maybe a little off. It's not catastrophic because as you can see the fluctuations are really small but it would be very nice if we could regularize this loss a bit more, which then brings me to another thing to try. My training data has 3,000 data points in it. So why are things like my window size and batch size powers of two that aren't necessarily evenly divisible into 3,000? What would happen if I were to change my parameters to suit, and not just the window and batch size, how about changing the filters too? So what if I set that to 60, and the LSTMs to 60 instead of 32 or 64? My DNN already look good, so I won't change them. So after training this for 500 epochs, my scores improved again albeit slightly. So it shows we're heading in the right direction. What's interesting is that the noise and the loss function actually increased the bits, and that made me want to experiment with the batch size again. So I reduced it to just 100 and I got these results. Now here my MAE has actually gone up a little. The projections are doing much better in the higher peaks than earlier but the overall accuracy has gone down, and the loss has smoothed out except for a couple of large blips. Experimenting with hyperparameters like this is a great way to learn the ins and outs of machine learning, not just with sequences but with anything. I thoroughly recommend spending time on it and seeing if you can improve on this model. In addition, you should accompany that work with looking deeper into how all of these things in machine learning work and Andrews courses are terrific for that. I strongly recommend them if you haven't done them already.