[MUSIC]. Right in this weeks audio class we're going to learn about how we can get more control over the audio. You've already had a bit of a preview of that last week where towards the end of the exercises you would have got into the depths of controlling audio and using the various functions. What we're going to do this week is look at a bit more detail about how audio is represented in the computer's memory Wha-what's this thing we're telling it to play? Wha-what's in there and what's it, what's happening? So we're trying to, pick a few of those details. And then we'll look at some more, control methods, for, for controlling audio playback using RAPI. So here's a, here's a starter slide showing you how Computers represent sound. So what have we got? We've got this, two blocks here. So, the, the block, the, the has the number zero through seven. So imagine that's a short snippet of sound. Only seven, seven little snapshots of sound. that would last a very, very short time. Because sound in a, in a machine is typically represented with a lot of numbers. So it's, it's very similar to, something that you might be more familiar with, which is the idea of, of a video, and, and film being chopped into frames. We do the same thing with sound in a computer. We slice it into frames, called samples, and then we just play those out through the sound card in sequence. Now, if we, if we look at those frames, what's actually in them, it's slightly different from the video. So in the, in the case of audio, there are a lot of frames, there could be up to, well, a typical sample, right say, for a good quality audio, would be 44,100 of these frames, of these samples per second, which is a lot of data. So this is, this is way compressed audio formats exists like mp3 because it makes it possible to reduce the amount of data required to store the audio but maintain the quality. But in this example we're going to be working with uncompressed data, which is just the raw numbers that represent that audio signal. So let's have a look. So in the first box here, you can see, I've illustrated what the real contents of a, of a single sample might be in the computer's memory. So I've represented it with the binary number, which, which converts to 27 in base ten. Those of you, in, in the numbers you'll be more familiar with. So you don't need to worry too much about that conversion. But it's just to show you that the computer's memory is taking up 16 of these zeros and ones to represent one of those samples. Sometimes it might take 24 if it's a higher quality audio. But in the examples we're working with here, typically it's going to be 16. So we use 16 zeros and ones to represent one single snapshot of sand. And, we use say, 44,100 of those snapshots to represent one second of sound. Okay, so what I've, I've, I've shown you here is that, that chunk of memory which is represented by those boxes can extend to you know, many thousands of, of spaces in the memory So we talk about the sample rate which is the number of those samples that are used to represent the signal per second. The higher the sample rate the more accurate the representation of the sound, the wider range of sounds we can represent. the bit depth is another property which is as I said earlier the number of 0s and 1s that I use to represent each of those. Snapshots. So if we have a very low bit depth, what happens is the samples are very inaccurate. Because they don't have enough range to represent all the possible values that sample might have. But if you have a higher bit depth. then you can, you can store a much more accurate representation of sound. Now I've got a, an animation which shows that. Perhaps more clearly. So the first animation I'm going to show illustrates what happens if you increase the sample rate. So we'll start with a very low sample rate and we'll see what a typical smooth signal looks like. Now, it will start off very blocky and then get much smoother as it goes. So let's have a look at the animation now. So, as the, as the pointer moves across the screen, the sample rate increases. So, you can see the signal is, kind of, getting smoother and smoother. And the smoother the signal, the more accurate It is in, as a representation of the original sound. So remember, earlier, I was recording myself hitting something. And and when I play that back, I want to hear an accurate recording of that. I don't just want to hear a really rough inaccurate recording of it. So, having a high sample rate allows that. So, on the next slide, we're going to see what the, what the representation looks like, as we increase the bit depth, is a kind of similar effect, in that it becomes less blocky as you increase the bit depth, but it's totally different. Okay. What we might say is that increasing the bit depth increases the vertical resolution, the resolution from top to bottom on the y axis, which represents how loud the signal was at any given time. Whereas increasing the sample rate changes the horizontal resolution, which is, if you like, how precise it is in time. So the sample rate represents the, the precision in time. Whereas the bit depth represents the precision in, in the actual measurements that we're taking. Okay. So that's the end of the theory section. We're now going to look at how we can get more control over this, this sound. And, and, and to, to understand what, what we're doing when we're changing those controls. So the first thing we're going to try and do It's to create an interactive speed slider which allows us to vary the speed of the playback. So you can see in this code here that I'm first calculating a ratio. So now I'm going to demonstrate how you can vary the speed of the sample playback. So remember the speed, relates to the sample rate if you like. So it's the speed of playback is how quickly we're flinging those numbers at the sound card. So, if you increase the speed, it's going to zip through that, that, that section of memory that, that's storing the sound faster. So if we double the speed, it should get through twice as fast. So let's just remind ourselves by playing it at normal speed. So player, speed, one. Let's just run this app and see how it sounds. So I'm running it now. So it's now building for the Android device. [SOUND] So that's, that's speed one, that's, that sounds very similar to how it did when I actually played it in the real world. What happens if I double the speed? Speed [NOISE] It's the 2. [MUSIC] So that you can see that it sounds higher. [MUSIC] So what's doing it, it's scanning through those [UNKNOWN] twice as fast as sending them out to the sound card. Obviously, it's really boring, having to continue, edit, and then run it on the device. It'd be much nicer if we could experiment with the speed in an interactive way. So what we're going to do is make a simple, if you like, slider that allows us to slide the mouse pointer across the screen, or a finger across the screen, and. Increase the speed. Now speed is, is useful so in the range zero to two. So that gives us an interesting range. 'Kay, we just heard what it sounds like if you play at speed two? Speed zero, it doesn't play at all, and then and so on, up to two. So let, let's just listen to that by making interactive slider. So what I'm going to do Is, in the draw function I'm going to continually calculate a new speed for the player. So I'm going to calculate, first of all, the ratio of the position of the mouse to the width of the screen. And you'll see all this, these float, things there. I'll explain what they are, they're for. So, I do float ratio. Now remember, float is a number that is not a whole number, it's a real number, so we can have say, 0.5 or 0.6. And in this case we want to, because we've only got the range 0 to 2 for the speed in this example we want to be able to specify that very precisely. So I'm going to take the floating point version of the mouse position and floating point version of the width,and that, that would cause it to do the calculation with these accurate numbers. If I didn't do this, it would do it in whole numbers and I wouldn't get my accurate ratio. So that's the ratio of the MouseX to the width. What that means is, when they're on the far left of the screen it'll be zero and when they're on the far right it'll be one. Now I'm just going to scale that up by two, so that's in the range zero to two. And then, I'll just call it player.speed with that value. Okay. So that should mean I can now, change the speed dynamically. What I might do, when I tap it should queue it back to the beginning again. We'll set it to loop. We'll set looping to true so that it's kind of You can continue to hear what's going on. So let's play this sketch now, let's just review that before we do it actually. So I'm just going to review what I just typed. Player set looping is set to true, so that it continually plays the sound. I've calculated the ratio of the position of the mouse or the, or the finger tap on the screen. Across the screen, scout it by two, and then sent into, for the speed, so I will be setting a speed to a range from zero to two, so the idea is that when it's on the far left of the screen you should play it slowly, when it's on the far right of the screen it should play quickly. Let's see what it does... [SOUND] Okay, so what you heard there was me moving my finger around and varying the speed of the playback, as, as it was playing the sample. Now if you do that with, with the break beat, just as, by comparison, you'll see that it has a slightly different effect. [MUSIC] Okay, so then you're getting towards more of a musical deejaying experience where you can slow the sound down and then speed it back up again. And it has a more performance feel to it rather than playing with the sound. [MUSIC].
