0:00

Hello, welcome back to the course on Model Signal Processing for Music Applications.

Â In the previous programming lecture,

Â we talked about the HPR model, the Harmonic Plus Residual model.

Â That was the idea of modeling a sound with a harmonic and residual component.

Â And we talked about the implementation in the SMS tools package.

Â Now, we want to go a step further and talk about the hpsModel,

Â the Harmonics plus Stochastic model.

Â So now, we are modelling the residual that we obtained with

Â the stochastic modeling approach.

Â So, we obtain a harmonic and stochastic representation of a particular sound.

Â So the diagram, the block diagram, is very similar.

Â The whole thing for the analysis is the same

Â until we obtain the residual spectrum of X, Xr[k].

Â And then instead of just going and summing it with a harmonic component,

Â what we are doing is approximating it with a stochastic approximation.

Â So, by assuming the magnitude spectrum and

Â then the synthesis is done by generating random numbers for

Â the phase, and combining it with a sort of an interpolated version

Â of these approximations so that we generate a complete spectrum.

Â And this stochastic spectrum can then be added back to the harmonic component.

Â The great advantage of these stochastic approximation is that now

Â we are more flexible, we can do many more things on this

Â representation apart from being much more compact.

Â See, this is stochastic approximation,

Â reduces the amount of information that is required to reconstruct a signal.

Â So apart from being a compactor presentation, is a more flexible,

Â and will be able to do quite a few things with that.

Â So let's start from the same code that we looked at in the last lecture,

Â which was this code in which, well now I changed the sound.

Â Now I use the foot sound, but basically it performs

Â the harmonic analysis by taking a 50 and detecting the zero,

Â so it performs the harmonic analysis steps.

Â And then it synthesizes those harmonics and

Â it subtracts them from the orignal signal.

Â It recomputes the spectrum of the original signal with the same window size and

Â type than the one that is in this generated sinusoidal spectrum.

Â So we need to multiply it by a window, and

Â we need to choose the window size that was the same

Â then was used for the sinusoidal synthesis.

Â And, we can then subtract the harmonics from

Â the original signal and obtain the residual.

Â And now in this lecture,

Â basically we focus on the stochastic approximation of these residuals.

Â These three lines are doing that.

Â So, the first is we are converting it to a DV scale.

Â So, we are converting half of the magnitude spectrum to a DV scale and

Â we obtain a new magnitude spectrum.

Â And then we call this function called resample with this, my new spectrum,

Â and of course with values that are above a certain threshold.

Â And we are converting it to a shorter array.

Â And the size of this new array,

Â is the size that we started from multiplied by stocf.

Â And stocf is our stochastic factor approximation.

Â Basically, is moving approximation or down sampling factor.

Â So basically, it means that the output array

Â will be 40% of the original spectrum.

Â And then, we are up sampling again to the size that we

Â started with in order that then we can do the resynthesis.

Â So in fact, resample can be use both for down sampling and for up sampling.

Â And it uses the DFT approach.

Â So, let's talk a little bit about that.

Â So, we can go to do the documentation on the resample of function within scipy.

Â So this is the documentation, so it takes an array,

Â it takes the size of the output array.

Â So either less or more, so you can down sample or up sample.

Â And well, there is some optional arguments also that allow us to give a particular

Â window or to give more multidimensional array and process only part of that.

Â And since Python of course is all open source,

Â we can also see the source code of this function.

Â And as I said, well, every sample uses an frequency-response

Â approach to do downsampling and upsampling.

Â And what it does is, it performs first the FFT

Â of the input signal then it will basically cut the spectrum or

Â append samples depending if its down sampling or up sampling.

Â And then, once we have recomputed a longer or a shorter spectrum by cutting or

Â appending zeroes, we perform the inverse of 50.

Â That's a quite straightforward, simple way of performing

Â interpolation or downsampling or upsampling.

Â So we go back here, so this returning for

Â the first three samples, we will return less samples and

Â then afterwards we will convert it to this up sample version

Â that will be a smooth version of the spectrum we started with.

Â Let's run this, so we can see how that works.

Â So, let's run test4.

Â Okay and now we can plot the original spectrum.

Â So, mxr is already the dB version of the spectrum and half of it.

Â Okay, so this is our residual spectrum.

Â And then we plot on top of that the stocEnv, that's of the same size but

Â a smooth version because it's a result of having first down tempo.

Â And then, up sample with interpolation.

Â Okay so, as you see we are very much in fact if we zoom into any of these regions,

Â we see that the green line is the smooth version but

Â it is of course is smoother than the original one,

Â but it follows quite well, the original signal.

Â So, let's change the stochastic factor of 0.4 and let's put maybe 0.1.

Â So then, that will be a stronger smoothing factor.

Â And if we save this, and then we compute it.

Â 8:08

Okay, now we plot again the original residual spectrum.

Â And now, let's plot the smooth envelope.

Â Okay, and this clearly shows the smooth envelope on top of that,

Â and is much smoother than before.

Â Okay, so that's basically what the stochastic approximation does.

Â And we now go into the HPS model code of the SMS tools but

Â we'll see the analysis function the synthesis function.

Â And also one function that does analysis and synthesis at the frame level.

Â And the analysis for example is basically the same then for the HPR model.

Â We only add one input parameter which is this stop f,

Â factored as a moving factor and then in the analysis,

Â we first perform the harmonic analysis then we subtract

Â the sinusoids from the original signal to obtain the residual.

Â And then we run the stochastic model now,

Â the one that we already discussed when we talked about the stochastic model.

Â So we just, the stochastic model receives the signal, the residual signal.

Â And it has the hope size, the same hope size that we're using and now the window

Â size that we used for stochastic approximation is just twice a hope size,

Â so in fact, the stochastic approximation does not require big windows.

Â In fact it's the other way around, the smaller the better so

Â we can capture time information, the stochastic components, the residual

Â components, the time information, the attacks of the nodes, etc.

Â Are most critical.

Â So, the smaller the window, the better.

Â In fact, if H many times is by default 128 samples,

Â here we are using windows of 256 samples.

Â So, we will be computing an FFT of 256 samples and approximating that, okay?

Â And of course then we return the harmonics.

Â And instead of returning the residual, we return this envelopes.

Â This sequence of envelopes that we are computing.

Â And in the synthesis part, well it's a similar situation.

Â We synthesize the sinusoids, and then synthesize the stochastic

Â component using the stochastic model synth that also we talked about.

Â So from these envelopes, we can synthesize the stochastic component.

Â And the function that puts it all together,

Â there is this hps model function that includes analysis and

Â synthesis using the harmonic plus a stochastic model.

Â So, it has some parameters.

Â The full parameters and then it performs analysis, and then it performs synthesis.

Â And in the process,

Â it plots certain intermediate values.

Â Okay so, let's go to the terminal and now instead of running the code or

Â running the code by calling the file, we can also run the function directly,

Â and that allows us to be able to change the parameters.

Â So if we import this function,

Â the HPS model function.

Â Okay, now we have imported.

Â We have executed.

Â Now, well we better to just imported with a given name.

Â For example, HPS.

Â Okay and now we can call HPS and the function within HPS.

Â So within this file there is this function, main.

Â So, we can just call main.

Â And here, now we'll be able to change the values of these parameters,

Â so let's just get the values by default.

Â 12:35

Okay, so this computes this analysis and synthesis of this sound,

Â of this sax-phrase with the default parameters.

Â And now, we can play around.

Â So of course,

Â we could use the interface that we have been showing in the demo classes.

Â But here, maybe sort of more intuitive in terms of understanding

Â what is going on and being able to go deep into the code.

Â To just replace parameters here, so for example,

Â now if I want to change the threshold and make it a smaller threshold.

Â Instead of minus 100 let's put minus 40,

Â then we will run the same code but changing the threshold,

Â and of course, changing the threshold has meant that it found only few harmonics.

Â In fact, it can see that everything is stochastic.

Â And in here, the stochastic representation is the one that is more prominent.

Â Okay.

Â And you can change any parameter.

Â You can change the sound, for example.

Â Okay, we can change the sound file or

Â we can change the minimum duration of the harmonics, etc, etc.

Â So, no need to explain that anymore.

Â Basically, we can just execute in any way we want.

Â Okay, and that's all I wanted to say.

Â So, this is basically an extension of the Harmonic plus Residual Model.

Â And we have seen how it works.

Â So basically,

Â this is the most complex model that we're going to see in this course.

Â And all the others can be understood as related with this one.

Â So if you understand this model, I think it's a good way to

Â understand a lot of things we have been talking about.

Â So, that's all for today.

Â So, I hope to see you next lecture.

Â Thank you.

Â