0:00

Hello, welcome to the course analyzing and processing for music applications.

Â This week we are focusing on transformations on

Â describing one application

Â of the spectral modeling techniques that we are covering in this course.

Â And in this programming lectures,

Â we are actually trying to understand the programming aspect of it and

Â the code that have been put in together on the SMS tool set package.

Â So in these lectures, I want to talk about the most sophisticated model that

Â we have covered, which is the harmonic plus stochastic model and

Â the transformations we can do with that.

Â So we have been describing this type of block diagram

Â in which after all this analysis that we do of understanding the harmonics,

Â extracting the information, and then subtracting them from the original signal.

Â We obtain the stochastic approximation, and now we are focusing on these two red

Â 1:06

blocks in which we can apply transformations to both

Â a stochastic representation and harmonic representation.

Â And of course then we can synthesize back.

Â One of these transformations is little bit special, which is the morphing.

Â The morphing requires two analysis of two sounds.

Â So we have one sound that performs all these analysis and

Â another one that does the same.

Â And we are basically interpolating these two set of functions.

Â The arrays of frequency amplitudes and the envelopes of the stochastic approximation.

Â 1:49

Okay, so let's go to first the code that performs these transformations.

Â And the one for the harmonics is harmonic transformations.

Â So we have this file called harmonic transformations in which

Â we do the transformation just on the harmonic component of this model.

Â And the ones that we have implemented, the transformations that we have in the SMS

Â tools, are the idea of scaling the harmonics then,

Â stretching the harmonics in a non-harmonic way.

Â And then the idea of preserving or

Â not, the spectral shape or the timbre of the sound.

Â Okay, so within this function, harmonic frequency scaling we can do

Â all this at the same time if we pass these three control parameters.

Â We're not going to go into the detail of this code but basically we can see

Â there is here, at the end, the three basic transformation lines.

Â Which we have the frequencies that these are the input

Â frequencies of all the harmonics of a particular frame and

Â a particular kind of set of harmonics, the ones that exist.

Â And we apply a scaling factor and multiplicity scaling factor.

Â So these would be the transposition, what we call transposition.

Â 3:26

And then once we can transpose, we can also do the stretching.

Â Okay, and this stretching is a factor that depends on the harmonic.

Â This variable invalid is basically the harmonic number if it exists.

Â So we are elevating this stretching factor to this value.

Â So it's a way to change the frequencies depending on the harmony.

Â There's several ways of implementing this idea of stretching.

Â We can do it in fact the other way around by having

Â the harmony number to the power of the stretching factor or this.

Â You can play around with these two implementations and

Â then there is the idea of timbre preservation.

Â So in this particular function we are not manipulating the magnitudes but

Â what we're doing is if we are preserving the timbre, then we are changing

Â the magnitudes in a way that we give to every harmonic the magnitude

Â that is closer to the frequency that it originally had.

Â So basically we have a spectral envelope and

Â then we recompute the magnitude in a way that it preserves

Â 4:50

the shape that originally had.

Â Then, we have the transformations that are specific for

Â the combination of the harmonic and stochastic components.

Â Which are in this HPS transformation file, and

Â there is basically two functions, one that performs time scaling.

Â And the time scaling is done on the three components at the same time,

Â the frequency and amplitude of the harmonics, and the stochastic envelope.

Â So it's perform in this core loop.

Â And basically in interpolates while in fact it just adds or

Â deletes frames depending on these time scaling parameter.

Â And then these more complex function, which is the morphing that accepts

Â the race from the two sounds, so the analysis of the two sounds.

Â 5:53

So the core of the function is just this loop,

Â that basically iterates over the whole set of frames of the sound.

Â It finds the harmonics that are present in both sounds,

Â because of course the interpolation between two sounds can only happen

Â if there is content in the two.

Â One has a zero harmonic, that becomes a very complex thing to handle.

Â So in this code it looks for the intersection, so

Â the harmonics that are present.

Â And then it just performs the interpolation of the frequencies,

Â the magnitudes, and the stochastic envelope when there is some

Â information existing in both sounds, of the frame of both sounds.

Â So clearly the number of frames has to be the same and

Â ideally the number of harmonics and stochastic envelope, the number

Â of elements over break points and the approximation function has to be the same.

Â And that's what it does, basically we have this function called

Â HPS morph function that wraps these functions and allows us to do analysis and

Â synthesis and is the one that is called from the interface.

Â So we have basically two functions.

Â One is analysis that returns the results of the analysis.

Â And then another is the transformation and

Â synthesis that performs the transformation and synthesis of the sound.

Â Okay, and if we run this we can just execute that so

Â we can just type run and hpsMorph.

Â 7:43

Okay, this will execute both functions.

Â Of course we can also execute one and the other independently by

Â importing the file and then just calling the two separately.

Â So this is the result of the analysis function that

Â performs analysis of the two.

Â And this is the result of the transformation and

Â synthesis that performs the resynthesized version of that.

Â Since we already have listened to these sounds so

Â there is no need to listen to them here now.

Â But please feel free to play around with these functions.

Â It is quite interesting what you can do.

Â And of course by programming you can extend these quite a lot.

Â And that's all.

Â Basically we have gone through the HPS part of the transformations

Â using SMS tools, these number of transformations that

Â are specific for the harmonic plus stochastic model.

Â And being the most sophisticated one, it can perform some interesting things.

Â But it's a little difficult to use it, so you have to know to do proper analysis and

Â to control these representations that they're a little bit more sophisticated.

Â 9:08

Anyway, so hopefully with these you got an idea of the harmonic

Â stochastic model in terms of each potential for transforming sounds.

Â Sounds that of course have to be harmonic and

Â they have to have a meaningful stochastic representation.

Â If those sounds fulfill that model then

Â the number of things we can do with it is really great.

Â So, thank you very much for your attention and I will see you next class, bye bye.

Â