How to backtest?

Loading...

From the course by Indian School of Business

Creating a Portfolio

100 ratings

Indian School of Business

Creating a Portfolio

100 ratings

Course 4 of 5 in the Specialization Trading Strategies in Emerging Markets

This course integrates all the learning from the first three courses and guides the learner about ways of building a portfolio of strategies and integrating the same into a hedge fund. In the first part of the course, you will be taught ways of measuring the contribution of a strategy to a portfolio in terms of risk and return. You will be able to appreciate the consequences of including a strategy to a new as well an existing portfolio. Next, you are taught various ways of conducting the tilting analysis in order to determine the optimal weight to be placed on each strategy. After this you will learn to develop techniques for minimizing overall portfolio risk. You will also get a basic overview of the regulatory framework that is applicable to hedge funds. You will know about different types of investors and the expectations of each type of investors.

From the lesson

Module 4

In this module you will learn the basics of trading strategies based on text mining and the importance of a benchmark to evaluate the performance of your portfolio. You will also see how to backtest your trading strategy. Finally, you will learn the importance of reporting and compliance in trading.

Strategies based on Text Mining9:44

Benchmarking12:34

Financial Instruments10:16

Backtesting5:02

How to backtest?13:40

Meet the Instructors

Ramabhadran Thirumalai
Assistant Professor
Indian School of Business
Prasanna Tantri
Senior Associate Director
Center for Analytical Finance

[MUSIC]

>> Learning outcomes.

After watching this video,

you will be able to understand how to perform a back test.

>> Now, how do you do a back test?

First thing you have to figure out when you read the paper,

I have told you right at the beginning itself that the two most important

sections in a paper that's useful for someone who wants to trade based on

the idea of that paper are, one is the section that describes the algorithm.

The second is a section that describes the data.

Now, if you want to trade in the US, you can just go to that paper,

figure out the sources of data, and you can download stock prices or

whatever information is required.

You may be trading in other countries.

So then, the first challenge is, where do you get that data from?

So for example, let's say your momentum strategy.

All that is required for momentum strategy is stock prices.

You need to have closing stock prices.

So you are to figure out the source that gives you closing stock prices in your

respective market.

Whereas let's say you are doing price earnings announcement drift.

In that case, you need detailed accounting statements, not just stock prices.

You need the cash component, accruals.

And to calculate accruals,

then go back to the module I showed couple of modules back.

That the way you calculate accruals, you need to know receivables, payables,

cash in hand, investments, lot of other things that data required.

So you have to understand,

you should figure out what shows gives you all these information.

One way is to go, get into the website of each and every company and download this.

That's going to be time consuming.

I'm sure, in most of the countries,

there will be databases that give you this kind of information.

So if it is US, you can just pick from the paper.

If it's your country, you are to figure out these sources.

That's very, very, important.

So step number one is to go to the paper, read what data the paper uses.

And step number two, find out the source.

And then obviously, download.

Once you have the data, very important point when you do a back test,

you should do the test thinking as if you're sitting at that time.

And what do I mean by that?

Let me explain.

Now, imagine you are trying to test whether post–earnings-announcement

drift works, if that's what your purpose, right?

Now, what do you do?

You downloaded stock prices, you downloaded earnings forecast,

you downloaded the actual numbers.

You calculated the standardized and expected earnings and all that, suppose.

Now, you want to see whether this work in the year 2006, let's say, right?

Now, let's say your financial year start from January 1st.

Now, my question is.

You already arrived at winners and losers.

Let's assume you have arrived at the stocks that you are to buy and

the stocks you are to sell, right?

So is the setting clear?

You are trying to see whether PEAD works, post-earnings announcement drift works.

You collected all the data, and you arrived at stocks that you are to buy and

the stocks that you are to sell, right?

And you're testing whether this strategy has worked in the year 2005.

That's just an idea.

Now, You take this trade January 1st.

Can you do this trade on January 1st?

Can you test whether, let's say your holding period is one year.

Can we take the price of winners on January 1st?

What do you mean by see when you back test.

Going long means what?

You take the price on day 1,

take the price on last day, and see what is the return, right?

So if day 1 is 100, last day is 120, your return is 20%.

That is for longs.

For shorts, day 1 is 100, last day is 120,

your return is minus 20%, that's correct.

Because you are short.

You sold it at 100, you are buying at 20, so you lost 20 rupees,

or $20, whatever, right?

So my question is for price-earnings-announcement drift,

can you take January 1st as day 1?

Think about it.

The answer is a big no.

Why? Because financial statements will not be

ready on January 1st, even if, usually, in any year.

So if December 31st is the year ending,

no company will publicly declare results on January 1st.

It takes time.

The accounts have to be finalized, the audit committee has to approve,

the board finally has to approve, auditors in-between audit processes are there.

And finally, this has to be announced.

It usually takes anywhere between 20 to 40, sometimes 60 days.

So our back test, starting from January 1st, is not a realistic back test,

because you are not doing the test thinking as if you are going to trade.

This will become just an exercise to find out

the price movement from January to December.

So if you were in 2005, you could have not have traded a period on January 1st.

So whenever do a back test,

you should ask yourself, you should think that you are in that time zone.

And ask yourself whether any data that I'm using, does it have a look ahead bias?

Look ahead bias is very important,

because now you know that sitting today, you know the January 1st price.

But on January 1st, all the accounting numbers.

But on January 1st, you didn't know January 1st, the accounting numbers is

11 for January 1st, the previous December 31st accounting number.

So give a gap of two months, then do the calculations or

figure out when they announce the results, and start your back test from that day.

I hope the point is clear.

This is very, very important.

Otherwise, your back test may give you misleading results.

All that I'm trying to say, let me summarize this.

All that I'm trying to say is that you should not

induce look ahead bias in your tests.

So any test you do using past data,

the test should be designed in such a manner that whatever

data was available at the beginning of the period, you should use only that.

You cannot use the data which came later.

So this is very, very important when you do a back test.

Second, another very important point is that

whenever we do a back test, we have this problem of survival bias.

You cannot just pick up companies that exist today and

go back ten years, calculate their scores, rank them and trade.

That's not correct.

Think why.

Ten years back, you didn't know that these are the companies that will exist ten

years forward.

There were a lot many.

In fact, every 20, 30 years, if you look at composition of many of the indices, you

will be surprised to see that most of the companies, or many companies, many famous

companies at one point in time will not be a part of the index after, say, 20 years.

This has been the history.

So you cannot use today's strongest companies and

go back ten years and do the back test and say that some strategy works.

No.

At that time, you should take what the companies.

Suppose if your strategy is to take top 100 companies, let's say.

You will trade only on top 100 companies.

Taking today's top 100 companies is a strict no, no.

You should take top 100 companies of that year, a year before that.

Again, not that year after, year before that.

So this will take care of survival bias.

That will give a realistic picture.

Because among today's 100 companies, top 100 companies, ten years down the lane,

50 of them may not survive.

So think about it, right?

The companies that have survived today, the top 100 companies that have survived

today, are the ones which did phenomenally well maybe over the last ten years.

But you could not have selected them ten years before.

Now, for back testing,

if you use these companies, you are implicitly assuming ten years back,

you knew that these are the 100 companies that will grow in the next ten years.

That's a very, very wrong assumption.

See, always stake the odds against you when you're doing a back test.

Nothing you will gain by getting fabulous back testing results.

You will have to get results in real reading, not in back testing.

So it always helps us put the stakes against us, not in favor of us.

So going back to this example, look at 100 companies at that time.

And among those hundred, some have survived, some have not survived.

You might have longed those who have not survived, or

you might have short those who have not survived, then you would have made money,

and then see what has happened now.

So this is what happens with today's 100 companies function.

So when you want to trade with today's 100 companies, some of them will survive,

some of them will not survive.

So that's another thing about awarding survival bias.

Third important thing that you are to remember when you do back testing is that

the prices that you'll get,

most of our trading strategies will have prices, right?

These prices that you will get will be closing prices.

Now, the problem with closing price is in many countries,

closing price is the average of last 30 minutes, 45 minutes, 1 hour, or whatever.

Now, if that is how your closing price is calculated,

then you will not be able to trade using this closing price.

So you have to figure out the last traded price.

The last traded price is the price at which one could have traded.

And you have to remember, there are stocks, supposed it's not traded in

the last three hours, four hours, or two days, you should not consider such stocks.

Because their prices, think it about it, right?

So you take a stock which was not traded for three days in a row.

You take the price for a year, in any year, you take three days stale price.

Now, in these three days, assume the market has fallen by 10%, for

some big event has happened.

This is a hybrid stock, and this would have fallen by 25%.

Now, if you use this for back testing, you will be using a price three days back,

which will be like 30% more than what it should have been.

And if your strategy says short this stock, and by chance,

the next day, if the stock trades, it will open 30% down, and

your strategy will say you will make 30% money.

Let me give you an example.

I think this point is not clear.

Let's say a particular stock is trading at 10 on day zero, let's say.

Day 1, 2, 3, the stock has not traded.

And the market has crashed during this time.

Now, you are doing the back testing on day 3.

And when you do this back test, suppose you take the price as ten,

which is day zero price.

And assume that, and your formula says [FOREIGN]

on day 3, right?

Now, you'll think in your back test that you're shorted at ten.

Remember, this ten is three days back.

This stock was never available at ten on Day 3 because it was never traded.

Now, on Day 4, the stock may open at seven after a gap of four days.

But what your back test will assume, you are shorter at ten, and

then the next day is seven, you'll make 30% return, which is totally wrong.

And similarly, it may go on the other side also.

So you should first see whether the stock was traded,

as close to closing as possible, and is it a liquid stock?

Stocks that do not get traded for days together,

it is better to avoid them, leave them out of the sample.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play