Other Applications of RL

Loading...

From the course by New York University Tandon School of Engineering

Overview of Advanced Methods of Reinforcement Learning in Finance

33 ratings

New York University Tandon School of Engineering

Overview of Advanced Methods of Reinforcement Learning in Finance

33 ratings

Course 4 of 4 in the Specialization Machine Learning and Reinforcement Learning in Finance

In the last course of our specialization, Overview of Advanced Methods of Reinforcement Learning in Finance, we will take a deeper look into topics discussed in our third course, Reinforcement Learning in Finance. In particular, we will talk about links between Reinforcement Learning, option pricing and physics, implications of Inverse Reinforcement Learning for modeling market impact and price dynamics, and perception-action cycles in Reinforcement Learning. Finally, we will overview trending and potential applications of Reinforcement Learning for high frequency trading, cryptocurrencies, peer-to-peer lending, and more.

From the lesson

Other Applications of Reinforcement Learning: P-2-P Lending, Cryptocurrency, etc.

Electronic Markets and LOB9:25

Trades, Quotes and Order Flow7:40

Limit Order Book8:52

LOB Modeling8:29

LOB Statistical Modeling10:40

LOB Modeling with ML and RL9:45

Other Applications of RL7:23

The Value of Universatility15:18

Meet the Instructors

Igor Halperin

This video will be a really short even though I will talk in it

about such interesting topics that

each one of them would probably deserve a full lecture,

if not a course on its own.

But we have to draw a line somewhere in this course,

so I thought that if we can't spend more time on these interesting topics

then at least we can indicate them for your future reference.

So, as we said a few times in this specialization,

the general class of problems that can be addressed

using methods of reinforcement learning is extremely wide.

In particular, many of tasks of

quantitative trading can be reduced to reinforcement learning.

In the course on reinforcement learning,

we talked about using RL for option pricing and stock portfolio optimization.

In this week, we talked more about

limit order book and the dealer problem of optimal execution.

In addition to their market investor and the dealer problems,

there is also a very interesting class of problems related to modeling market makers.

Their optimization problem is different from those of the investors and

the dealers but they're also amenable to methods of reinforcement learning.

Those of you who are interested in this topic can

start with a paper by a M. Dixon on using

reinforcement learning for this task as well as you

can play with his code that relies on tradinggym,

which in its turn is built on top of open AI gym library for reinforcement learning.

Another interesting potential implications for RL is

credit management of a loan portfolios in peer-to-peer lending or P2P lending.

Platforms that match online lenders and borrowers such as Lending Club,

Prosper, or OnDeck become very popular in the last years.

It's not anymore your mom and dad little business as now

these activities are funded by very serious folks such as hedge funds and banks.

Questions of optimal portfolio management of

warm portfolios are therefore quite important for these players.

The main topic is always more or less the same,

which is optimization of risk-return profile

in the context of sequential investment decision-making.

Reinforcement learning can be very useful for these tasks as

well as we discussed at length in our courses.

For P2P portfolios, portfolio optimization can be formulated as

convex optimization is constraints in a similar way to

how we did with stocks but there are also some differences.

For example, you can only go long in these markets as well as each one are computed

differently but the mathematical and modelling approach

is still largely the same as for the stocks.

Application software RL for P2P lending are currently an area of active research.

Another direction I wanted to mention here would be

potential applications to keep the currencies trading.

I believe that if you ended up taking this course chances are that

you're already short or familiar with cryptocurrencies.

Bitcoin or Ethereum are the most famous of them but there are many others,

more than a 100 competing cryptocurrencies that

are available for both investment and analysis.

In 2017, the total volume of cryptocurrencies markets in the US was about 120 billion,

out of which about 40 billion was in Bitcoin.

So, cryptocurrencies markets are similar in many ways to conventional,

financial markets but they're also differences mostly due to very high volatility,

absence of regulation, and vulnerability of investments.

As one example of using RL for optimal management of cryptocurrencies portfolios,

you can take a look at the paper referenced on this slide.

This paper combines reinforcement learning with e-current neural networks and LSTM,

for learning a state representation and a reduce dimensionality of the state space.

There are some other papers probably out there which do similar research.

Finally, more on the theoretical side,

I would like to mention a few interesting topics of perception-action cycles.

One of the very first diagrams in this course was this one.

We talked about differences between perception tasks and action tasks and how

supervised and unsupervised learning solve

perception tasks while enforcement learning solves action tasks.

There exists a very interesting body of work on information theory.

Based approach is where perception and actions

are integrated together into what is called perception-action cycles.

There are some references for you there if you want to follow

and explore these interesting research and I believe that

it has lots of potential as it allows us to bring

the feature selection problem directly into an action optimization task.

For example, in our toy model presented in the third week of this course,

signals CT are supposed to be found from an independent analysis of Alpha Research.

Such analysis, even though it's very available,

it's still not the same as selections or social features or

predictors directly at the level of Bellman equation and reinforcement learning.

All these are topics that you can explore on your own after you finish this course.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play