0:00

Okay, let's start with TensorFlow which will be one of our of our main tools in

this course on specializations, alongside some other libraries such as.

What is TensorFlow?

TensorFlow is a high quality production ready and

the resource friendly library for machine learning.

After it was Open Source by Google in November 2015,

it quickly became one of the most popular frameworks,

both in the community of machine learning researchers and among practitioners.

As an example, I can only mention that most of relevant tasks that

Google uses in its business are nowadays powered by TensorFlow.

And soon, this might give you an idea of what power you can get once you

master this great tool.

As you will see, in a bit later, TensorFlow is not only a great tool for

machine learning, but also is very useful to the applied math in general

due to its flexibility, and computation, and analytical power.

Let's start with how TensorFlow is organized.

The main idea implemented this time in TensorFlow is the idea of a data flow

computational model.

This idea, as simple as it may sound turns out critical for prioritization,

scalability and portability of practical, machine learning algorithms.

Let's see how it works.

A data flow graph is a graph with nodes that represent some units of computation,

and edges that represent data consumed or produced by any given node.

In other words, edges represent the flow of data on the graph.

And that's why it's called a data flow graph.

Let's see how we can express a simple function of two variables,

x and y, shown here x squared times y + y + 2.

To this end, we first introduce a few sorts of nodes.

First, we will have blue squares.

Like these ones that represent inputs to the function.

They can be variables, like these ones, x and y, or

they can be constants like the node that for a fixed number, 2 in this function.

Other nodes like these ovals represent operations, or ops for short.

That are performed on these inputs and on outputs of other ops.

Now let's make one more step with this dataflow graph and see what happens

if our inputs to the graph, not the regular numbers, but other tensors.

But what are tensors?

If you are familiar with multidimensional arrays in Numpy,

they are exactly these tensors.

In other words tensors are multidimensional mathematical objects that

can be stored as multidimensional arrays in Python or other languages.

If you've never worked with multidimensional arrays in Python,

but are only familiar with matrices or two dimensional arrays,

you can think of a list of such matrices.

This would be tensor of rank-3.

Why a rank-3, what is a rank of a tensor?

The rank of a tensor is simply the number of dimensions of the tensor.

And this number would be 3 for

our example because we will have three coordinates to store our list of matrices.

The number of the matrix, the row and the column.

One very natural example of using rank-3 tensors in finance would be stock data.

In such tensor one dimension would stand for the date, another one for

the stock ticker, and the third one for

all features of this topic that you will want to include in your analysis.

For example, if you fetch stock data for a set of stocks, and

a set of dates from who using it will return an object called a data panel

which implements a rank feed tensor of exactly this type.

3:58

Please note that, no matter how many fishers you want to include in this

analysis, your data will still be a rank-3 tensor.

It's just the number of elements along the third dimension that will increase.

So once we let tensors live on the dataflow graph, we get TensorFlow.

It's name is an exact description of what it does.

It computes a flow of data called its Tensor on a graph.

More precisely, does it in two steps.

First it allows you to construct a dataflow graph for

your problem using a high level Python API.

Then it lets you execute this graph on a single device

such as a CPU or a GPU, or on a network of such devices.

As I mentioned earlier, TensorFlow was open sourced by Google in 2015.

And since then became one of the most popular tools for machine learning.

We will see why this is so by the end of this week, and even more so

going forward with this visualization.

5:31

You can think of it as a detailed project specification that you can give to your

assistant that explains in details what and

in what sequence should be done in a project.

The actual computation happens on the second step when

you pass your graph to a device such as CPU or a GPU.

This device will translate your graph into a highly optimized C++ code and

will do the calculation.

You can also make some nodes on your graph instead of the whole graph lazy.

And wait for some other nodes to be executed before kicking in.

While lazy evaluation is very important for an efficient working of TensorFlow,

it's even more important for you, as a user, to understand another

key feature of TensorFlow, namely automating differentiation.

6:57

That allows it to compute all derivatives sequentially.

The technique that is used in such sequential calculation is just

the chain rule for a derivative of a composite function.

To recall the chain rule, it says the following.

If an argument of function f is given by another function

n of an independent argument x, then the derivative for

function f with respect to x is given by a product of

the derivative of the function respect to its argument n,

times the derivative of function n with respect to x.

Let's now see how it works using our previous example.

9:20

Now we calculate the derivative with respect to the node n3,

which carries the input y.

In this case we have to remember that the node n3 is

connected to two nodes, n5 and n6, on the graph.

Therefore, we will have two terms in the general,

that we use to compute the derivative.

The final answer for this derivative will be x squared + 1,

which is certainly a correct answer,

as we can check it by differentiation of our function f.

And finally we compute the derivative with respect to x.

Again, the number 2xy here is exactly what we get

by differentiation of function f with respect to x.

So this is it for our first theoretical introduction to TensorFlow.