0:16

- It's a, I guess Computer Science's attempt

Â to mimic a real,

Â the neurons and how our brain actually functions.

Â So 20, 30 years ago a neural network

Â would have some inputs that would come in

Â they would be fed into different processing nodes

Â that would then do some transformation on them

Â and aggregate them or something

Â and then maybe go to another level of nodes

Â and finally some output would come out.

Â And I can remember training a neural network

Â to recognize digits, handwritten digits and stuff.

Â (music)

Â 1:00

So a neural network is trying to use

Â a computer program that will mimic

Â how neurons, how our brains use neurons to process things,

Â brains to synapse, neurons to synapses

Â and building these complex networks that can be trained.

Â So a neural network starts out

Â with some inputs and some outputs

Â and you keep feeding these inputs in

Â to try to see what kinds of transformations

Â will get to these outputs,

Â and you keep doing this over and over and over again

Â in a way that this network should converge

Â so these input, the transformations

Â will eventually get these outputs.

Â The problem with neural networks was that

Â even though the theory was there

Â and they did work on small problems

Â like recognizing handwritten digits and things like that,

Â they were computationally very intensive,

Â and so they went out of favor.

Â I stopped teaching them,

Â well, probably 15 years ago.

Â Then all of a sudden we started hearing about deep learning.

Â I heard the term deep learning.

Â This is another term that

Â when did you first hear it?

Â Fours years ago, five years ago.

Â So I finally said,

Â "What the hell is deep learning?

Â It's really doing all this great stuff.

Â What is it?"

Â I Google it and I find this is neural networks on steroids.

Â What they did was they just had more

Â multiple layers of neural networks

Â and they use lots and lots and lots

Â of computing power to solve them.

Â Just before this interview

Â I had a young faculty member in the marketing department

Â whose research is partially based on deep learning.

Â She needs a computer that has

Â a graphics processing unit in it

Â because it takes an enormous amount of matrix

Â and linear algebra calculations

Â to actually do all of the mathematics

Â that you need in neural networks,

Â but they are now quite capable.

Â We now have neural networks and deep learning

Â that can recognize speech, can recognize people.

Â If you're out there and getting your face recognized

Â I guarantee that NSA has a lot of work

Â going on in neural networks.

Â The University, right now,

Â as Director of Research Computing,

Â I have some small set of machines

Â down at our South Data Center

Â and I went in there last week

Â and there were just piles and piles and piles

Â of cardboard boxes all from Dell with a GPU on the side.

Â Well, a GPU is a graphics processing unit.

Â There is only one application in this University

Â that needs 200 servers,

Â each with graphics processing units in it,

Â and each graphics processing unit

Â has the equivalent of 600 cores of processing,

Â so this is tens of thousands of processing cores.

Â That is for deep learning.

Â I guarantee.

Â (music)

Â 4:12

Some of the first ones are speech recognition.

Â Yann LeCun who teaches the deep learning class at NYU

Â and is also the Head Data Scientist at Facebook

Â comes into class with a notebook

Â and it's a pretty thick notebook.

Â It looks a little odd because it's like this.

Â It's that thick because it has

Â a couple of graphics processing units in it

Â and then he will ask the class

Â to start to speak to this thing

Â and it will train while he's in class,

Â he will train a neural network to recognize speech.

Â So recognizing speech, recognizing people, images,

Â classifying images, almost all of the traditional tasks

Â that neural nets used to work on in little tiny things,

Â now they can do really, really large things.

Â It will learn, on it's own, the difference between

Â a cat and a dog and different kinds of objects.

Â It doesn't have to be taught.

Â It doesn't, it just learns.

Â That's why they call it deep learning,

Â and if you hear, he plays this.

Â If you hear how it recognizes speech and generates speech,

Â it sounds like a baby learning to talk.

Â You can just, you're like

Â (babbles)

Â 5:55

You need to learn some linear algebra.

Â A lot of this stuff is based on matrix

Â and linear algebra, so you need to know how to do,

Â use linear algebra and do transformations.

Â Now, on the other hand,

Â there's now lots of packages out there

Â that will do deep learning

Â and they'll do all the linear algebra for you,

Â but you should have some idea

Â of what is happening underneath.

Â Deep learning, in particular,

Â needs really high powered computational power,

Â so it's not something that you're going to go out

Â and do on your notebook for, you could play with it,

Â but if you really want to do it seriously

Â you have to have some special computational resources.

Â (music)

Â