Clusters

Loading...

From the course by H2O

Practical Machine Learning on H2O

12 ratings

H2O

Practical Machine Learning on H2O

12 ratings

In this course, we will learn all the core techniques needed to make effective use of H2O. Even if you have no prior experience of machine learning, even if your math is weak, by the end of this course you will be able to make machine learning models using a variety of algorithms. We will be using linear models, random forest, GBMs and of course deep learning, as well as some unsupervised learning algorithms. You will also be able to evaluate your models and choose the best model to suit not just your data but the other business restraints you may be under.

From the lesson

Everything Else!

Pojo And Mojo5:49

Driverless AI1:58

Meet the Instructors

Darren Cook

Throughout this course, I've been telling you that the scalability of H2O,

and in particular its ability to scale to

large datasets just by adding more and more machines,

is a key feature,

a key advantage of using H2O.

But we haven't really looked at what's involved in setting up a cluster.

This video is just going to take a high-level overview of H2O clusters.

So, the first thing to be aware of is that each of the nodes in

your cluster should really be on the same LAN or in the same data center.

Using client providers, such as Amazon EC2,

to set up your nodes works very well.

Each of your nodes,

each of the machines in the cluster,

should have the same size memory.

So there's no point setting up a cluster with

four gigabytes on one machine and 32 gigabytes on another.

You're better off just using the single big machine.

If at all possible,

you should still go for the biggest most powerful machines you can.

So, if you have the choice between setting up a cluster of 16 machines,

where each of them has four gigabytes of memory and two cores,

versus a cluster of just two machines,

but each of them has 16 cores 32 gigabytes,

so that's 32 cores and 64 gigs of memory either way,

you should go for the two machine model.

This isn't just because of reducing communication overhead,

though that is a big factor,

but also because the way H2O clusters work,

if any single machine dies,

the whole cluster dies and you're going to have to start over again.

So, fewer machines means a lower probability of a bad event happening.

To set up the cluster,

there's two main approaches.

Firstly, you can use a flat file,

this is a list of all the IP addresses of each of the nodes in your cluster,

and you distribute this to each node before you start them up.

Alternatively, you can use auto discovery,

which is where you describe the network and

each machine will go out and scan for the other machines in the network.

So, it's going to take a bit longer to start

up a cluster because it has to do all that scanning,

but it's less manual work in advance.

If you want to learn more about clusters in H2O,

I recommend you go to docs.h2o.ai,

look in the user guide,

and there's a section on clusters that will give you all the information you need.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2018 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play