Cloud computing systems today, whether open-source or used inside companies, are built using a common set of core techniques, algorithms, and design philosophies – all centered around distributed systems. Learn about such fundamental distributed computing "concepts" for cloud computing. Some of these concepts include: clouds, MapReduce, key-value/NoSQL stores, classical distributed algorithms, widely-used distributed algorithms, scalability, trending areas, and much, much more! Know how these systems work from the inside out. Get your hands dirty using these concepts with provided homework exercises. In the programming assignments, implement some of these concepts in template code (programs) provided in the C++ programming language. Prior experience with C++ is required. The course also features interviews with leading researchers and managers, from both industry and academia.
Interview with Marcos Aguilera
Loading...
From the course by University of Illinois at Urbana-Champaign
Cloud Computing Concepts, Part 1
546 ratings
University of Illinois at Urbana-Champaign
546 ratings
Course 1 of 6 in the Specialization Cloud Computing
From the lesson
Week 4: Key-Value Stores, Time, and Ordering
Lesson 1: This module motivates and teaches the design of key-value/NoSQL storage/database systems. We cover the design of two major industry systems: Apache Cassandra and HBase. We also cover the famous CAP theorem. Lesson 2: Distributed systems are asynchronous, which makes clocks at different machines hard to synchronize. This module first covers various clock synchronization algorithms, and then covers ways of tagging events with causal timestamps that avoid synchronizing clocks. These classical algorithms were invented decades ago, yet are used widely in today’s cloud systems.
Meet the Instructors
Indranil GuptaProfessor
Department of Computer Science
I got my degree from Cornell University,
that was back in 2000,
and at that time,
I was working on very theoretical problems in distributed computing.
My adviser was Professor Sam Twigg.
He was looking at problems like consensus and atomic broadcasting and so forth.
And in 2000 after I got my degree,
I joined the Compaq Systems Research Center,
which used to be Digital Equipment Corporation,
and the folks there were very, very applied.
And so, I switched a little bit in
my career and I started doing more applied system stuff,
but still in distributed systems.
And so, from that time until now,
I've been doing a mix of theory and systems work,
all in distributed systems.
And so, in around 2002,
HP acquired Compaq and so I was part of HP Labs from 2002 until 2008.
In 2008, I decided to join Microsoft Research.
So, I've been here for the past six years.
And more recently, I've been working on still distributed systems but
focusing more on distributed storage systems for large data centers.
So, Microsoft is best known for its Windows operating system obviously,
and has been known for many years.
But Microsoft has done quite a bit of work in
the Cloud computing space in the recent past.
So, when someone says Cloud computing,
what comes to your mind as a Microsoft employee?
What do you think constitutes Cloud computing?
Yeah for me, Cloud computing has to do with utility computing.
The notion that there are large data centers that are under control of somebody else.
And then, you can go and run your software there,
and the Cloud is going to provide a lot of resources for you,
and some Clouds will provide key value storage systems,
some Clouds will provide databases, that is fine.
But the main thing about that is that the Cloud itself is being managed by a third party.
And you as a user of the Cloud,
this is space paying for that as opposed to buying your own machine.
I understand different people have different notions of what is Cloud computing,
but for me really,
Cloud computing has to do with utility computing.
There's a third party controlling the Cloud,
and then you can go and run your jobs there.
And then, you pay by using it.
You don't have to buy your own machines.
So, distributed systems has the research
in the area and the concepts in the area have existed for many, many decades.
Can you say something about the relationship between Cloud computing and
distributed systems or distributed algorithms?
Yeah. Well, if you're going to be using the Cloud,
that means automatically, you're going to be using the distributing system, right?
Because the machine is not next to you.
And in most applications that also running in the Cloud,
are going to be applications that are distributed.
There are a lot of people for example,
that what they put in the Cloud there are web applications,
and those things are being accessed remotely by clients and so forth.
Of course, you can vary the amount of distribution that you're doing it.
You could have just have simple server there where your system itself
is one machine perhaps because you don't have to scale to such large sizes,
and the clients are going to be out on the web.
But still, it's a one type of distributed systems.
But now, once you've put the computers away from you,
now you have to already start working with the distributed system.
And so, that's something which is inherent to the Cloud in my opinion.
So, are distributed algorithms, distributed systems concepts,
things like, Axles, or leader election,
are these useful in Microsoft's Cloud systems both internally or externally?
Yes. In many of the Clouds,
these things are extremely useful, right?
Because what's going on is that, the third-party vendor,
which provides the Cloud needs to provide
a good service and they want to make things for tolerance.
And ideally, they have lots of machines running there,
and they want one person to be looking over all of these machines.
And when you talk about algorithms such as boxes,
and now algorithms for consensus for replication and so forth,
these are the mechanisms that are ensuring
the faults tolerance to the Cloud so that you don't have to
have as many people there to fix the machines when they break, right?
And so, now many people have made this argument that,
when you scale to a large number of machines,
then you're going to have failures a lot of the time.
And then, once you have you failures a lot of the time,
you can have people trying to fix this failures all the time as well because otherwise,
I think the cost starts to go up pretty quickly.
And algorithms like boxes,
I mentioned is used for replication.
But there are many other distributed systems
algorithms that are used in the Cloud like with system PowerShell for example,
for the load balancing and so forth.
And those things are extremely useful there.
So, you have published quite a bit of
research results in the last few years in the Cloud computing area,
and of course, the students in the class are
learning about key value stores NoSQL storage systems.
Can you say a little bit of what you found in
your research about Cloud computing in general,
but also key value stores NoSQL storage systems?
Yeah. One thing that we did,
and that was in 2011,
we started working before this in even, 2009, 2010.
It was the problem of geo-distribution and geo-replication.
And at the time,
a lot of the distributed systems that were running inside
a data center were limited to just one data center.
And we thought, okay, but there's
all these working wide area networks that the research community has already done.
And on the other hand, when you run things in one data center,
you are subject to problems like,
disasters that will cause your application to go down.
And also there will be a lot of ideas from
a wide area networks that could be applied within a data center,
and we start thinking of the,
'what do we need to do this', right?
Well, we had study different,
if you look at wide area P2P systems for example,
there's a lot of concern about churn,
and about people that are free riding in your system,
and those things are perhaps less of a concern when you are inside a Cloud,
and you are running the infrastructure because there are basically you there.
I mean, there's all the third party people that are running the Cloud as well.
But you control the infrastructure and usually don't have
free riders because you're charging people to use the Cloud.
And so, similar things which apply to
previous research like free riding and so forth doesn't apply to the Cloud.
But there's many other techniques that they did for example,
techniques to high latency when you have to talk to free mode parties.
And we started to think, "So, what are all these techniques that we can use,
and what are the things that we are still missing,
that we need to develop?"
And so, one the things that we learned is basically,
we develop our techniques for executing transactions
efficiently when you have a geo-distributed system.
So you have multiple data centers that,
inside a data center,
the connection is very fast across the data center,
as it takes much longer to communicate.
And so, you have this kind of hierarchical distributed system.
And now you would like to efficiently execute transactions across the data centers,
and well, some data maybe local,
some of the data is remote,
and you'd like to have some reasonable consistency conditions or
isolation properties for your transactions that makes
sense and that allow your transactions to execute efficiently.
And so, we developed like an extent that allow you to
do this by taking some ideas that are already existing,
and when there were holes,
we had to fill these holes.
So obviously, there's a lot of work in the NoSQL space,
which is making it more powerful and more expensive.
Does this means relational databases,
the old SQL databases they're going to go away in the next few years?
I hope not there. So, what I can see from
the NoSQL community is that there's a lot of work that tries to
make NoSQL closer to SQL by any functionality that NoSQL did not have,
but that they wanted to have.
And so, take for example secondary indices,
or take for example that they being able to do JOINs,
and or, take the ability to do transactions for example.
When we started off with NoSQL,
this simple key value storage systems we had none of that.
All you could do is just read and get,
read and write or get input key value pairs.
And then people realized, "Okay, I need something more than that."
Actually, they started working on it.
Okay, let's have analytics now or let's have the ability to do transactions.
And so, if there's a real function that is there already provided by SQL.
So, what you can see is that there's a lot of work from
NoSQL that was only trying to add all these features that they're missing.
Now, when you say relational databases,
that intrinsically means that you have a schema,
and this is one thing that the NoSQL folks that,
to our advantage, we don't have schemas.
But now, I think schemas,
are a good thing in a way,
but you don't be shackled by the schema in the sense
that you want your data to be always fit the schema,
and so there's some room for flexibility.
That's how you can depart from just traditional relational SQL databases,
but having a schema allows the storage system to understand what your data is.
And if you want to implement some of the more powerful operations,
like JOINs for example,
you need to have the storage system know about what your data is.
And so, a schema is really something which
the application developer is given
some information from the storage system about what the data looks like,
and that is helping the storage system make those decisions.
And if we removed that information,
it becomes difficult to act in storage system layer.
And so, I think having a schema is good.
But again, I think there is a way to
get the benefits of both having a schema and not having a schema,
which is not to shackle the programmer
with forcing in all that data to be fit exactly in the schema.
And there are ways to do this.
Like you can have blobs for example,
where you can just put structured data in the schema,
and the application is supposed to deal with the blob itself.
And that is already supported to some extent by SQL.
But I guess the bigger question is,
how do you make the database,
or the storage system do interesting work
in those blobs when it doesn't have any information about this?
And I think this is still an open question of how to do that.
Of course, you can always try to index stuff as being free text for example,
but you're still limited, right?
Ideally, you'd like to have the storage system do a lot of interesting work
with these unstructured data or schema that's data parts of the data itself.
But at the same time,
you don't want the application developers to
be constrained about everything needs to fit exactly schema, right?
And I think that's the challenge.
So, it looks pretty exciting for the entirety of Cloud computing,
but also key values to store our secret service systems.
Let me ask you something which is a bit of a speculative question, of course,
I miscontact your opinion,
might only be your opinion and not Microsoft's opinion.
What do you think Cloud Computing might be headed in the next say, 5-10 years?
What might it look like five years from now?
In my opinion, I think Cloud computing is the future of software right in.
So, now, in my opinion more and more software which
is not just something that you buy a shrink wrap from the software store,
but rather, something that you rent and runs in the Cloud.
You can see a little bit of that with,
for example, software for the office, right,
where you can do spreadsheets and you can do
a word processing and you don't necessarily or even mail, for example, right.
So, you don't buy or you could,
I guess, download your own mail client and that's still useful.
But there's a lot of things that you're not buying
these stuffs anymore because you're just running these off from the Cloud,
and the benefit of that is that you don't have to upgrade your software and then,
if there's any problems with the stored systems, for example,
you don't have to deal with fixing that,
it's just the Cloud that will take care of it.
And, I think, that's we're going to see more and more stuff running
the Cloud in the future as opposed to running your own machines.
And this is even true for
the machines that you're carrying in your pocket for your phones,
for example, there will be smart watches as well.
And those things don't have much compute power or storage power, and so forth.
And so, what's going to happen is that more
and more of those applications will be running part in
the Cloud and there will be
some intelligent user interface to actually just display stuff nearby.
And maybe a little bit of local computation as well.
But, I really think that the Cloud will be
the future of whatever software will be running in the future.
Of course, there's always going to be need for some types of software locally.
For example, in the study of car,
you have all of these embedded systems for doing ideas or for
doing drive by wire and that type of stuff needs to run inside the car and run.
Okay, that's true that there will be cases where there's local competition as well,
but I think, by and large most of the computation will be in the Cloud eventually.
You ask about five years.
I don't know if that's going to happen within the next five years,
but, I think, we'll eventually get there.
Last question, here, you've been at Microsoft Research for many years now.
What do you like about working here, just work environment,
the colleagues, and whatever the company provides you?
Because there's a lot of things to like in Microsoft.
And I really enjoy working here,
but having good colleagues is always excellent.
When you have this idea which you think is a nice idea.
You talk to your colleagues and they'll say, "No, this sucks."
And then you realize, okay, you're just
saving your time not working on these ideas because
there's all these XYZ things that you were not thinking about,
but based on their experience or
just based on having a fresh view of things they realize,
there are some shortcomings,
or they can say, "This is a good idea.
I agree." And then they can take
this feedback that you have from your colleagues and decide what to do with it.
And so, now, having great colleagues is a good thing.
And this is something we just don't find in
many other places not something which is specific to Microsoft Research.
Another great thing about Microsoft Research is the ability
that you have to work on the problems that you'd like to work on,
as opposed to having somebody tell you,
"Today or this year,
next year, you're going to be working on XYZ.
The notion that you can go to conferences,
and you can get inspired about new ideas,
and you have your own idea and work on that.
Or you can talk to colleagues and then,
you understand that their ideas have value and are very interesting as well.
But you have something to contribute there
and the side to engage and start working with them.
Now, the freedom to be able to do this is a very nice thing.
Third thing that I like about Microsoft Research is that,
there's a lot of resources available for research and so,
you can see Microsoft is sponsoring a lot of conferences when it comes to trouble,
for example, I can go to conferences and
I don't have to worry about who's going to pay the bill for these things.
And in terms of interns,
for example, we are getting interns to come here for the summer.
And it is no notion of shortage of these things in the sense
that there is only a few number of slots
available and the researchers have to compete against them.
There's just resources are available, right.
And I think, that's really important for research like if you don't have resource,
especially, if it's experimental research, right?
If you don't have the resources, it becomes very
difficult for you to make progress and definitely,
the Microsoft Research Resources is not missing at least not at this point.
Nobody knows what's going to happen in the future,
of course, but that's the idea in the current situation.
Thank you. Thank you for taking the time to speak with us.
It was my pleasure.
Explore our Catalog
Join for free and get personalized recommendations, updates and offers.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2018 Coursera Inc. All rights reserved.
