So let's compare a little bit about, all right, so maybe you don't have Google BigQuery yet. Maybe you have something that's on-premises. And why you should potentially move to the cloud, Google Cloud platform, over potentially an on-premises solution that you have. So storing data and storing objects on Cloud Storage is is super cheap, as you can see. And again, as we mentioned in the New York City cab example you can focus on building awesome queries and wowing your audience, instead of wowing your electricity bill by paying for rooms and rooms full of servers. And again, I joke about this stuff, but it's only in the fact in the last like, you know, 10, 15 years that a lot of this has become available. That you as a data analyst can leverage the power of literally, just planet-scale data centers that Google has has built for its own operations and then made available through the Google Cloud platform to you all. And that speaks to the point of massive scalability is it's built to Google scale. So this is just a very fun graph to show. If you're trying to process even just gigabytes of data back in the day, back in the in the 80s, you're looking to pay upwards of $100,000 per per gigabyte. Whereas now it's $0.05 to equivalently buy one of these hard drives online. And of course, you know, buying terabyte hard drives now is, consumers can buy them now. But again, that's not the only piece that goes into processing big data. Yes, storing data is fantastic. You need to have a ton of storage. But that's not the only piece to actually process big data. So what are the other pieces that are involved? So you got your hard drives, great, you're sure storing the data. But if you're going to ask, you know, computation questions like say, I wanted to do the process, how fast those New York City taxi cabs are moving. So I need to do it like the time over the distance. And it's going to be a computation I need to run. So I've got to have some kind of processing power that I can then attach to my hard drive. So, okay, I need a server. And then okay, well, in order to talk to these servers, we need some kind of networking. That's that step three there. It needs to be fast. But if my data analyst team is in London and my servers are located somewhere else, say home base is somewhere else in the world. That speed to reach that data could be throttling our network as well. Not to mention the army of folks like database administrators and server technicians as part of step four, that you need to actually maintain this massive amount of growing infrastructure that you have as well. So why do it, right? So here is another kind of scary pie chart. So you run in the door as a data analyst, now you run the door as an analyst. And you really want to spend your time creating those insights. But at the same time, it's a small shop and you have your own cluster that you need to maintain. And as part of that, you've been drafted into the apprenticeship for your hardware team as well. Because in order to maintain the awesome querying on the boxes that you have, you need to also monitor them. in case hard drives fail. You need to performance tune them for specially written queries or jobs that you have running. Any type of software updates that comes out, you need to manage and install those, make sure they're all backwards compatible. Provisioning of the resources that you have. So say you have a marketing team that also wants to query using your cluster as well, but you have a data analysis team that takes higher priority. You have to manage all that as well. Not to mention that say next year, you did such an amazing job that your CEO wants to say okay, you guys have done phenomenal as data analysts. We're going to give you 10 times or 100 times more data. And you'll need to process that, as well, with your growing and growing infrastructure. So it becomes a battle again, of growing of that infrastructure versus growing deeper into more and more sophisticated analysis techniques and things like building machine learning models as well. So it becomes almost unsustainable to keep building and building and building and maintaining infrastructure if your core focus wants to be on data analysis. And what the benefit that you get, of course with the Google Cloud platform is you don't have to focus on infrastructure. If you want to write amazing queries as you saw, you just write and learn some basic SQL. By the end of this specialization, you'll be proficient in SQL if you're not already familiar with it. And then you can just type that query into the query editor and click Run. And that infrastructure already exists, right? Speaking of infrastructure, this is what you're running it against. So this is not to kind of make this like a scary image, right? You don't have to maintain these servers, if you had to guess, right? What would this be? And I hope that the ethernet cables here are actually color coordinated with Google's logo. That would be pretty awesome. But if you haven't guess this, this is one of Google's data centers. So Google itself had to deal with processing sheer massive amounts of data. Google's mission is to democratize and get fundamental access to the information of the entire world, right, to organize the information of the entire world. And storing all the world's information, as you might guess, needs a ton of those resources, as we mentioned earlier, right, the hard drives, massive data centers, and everything that goes into that as well. So it's a problem that Google had to solve. And that's exactly what makes Google, Google. So this is the networking capacity, the fiber optic cables, all the servers and all the armies of engineers that make all of this run. And then, thankfully for all of us data analysts, Google has then opened that up through the Google Cloud platform as a service model. So if you wanted to leverage the power of Google's infrastructure, you can do so through the cloud. And that's the main benefit. So the cloud will run on Google infrastructure. Speaking of Google infrastructure, here's a very exciting map that I always like to show. So when I first joined Google and I saw this, I was like, wait a minute, you can't show that. That's the data centers where Google has all of its operations and that sort of thing as well. And thankfully, they've got good security there as well. But yeah, this is all public knowledge. So Google, in order to deliver these search results around the world and these funny cat videos for YouTube necessitated a massive planet scale network and operations to deliver this kind of data as well. So Google Cloud Platform is built on that. So if you wanted to do big data analysis and you had your analysts in London and you had your clients in Singapore and your data needed to be everywhere, in effect, then you could leverage Google Cloud platform and the scalable storage in your data centers and the network that it has, to make that all happen at scale. So here's kind of a general summary if we didn't scare you enough. You have to assemble your own hardware if you're doing it the old assembly way, right? You have to get your storage and your processing power and how much you actually going to store in memory versus persistent disk. And build up that great networking of yourself. Or you can use the mother of all data centers and the Google Cloud platform to make that happen for you. And really, the main point here is to get your entire organization to have almost no switching costs to get on to something like Google Cloud platform. And just really excite your analyst team and evangelize your folks who are interested and curious about data, but didn't want to install a bunch of things on their machine or worry about learning so many new different technologies, when they can just focus on writing simple queries and executing those inside of BigQuery. And that's the formation of this course as well, is the last step, is a little bit of knowledge in SQL and getting familiar with what some of these big data tools are. And once you have that familiarity, you can take that back to your organizations and basically say, hey, it's not so bad. Here are the tools. Here's some sample queries. Let's, you know, let's give it a whirl. One of the key words when it comes to talking about Google Cloud platform, of course, is that scalability. So here again, if you did it yourself and you wanted to buy the servers and manage the infrastructure, you could run into two potential issues here on the left. So on-premise, you could have over capacity or under capacity. So the first one is you don't have enough machines to process that amazing query. So you're running that New York taxicab 14 gigabyte query and you have a small machine and you're going to be sacrificing time. So it didn't cost you as much, because you only have one machine. But you're going to sacrifice time. So it's going to take much longer to process that query. Or say you want the opposite, say you blew your entire I.T. budget for the year and you've bought the best of all machines and you're paying the electricity cost to run those machines and the updates of the software, and the initial capital expenditure outlay to get that. And your query is mega fast, right? So your query processing time is really fast, but you're still paying for the massive amount of computing power. And the interesting point to note here is if you have a server and you have your storage, your hard disk platters, your persistent disk and your CPU all in the same box right, that's a server, you're paying for electricity to the CPU even if you're not using any queries, right. So you're still burning electricity just to keep that data stored on a persistent disk to keep that data alive even when you're not using that for querying. Contrast that with the Google Cloud platform on the right and the resources are there when you need them. And they go away when you don't. So you process a query. Google realizes, wow, this is a mega query that you're processing, which is awesome, right? So you've written some really complex analytical query. Google will automatically ramp up these cloud virtual machines behind the scenes. And then when your query is done, it'll ramp them down. And you only pay, in BigQuery's case, for the amount of data that you're processing. Fully managed behind the scenes, you're not setting up virtual machines and processing this data yourself. All this is handled behind the scenes and you just need to write a little bit of SQL and run that in the interface. So I mentioned kind of in passing, one of those key enabling features of Google Cloud platform, and that is the decoupling of storage and compute power. So on-premise, both those are co-located right on that server. Within Google Cloud platform, if you want to store data, but you don't want to process it yet, say you want to store in stage 10 petabytes of data. On-premise you'd be dumping that into many, many, many, many, many hard drives and paying for the electricity costs to store all that. Google Cloud platform, you're just paying for what you use. If you want to store it, great, awesome, it's there. Whenever you want to run compute power on it, then you're charged again for the amount of bytes that you process. But storage and computing power are two separate concepts which enables you to optimize for both. More efficient resource allocation is the key take away. This is the first key takeaway, is that BigQuery in particular, will scale automatically to meet the demands of your query, and you're just going to pay for the amount of querying data that you process, and if you're storing data on BigQuery, the amount of storage on a persistent disk that you're going to actually store behind the scenes. But again, the key benefits here, that it's fully managed. You don't have to worry about the replication of the data behind the scenes. You're not paying for the electricity to keep the data centers running, or for the new software updates for the latest and greatest version of a database software that comes out. All those updates are coming to you automatically from the BigQuery team.
