[MUSIC] You may have recently heard scary statistics about machine learning like just one out of every ten data science projects actually makes it into production. Or 87% of projects don't get past the experimental phase and are never used. With those kinds of odds, you may be inclined to run the other way. In the previous two videos, we talked about steps you can take to mitigate machine learning risks. With knowledge of how to build up your capacity for doing ML as an organization, using the five stages of adoption, a five-year planning horizon, and a strategy for broad level impact. Let's now talk about how to develop an experimental mindset and why that's an essential part of your machine learning strategy. An experimental mindset is an organizational culture that moves you beyond what you already do well to experiment with new ideas and directions. Experimentation is the best way to identify if your machine learning problem and data can drive your business objectives. And perhaps the idea in its original form won't yield the desired results, but as you build different models with your data for various uses and monitor the outcomes, you'll get a better sense for what problems and applications are most likely to be successful for your business. There are many flavors for how to approach experimentation. We recommend using a few steps to iterate through this approach, and let's walk through these steps together. To start, you need your prerequisites in place. This means having your organizational challenges and objectives identified. As we've said before, the aim is to align your machine learning applications with your strategy so that you're making headway in the direction that your organization most values. You don't want your qualms to be another statistics that didn't make it to production. Next you want to look for opportunities in the data and the machine learning knowledge and experience that you've already acquired or can easily access. Especially in the early stages it doesn't make sense to make an enormous investment. But this shouldn't prevent you from experimenting, these two things aren't mutually exclusive. To leverage your machine learning knowledge, ask your domain experts, your data scientists, your chief data officer, or your machine learning partners. Your goal with accessing this knowledge is twofold. First to generate ideas and second a sanity check to make sure the idea makes sense for machine learning. This early process will be one way of safeguarding that your experimentation doesn't steer too far in the wrong direction. Now start experimenting, keep in mind this is a learning opportunity. So pay attention and note what goes well and what doesn't across levels. Scientists, engineers, managers, and executives all have lessons they can take away from the experimentation. And the next qualm you build and deploy will be better for it. Lastly, is the evaluation of your results. As we just discussed, it's important to document or discuss with folks to understand the results of your experimentation, but this isn't evaluation, evaluation will happen in real time. But best is if you can sit with all the facts so that you have a broad sense of the development and deployment issues and you can use those to inform and improve how you go about executing your next idea. We recommend using a tool like the AI hierarchy shown here when you go to evaluate as it can help up identify where along the process you miss stepped. There may have been multiple steps where your idea went awry. It also helps emphasize the recommendation to build vertically, thus build out as quickly as possible and then build out horizontally. This is another way that you can think about experimenting. It isn't necessary to have all of your data cleaned before you start to experiment as the experimentation can be small scale especially if you're in the early stage organization. Once you found success with a small qualms, you can start to worry about scaling. And scaling will also be part of experimenting, we'll discuss this more in a future video. Given that we've encouraged you to experiment so that you can explore ideas that may be valuable for your business, we should also note that experimentation is not just about success. There will be failure and it is a necessary part of the process. Each time a project fails to reach deployment, you can evaluate why. It may be that the problem or data weren't quite aligned with your business objectives. So then you can consider how to adjust this experiment and try again. Or maybe the qualm answered the question you wanted and was well aligned with your strategic goals, but once you went to deploy it you found that you overestimated how effective it would make that process. This helps you improve how to generate metrics for your qualms. All these to say that you need to manage your expectations, it's unlikely that anyone qualms will be the gold mine. Instead, each experiment will help you learn and build your knowledge and experience so that you get better and better at using machine learning to drive your business goals. This process may seem obvious and intuitive, but it's difficult for many organizations as it can be easier and safer to just stay in your lane and not take any risks with experimenting. However, an unwillingness to experiment is the kind of decision that can make your business and its qualms go stale. So instead, consider what machine learning idea you could try in a short amount of time, say over the next two to three months, and just go find out what happens.
