We will give a high level overview, of the concepts we have covered in this course related to Design of Experiments for Improvement. We will also mention some topics we did not cover. The area of experimental design, is fairly broad, and the concepts we omitted, can quickly become very mathematical. But, if you choose to study these topics on your own afterwards, you will see that they build on the ideas we did cover in this course. You now have a solid foundation to base your self-learning on. All further, advanced experimental tools build on these concepts. We started the course by looking at some basic terminology: factors, outcomes, variables, low and high levels, and so forth. Very quickly we learned how to interpret - visually - the results from an experiment. And that was crucial; a visual interpretation is so important, and not having to run and rely on software. This is a theme we've seen throughout the course. We've resorted to visual tools the entire way. Cube plots, to visualize the results; Pareto plots to identify, or screen out factors, that appear important and those that are not. And finally, in the response surface module we looked at contour plot, to visualize the surface we are moving on. We also learned along the way, at several point how NOT to run an experiment. Changing one factor at a time, is something we have known for at least the last 8 decades as being inefficient, especially if we want to learn about and exploit interactions to reach that optimum. We learned how to set up our standard order table, to assist us. There are 2 to the k experiments in a full factorial. And once that full factorial was run, we saw how to manually create a simple prediction model. Remember that "high minus low", "high minus low" idea? No software was required. So by the end of the second module we saw that experiments often had more than one outcome. We might want to decrease pollution as much as possible, but also do so cheaply, and obey safety, or regulatory constraints. In systems where this is the case, we must either reformulate our objective to include multiple outcomes - maybe by using a weighted sum, for example - or by visualizing the overlapping, competing criteria on two contour plots. This visual approach is, again, very effective. We can see the trade offs in our system, and communicate with our colleagues effectively that don't understand the terminology of response surfaces and optimization. Furthermore, if things change in our system, we can quickly see how to compensate for them. Now in the third module of the course, we started using software, to speed up our hand calculations. We used a high quality, freely available tool to do that. The R software has many packages available to extend its functionality. But there are though other software tools, and some that are specifically designed for experimental analysis, feel free to download their trial versions and test them out for your own needs. We liked R, because of its traceability in the code. We can always go back, and reproduce our results. See where we've made mistakes and even share that code with our colleagues. You might be wondering about formal statistical tools that you might use to make your work more analytically: such as p-values, confidence intervals, analysis of variance, and so on. These are absolutely available, and have been there all along in the R output. As you've seen, we've been far more reliant on visual tools in this course, and less so on detailed statistical knowledge. In the fourth module of the course we started to look at fractional factorials. We use these when we have a large number of factors, and want to practically reduce the number of experiments to some lower value. We know that there's no free lunch, and that aliasing will occur. But we have this trade off table to help guide us in that choice. We learned about blocking for nuisance factors, and we also covered the idea of covariates in that fourth module. I had also mentioned the concept of definitive screening designs, which are emerging as a more effective design than fractional factorials. Perhaps this is a good time, to mention the book by Peter Goos and Bradley Jones. That book starts where this course ends. It's a great book, written in conversational style, that would help you peer into the minds of statisticians as they actually plan complex experiments. They cover topics, that many of you have asked about: response surface methods with categorical factors, screening designs, mixture designs, blocking and covariates, as well as the very practical requirements of a split plot design. Those are important topics, in practical experimentation. But, they go beyond the level we have intended for this course. Those topics build on the concepts we have covered though. Then we started the last module of this course: experiments to move outside our region where we started, and seek out an optimum. We initially looked at the single factor case. Mainly because we can easily visualize that, and illustrate the important concepts of noise, model prediction error, lack of fit, and building and rebuilding the model as we go. We applied those concepts to the idea of optimizing in two dimensions, and we saw a sequence of videos on the details on how to go about that in the fifth module. Even though those last videos were long, they covered some digressions, on the practical aspects of dealing with constraints and making mistakes. Now the response surface idea, expands in a natural way, to the case of three or more variables. We can also bring in the idea of fractional factorials. This will reduce the number of experiments required. The only thing to be aware of is aliasing. Because, remember, as you approach the optimum, it is those interactions and quadratic nonlinear terms, that will start to dominate. You need sufficient resolution at the optimum, and a fractional factorial may not provide that for you. Now this course does not end here. On the website there are some practice problems to try out. I've also posted a list of resources that relate to the area of designed experiments. If you come across any others, please share them with your fellow students, and make a short post in the forums, or email me. We'll keep that list up to date. As you might start to realize now, the topic of Designed Experiments spans into many other application areas. Also keep posting in the forums about how you've used Designed Experiments. This is a topic that applies to many application areas, which is one of the reasons why we chose to teach this course. So this is the end. I will thank specific people in the credits that follow, but by far the biggest thanks goes to many of you, on the forums, both the current and prior students in this course. Your participation and questions have lead me to learn so many interesting ways of using and applying experiments. I have made improvements to my own life, and career because of it. Thank you also for your feedback. We keep collecting your suggestions, and we use them for future iterations of this course. We do ask that you take a few minutes and fill out our final survey. There are two simple questions we would like you to answer, and there are a few other optional questions if you have time. So, thank you again for your time and effort. Remember to keep disturbing, and observing.
