This week we will begin with module 2 - Carrying out classification techniques. We will learn how to apply the steps we have discussed in the Module 1, to a real-life classification problem. Firstly, business problem will be defined. We will learn how to apply one of the classification algorithms, namely, classification tree. Various steps are defining business problem, deriving data mining problem from the business problem (i.e., what is the output variable we need to predict), identifying input variables, dividing the data between training and test data, building a model using training data and applying the same on test data, calculating error rate from test data as well as business impact of the same. We will also discuss the concept of overfitting (what is overfitting, why overfitting takes place, how to reduce overfitting) validation data and cross validation in context of classification tree. We will also learn how to install and apply two different tools, R and Orange, for carrying out the solution. Peculiarities of different tools, one having mainly command based tool (R) and other having advanced visualization (Orange), will be discussed.

This week we will begin with module 3 - Carrying out regression techniques and discussing various business applications of classification as well as regression techniques. We will learn how to apply the steps we have discussed in Module 1, to a real-life regression problem. Firstly, business problems will be defined. We will learn how to apply one of the regression algorithms, namely, multiple linear regression. Various steps are defining business problems, deriving datamining problems from the business problem (i.e., what is the output variable we need to predict), identifying input variables, dividing the data between training and test data, building a model using training data and applying the same on test data, calculating error rate from test data as well as business impact of the same. We will also discuss the concept of overfitting (what is overfitting, why overfitting takes place, how to reduce overfitting). Various error measures related to regression, and their interpretation will be discussed. We will apply two different tools, R and Orange, for carrying out the solution. Various business applications involving classification as well as regression will be discussed.

This week we will begin with module 4 – Target Marketing. Many a time, the cost of sending an offer is a substantial part of the gross profit. Hence to maximize the profit, it is imperative to send an offer to the selected audience (target audience) that is more likely to buy the product / service) rather than send the offer to everyone. We will start with a technique that was used for target marketing prior to datamining. We will then discuss how datamining is used for the same. We will discuss how to define target marketing problem, how to carry out the same, tools and techniques of evaluating the solution, using classification as well as regression as appropriate. For classification problems, we will introduce Logistic Regression algorithm, while for regression we will use multiple linear regression. We will apply two different tools, R and Orange, for carrying out the solution. Missing values and outliers are two issues that one has to address for cleaning the data before we can apply the solution. We will define what are missing values and outliers, different ways of handling the same and then usage of tools; R and orange to address these.

This module will discuss one of the unsupervised techniques, namely, Association, which is more popularly known as Market Basket Analysis. Here one does not predict any variable (as in the case of supervised learning) but finds out which products / services are bought together / used together, at same time or within certain timeframe, more often. In other words, one finds out what is the strength of the association between various products / services. This module will talk about various parameters that define the strength of the Association, tools and techniques of carrying out Association. It will also talk about various business applications, standard as well as non-standard, in different scenarios: Offline grocery shopping, online grocery shopping, buying from e-commerce sites, predictive maintenance, designing websites. We will also learn how to use R and Orange to address the Association problem.

This module will discuss one of the other unsupervised techniques, namely, Clustering. Many a times, building one model for the entire population (customers, organizations, etc.) may not work out well, in the sense that models will have unacceptable level of errors. This happens because all the records in the population could be very different. Hence one will first divide the entire population into various clusters (or segments or groups), each cluster having more similar records. Clustering is thus about dividing heterogeneous population into clusters, so that records in a given cluster, number of records are more homogenous. It is an intermediate technique, in the sense that further work on some clusters need to be done to derive business benefits. Typical applications are about customized treatment at cluster level. One of the most important application is market segmentation, where single strategy may not work out for entire population, but different strategy for each cluster might give better results. We will talk about two different techniques for carrying out clustering, namely K-means clustering and Hierarchical clustering. We will discuss the concept of Distance Function, because it at the core of clustering. Records belong to same cluster, will be nearer to each other (relatively smaller distance).We will also talk about business applications of clustering. We will use two tools, R and Orange for carrying out clustering. This module will also briefly talk about dimension reduction, i.e., reducing the number of input variables (without adversely affecting the benefits or having even better results). Mainly, Concept of PCA (Principal Component Analysis) will be discussed.