In this lesson we will discuss two products that are very helpful for data scientists. Both came to IBM with the SPSS acquisition in 2009. First is IBM SPSS Modeler. Let's review the different tool categories we discussed previously. IBM SPSS Modeler includes data management capabilities and tools for data preparation, visualization, model building and model deployment. The product was created by Integral Solutions Limited in the United Kingdom in 1994 and was originally called Clementine. It was acquired by a company called SPSS in 1998 and SPSS was in turn acquired by IBM in 2009. SPSS Modeler is a data mining and text analytics software application. It's used to build predictive models and conduct other analytics tasks. It has a visual interface that enables users to leverage statistical and data mining algorithms without programming. One of its main goals from the beginning was to create complex predictive modeling pipelines that are easily accessible. A sample modeler stream shown here includes one round data source node, three triangular graph nodes, one hexagonal node for computing, a new variable, and a square node for an output table. Below the canvas, we can see the rich node palette with separate tabs for data sources, record in field operations, graphs, models, output and so on. Nodes and different tabs have different shapes with Pentagon's used for modeling nodes. Let's examine the sample stream that comes as an example with the product. It starts with a data set of telecommunications records and the goal is to build a model to predict which customers are about to leave the service otherwise known as churn. The data source is shown by the round node on the left side, a hexagon type node typically follows a data source node and it enables us to specify roles, target predictor or none. And measurement levels such as continuous nominal or flag for all variables. The term flag is used to denote a variable with two categories one of which can be considered positive and the other negative. In this example the measurement level for the churn field is set to flag and the role is set to target. All others are set as predictors and inputs. The original data set has many fields and some of them are not relevant to the target variable, so we first need to decide which fields are more useful as predictors. There is a feature selection modeling node that helps to do this. After the stream with the feature selection node is executed a yellow model nugget gets created below it in the flow diagram.Using that nugget we can generate a filter node that filters out the variables that are not good predictors for the target. The data audit node located below the filtering node shows various properties of the data such as numbers of outliers in each variable and the percentage of valid values. It can also help to create a special node for missing value imputation that is replacing missing values of a variable with some valid values that can be selected based on domain knowledge. Here variable log toll has greater than 50% missing values and we will specify a value the mean to replace them. A super node in modeler is a special node that is not found in the palette but is created by the user with special functions included in it. The data audit node enables us to create a super node for imputing missing values. It is shaped as a star and shown on the right of the screen. Finally we attach the logistic regression model node to the stream and click run. Another model nugget appears and by clicking it we can see various model information and other output. in the output window that opens when we
click on the model nugget the summary tab shows the target inputs and some model building settings. Based on certain advanced output settings that were specified before the model was built we can also see a classification table, accuracy, and some other generated outputs for the model. Note that these results are based on training data only. To assess how well the model generates two other real-world data you should always use a partition node to hold out a subset of records for the purposes of testing and validation. Then, in the model setup screen select the use partitioned data check box. This will help detect and avoid model overfitting. Overfitting is defined as having significantly higher accuracy on the training data. Data used for training the model then on tests or unseen data. The yellow model nugget added earlier can also be used to compute predictions, also called scores on the original data or on a new data source. All we need to do is to connect the data source in question to the nugget, make sure it has the predictor variables used in the model, and create an output to a table or other structure for storing the scores. We can also specify settings for scoring inside the model nugget. Note that if the model was built on transformed predictor data, the same data transformation steps would be applied to the new data before it can be scored by the model. The analysis node is the final node in the stream. It attaches to a model nugget and when executed it will compute some model evaluation metrics, auch as a confusion matrix and accuracy. In this example we've only looked at a logistic regression model. IBM SPSS Modeler offers a rich modeling palette that includes many classification, regression clustering, Association rules and other models. It also contains large selections of data source types, data transformations, graphs, and output notes. And we haven't even talked about text analytics, entity resolution and many other features of the product that can be extremely helpful to data scientists. We could create an entire course on IBM SPSS Modeler alone. You've learned how IBM SPSS Modeler helps analysts to create powerful machine learning pipelines using graphical interface. Next, we will talk about the original SPSS product now called IBM SPSS Statistics.
