Welcome back. In this video, we will discuss technology and the idea of mistakes. In designing technology, we have a problem in the world which we are trying to solve, such as identifying individuals who need therapy and the phenomenon of interests that needs to be translated into an outcome variable. The first step of any program is to create an outcome that measures something that you are interested in. But the outcome is actually constructed based on the phenomenon of interest, what we're really worried about or interested in the world, as well as what is easily measured and labeled by a program with AI and data analytics. Accuracy is often used as the measurement of working data program, where accuracy is how well the program predicts or categorizes people. When measuring how a technology's accuracy, companies often report the number of true positives or how many of those who cheat are caught by a cheating detection program. Here we are also worried about the proportion of mistakes, those false positives and false negatives. False positives, or when we label someone positive but a wrong, whereas false negatives or when we're mislabeling someone and we should have. The types of errors, false positives versus false negatives, are often not morally equivalent. For given time text, what types of mistakes are important? Which types of mistakes are morally fraught? For example, a false positive in the criminal justice system, falsely finding someone guilty is more morally problematic than a false negative because we have a presumption of innocence in the United States. However, which type of error is preferred is based on the context of the decision. In addition, the meaning of accuracy can vary from project to project, attempting parody across types of mistakes or in reporting confidence intervals, where all types of mistakes are reported as similarly important, could mislead someone as to the ethical implications of the data analytics program. Accuracy for the majority, even if meaningful, does not mean a program is accurate for all groups. Programs be more or less accurate for a particular type of mistake for the same groups, for example, facial recognition in white men and not others, for example, facial recognition in black women. For some programs, however, the question as to the outcome of interest and how to measure if the program works is more ethically salient. For example, in predictive policing, law enforcement uses a program to predict if a crime is likely to occur in a particular neighborhood. Police can then deploy officers to that area. A popular predictive policing program, PredPol, focuses specifically on petty crimes and misdemeanors in the hopes that arrest for a petty crime may reduce felony crimes. However, the program predicts crimes based on training data of where people have been arrested in the past. A common complaint is that these programs are based on bias data and therefore predict in a biased manner. Predicting policing also exemplifies the ethical questions about creating outcomes and measuring accuracy. When petty crimes are the outcome of interest for PredPol, focusing on misdemeanors is not always the way to decrease crime. Petty crime does, however, provide more data points and would therefore show more impact if one is to be found. Second, in measuring mistakes in this situation, what is the preferred or morally problematic type of mistake? Mistakenly over-policing an area with low crime or under-policing an area with high crime? Which mistake is the program more likely to make? Third, how would one measure if such a program is accurate or works? When the prediction leads to immediately to action such as officers being deployed, measuring accuracy becomes tricky. We will call this creating accuracy. We face the challenge in prediction technologies in creating accuracy, where individuals with a particular outcome variable are treated differently than those with a different predictive score, making measuring accuracy very messy. In other words, when we measure and compare the end state of those labeled and those not labeled, is that comparison, comparing similarly situated conditions. Predictive analytics runs into the possible problem of creating accuracy and categorizing someone with an outcome variable: promotable, hearable, trustworthy, high likelihood of recidivism, etc., which pushes the individual into a course of treatment that then creates the outcome pre-created by the program. The case of predictive policing is perhaps the quintessential example of creating accuracy. Identifying a particular neighborhood as possibly being more likely to have petty crimes, the prediction, then leads to more officers descent to look for crimes in that neighborhood. The treatment, a larger number of officers finds more crimes and arrest more people. This is the standard argument as to how predictive policing can feed into creating a new reality and create the perception of being accurate. The case of Chicago's predictive policing program isn't even more stark example of creating accuracy. Robert McDaniel was visited by Chicago police and told that the Chicago Police Department predicted, based on his proximity to a known relationships with shooters and shooting casualties, that McDaniel would be involved in a shooting. He could either be the shooter or he might get shot. McDaniel was also told that the police would start watching him. While McDaniel had no violent history, he was suddenly under constant surveillance by the police. The data analytics program had made a prediction. The police department was the immediate treatment. However, this increased attention and visits by the police looked suspicious to those in his neighborhood who thought he was then working with the police. McDaniel was then shot twice by those in his neighborhood who believed he was a snitch, given the amount of police attention he was receiving. In this case, the program quite appears so accurate because the prediction was that McDaniel would be involved in a shooting and he was shot twice. This program problem occurs frequently in predictive analytics. At a parole hearing, courts may label a prisoner with a higher recidivism score, predicting they're likely to commit more crimes, for example, campus in the last module. The courts then place additional requirements on those paroled prisoners where they can live, drug tests, how frequently they need is check in, who they can live with, etc, which make them more likely to commit a parole violation. The issue is similar to measuring the treatment effect in medical research. Only a medical research we are actually interested in how effective the treatment is. There's an incentive to measure its effect. Here for some companies, developing a data analytics program, measuring the treatment effect means acknowledging that the data analytics program is perhaps not as accurate as we once thought. The more impact that treatment has, for example, sending more police to a given area, the less the data analytics program actually works. Even when not making a mistake, data analytics predictions that require a particular treatment, for example, predicted likely to succeed and given additional help, predicted likely to be shot and given additional scrutiny, can undermine the individualization of the subjects and can harm the dignity of not being seen as an individual. The deindividualization of the person is a tendency to judge and treat people on the basis of group characteristics instead of on their actual own individual characteristics and merits. In other words, the prediction and immediate treatment of someone as a cheater, a fraud, a criminal, not hearable, based on the possible characteristics they share with others, is seen as undermining their dignity as a human. The idea that in some circumstances, people have a right to be treated as an individual. Virginia Eubanks refer to this phenomenon as the flattening of the individual to a set of data points. Is there an area where we should not make decisions because people like you are never successful or get into trouble? Therefore, the examination of technologies would need to question the creation of the outcome variable, the measurement of accuracy, including the types of mistakes in which are preferred, and the possibility of creating accuracy through the program. See you soon.
