AI models and biases

Max
2 min readFeb 24, 2021

Chapter 1 of Kathy O’Neil’s book, “Weapons of Math Destruction” discusses the use of statistical models on different fields. She begins by discussing a discrete, open field that heavily relies on models: baseball. Baseball is very commonly used to train one’s statistical modelling skills for two reasons: all MLB data is open to the public (anyone can find and use the data to be implemented into their model) and baseball statistics are discrete (there are distinct, discrete actions for each part of baseball that could be reasonably implemented into a model). It is further explained that models such as these can transform, but not necessarily cause to flourish or wither, previous fields that heavily relied upon gut instinct. These fields take upon add a new, extremely powerful form of analysis that is, fundamentally, by way of calculation, able to provide more precise and accurate steps to achieve some defined goal (winning games, in the case of baseball). But, to use a cliched phrase, with great power comes great responsibility. The blind spots in the incomplete translation of apparent factors and apparent goals to their software description lead to unfortunate model biases. An example O’Neil uses is the Washington D.C. school system and their model on teacher expectations. She explains that the model used to evaluate teachers used only parameters that could be translated to cold, hard data, such as test scores. This, of course, leaves out a important factors like: “how much the teachers engage the students, work on specific skills, deal with classroom management, or help students with personal and family problems” (26). These factors would be almost impossible to translate into a model so this is an example of guaranteed model bias and one in which a statistical model should be of at least non-job-determining use. Yes, while its important to rely on personal accounts in the case of teaching, data can still be used to maybe at most raise questions, however it is purely an ethical question at that point.

--

--