Weapons of Math Destruction
by Cathy O'Neil
I was pretty cynical about this at first as I thought I’d heard it all before. I think this should be required reading for anyone working with data. I can’t shake that the injustices created by the use of data as described in this book are just the tip of the iceberg.
Just like with Humankind’s homo puppy I winced at the use of the WMD acronym, a purposefully overloaded term.
Odds & Ends
Three factors to systems deemed “Weapons of Math Destruction”
- Opacity: Are the rules of the system transparent to those that are being judged and analysed?
- Scale: Does the model have the capability to grow exponentially?
- Damage: Is the model unfair? Does it have a “pernicious” feedback loop that only makes it more unfair?
Clopening is when a worker is scheduled to both close a location at night and then re-open it in the morning. This can be stressful and result in an erratic schedule for workers balancing a number of responsibilities.
A large amount of Facebook users believe the algorithm behind the scenes is just presenting factual information and isn’t tailored to them.
In 2013, when a University of Illinois researcher named Karrie Karahalios carried out a survey on Facebook’s algorithm, she found that 62 percent of the people were unaware that the company tinkered with the news feed. They believed that the system instantly shared everything they posted with all of their friends.
I wonder if this is still the case eight years later?
O’Neill uses the phrase “birds of a feather” a few times to discuss how members of the same social network will often behave in similar ways. One example is that they may click on the same ads on Facebook.
Some recidivism analyses use this concept and inadvertently encode bias by taking acquaintances, jobs and credit rating to predict behavior. This sort of data would be inadmissible in court.
Interesting dataset alert:
A few years ago, MIT researchers analyzed the behavior of call center employees for Bank of America to find out why some teams were more productive than others. They hung a so-called sociometric badge around each employee’s neck. The electronics in these badges tracked the employees’ location and also measured, every sixteen milliseconds, their tone of voice and gestures. It recorded when people were looking at each other and how much each person talked, listened, and interrupted. Four teams of call center employees—eighty people in total—wore these badges for six weeks.