Part of my job is knowing the difference between a real problem and a sensor lying to me, so a book about algorithms quietly standing in for judgment and getting treated as objective truth was always going to land with me. Cathy O’Neil’s whole point is that a model isn’t reality, it’s a simplified guess wearing reality’s clothes, and once enough of them get built by people who never have to answer for the wrong guesses, the ones without power get run over by systems nobody can even explain to them.
She’s not going deep into the actual math behind these systems, this is a superficial look in the sense that you won’t come away able to build one yourself, but that’s not really a flaw, the algorithms are opaque by design, and the whole book’s argument is that the opacity itself is the harm. Insurance pricing, teacher evaluation formulas, predatory ad targeting for for-profit colleges, the recidivism scoring that feeds back into sentencing, she moves through each one with a real anger under the prose that never tips into shrillness.
If you work anywhere near tech or policy, this is the kind of book you need more than the tenth book about writing cleaner code, because it’s the one that tells you what happens when the code you wrote quietly starts making decisions about people’s lives and nobody built in a way to appeal it. My one complaint tracks with what I’ve seen other readers say, the chapters cover a lot of the same ground across different domains, and by the fourth example you’ve got the pattern down, but O’Neil clearly wanted the sheer accumulation of cases to make the point that this isn’t one bad actor, it’s the whole architecture. It worked on me. I’ve been side-eyeing every automated system at the plant a little harder since.