Human Detection Software
We need to talk about where training sets come from.
Dear [Redacted]:
So, I have an idea for a photo essay. It’s actually pretty straightforward.
I’ve been hesitant to share these images until recently. This is one of those things where… I questioned my perception, when I first noticed it. So I tried and tried a few more times. And what I saw did not change.
They’re literally photographs of invisibility.
Here are three albums.
The first album is from last spring. It shows a makeup display in my local drug store. I noticed the shades of foundation they were selling covered a remarkably nice gradient, so I tidied up the display to make sure everything was in order and took a couple pictures. And while I did, I noticed something weird about the facial detection software in my phone. I took a bunch of screencaps at several angles to try to capture what I saw. It left me chilled.
There are a few things I want to do with comparing the gradient of makeup colors offered, with those actually represented in the display, and the apparent rate at which the bottles have been selling out. (I think I can do all of this with just GIMP and Nearest.)
There are clearly a few things contributing to the problem, here. For one, the level of vibrance in the different models’ makeup is highly varied. The post-processing has led to a highly uneven degrees of contrast and white balance. And I was especially struck by how the smaller and odd-angled portraits registered just fine.
Whoever directed this art layout was comfortable with all of this. I didn’t see anything wrong, myself, until I saw the display through my phone’s eyes.
Over several months, I saw the same thing again, manifest in a couple different ways. And about a year after I first saw it, I saw someone else talking about the same thing.
The next album is from November. It shows me, getting, uh, very realistic mid-process fitness results, or probably just deciding to start working out again. (I take a lot of “during” pictures. I barely believe in “before” or “after.”) And again, I noticed the weird thing.
I take very few selfies. To be honest, I didn’t realize that facial detection was supposed to be a stock feature on my phone that I wasn’t getting, until… until this.
If you look closely at this sequence you can see how my expression goes from neutral, to What? to Not this again, to Yeah, this again. This is what it’s always like: I’m minding my business, and something slightly off happens, often having to do with an apparent oversight. And I check myself, because I have normal-person levels of humility and self-doubt, and I like being thorough.
That’s the hardest thing about being socially isolated while trying to do unusual things, and encountering systemic problems. You tend to check yourself. To hold back, bite your tongue, seek a second or third opinion.
But… it’s usually not just me.
The last album is from the cover of a book. I took a few shots of it in April when someone asked me for advice on how to support their nephew, who’d been experiencing very cruel racist teasing in his gymnastics classes. I gave them some pointers for being supportive and recommended a couple books, including this one. And again, I noticed the thing. That person happened to have solid knowledge of computer vision systems, so I reminded them that they know how training sets are constructed, and how data validation processes are actually conducted, for machine learning models. They seemed deeply moved. For a moment.
These are not the same photos I took in April. I took these more recently, and under two different kinds of light, just to be sure. Under these multiple trials under varied conditions, I could not get the face of the child in the lower-right to register as human, even momentarily.
Finally, there’s one picture of a page out of a sticker book that my phone thinks contains a face, sometimes, but not consistently enough for me to screencap it. I’m not sure what it’s seeing. There are two blobs with a bigger blob underneath? It doesn’t look anything like a face to me.
One evening while I took pictures of an art project, I noticed that my phone kept granting this page of stickers a flicker of human recognition. That gave me pause. According to the societal systems that put this cameraphone in my hand, this page of blobs is more personable than my actual smiling mug.
What is its target? What is it actually going for?
This software has only one task: detect human faces. This isn’t even a sophisticated task compared with like, identifying individual people in a crowd, or discerning between people with similar hairstyles or eyeglasses. And it Does. Not. Work. And apparently, similarly crummy software is basically ubiquitous. (video, 8 minutes, kinda mindblowing.)
Fully half of the people on this planet are a deeper brown than I am. We would all like to register as human, with our faces.
Yet we don’t.
This is a kind of context that I cannot describe with words.
The is the closest I can get to capturing the feeling that I encounter in the course of my work, on a nearly daily basis.
It transcends and comprises morality and intention. It shows the priorities that are baked into every layer of our society. This is not a single person’s mistake or malicious deed. Like the cumulative effect of heavy-metal contamination in groundwater: it comes from every direction.
Unspoken trends in photography (including the choice of chemical processes for film), graphic design, data collection, feature selection (seriously… what is up with the feature selection for this model?), as well as social processes reinforce these issues.
Nobody important enough is affected.
Even if someone does notice and point out the issue, they are routinely told “I don’t think that really matters” or “That isn’t really a problem.”
This isn’t a deep learning problem that I can really resolve as an end user. But, as an end user, I can describe what I see and try to submit decent bug reports. As a tech professional… well, I’m in a bit of a bind.
More than a few times, my clear and actionable statements regarding process validation have been met with a vague justification for how business value is an excuse for ignoring what data says or for falsifying it outright, and that my lack of experience is the only reason I don’t understand this.
Not long ago, an old friend told me that I must have performed a sampling error when I identified a particular event as pertaining to racism. [Redacted], I was not amused. That person is not a “math-person.” I’d already put more time than it was probably worth into correcting some egregious statistical buffoonery they’d posted on Facebook in previous months, such as a statistical analysis that used a weird comparison of linear regressions to “prove” that some process or other made no difference to COVID infection rates, when a basic t-test was what was called for. Since I was talking about a hand-selected one-off event, the notion of a sampling error didn’t even apply. Also, the incident in question was totes racist and actually very damaging.
Not long before that, a newer acquaintance told me that all I had to do to get more recognition at work was promote myself harder. I asked how to promote myself harder when the challenges I faced included being categorically prohibited from committing my code to remote, or “showing people the things [I] know about.” He didn’t have an answer.
These photos of invisibility are basically synecdoche for why I continually feel like I’m being pushed out of tech, and why I’d want to give up.
Let’s contrast:
To be fair, my phone is not the newest model. And Google Photos recognizes more faces in the photos I’ve shared here than my phone’s camera does. The various implicit definitions and thresholds of humanity are, slowly, becoming more inclusive.
Now, let’s compare:
Here’s a human-layer version of the exact same thing, in the context of climate activism, earlier this year. This isn’t a technology bug; it was someone’s choice, and it was based on the same priorities. The justifications provided in this instance are interesting in their ultimate stupidity, if you don’t mind my using such strong language: the background was too distracting. The active discounting of the salience of an African activist’s participation in a summit on climate change held in Africa is especially conspicuous.
This phenomenon is like, the enemy, if I were ever to conceive of one. An anti-goal. It’s utterly impersonal. And it wipes me out. This is erasure.
I don’t know what I can do for you, to enhance and enrich your life. But I can tell you pretty confidently that technical proficiency isn’t what’s been holding me back, nor has it been lack of passion. And, I can tell you that I feel like I actually have less technical skill now than I did two years ago, because these multiple forms of systematic erasure have diverted almost all of my attention. This isn’t the product of some kind of narcissistic delusion on my part, though I almost wish it were. In fact, it is a sorry waste of my precious time.
These are real, deep, large-scale people-problems. I’m doing what I can to move through them. I want to have fun making things with computers, and dive deep into artificial minds. I want to use computers to help people learn, at scale, so that we can all become better informed, better at introspection, and become fairer decision makers. That’s what I hope we might do together.
Are you in?
[This was originally an email I wrote to a specific person. He explained to me that the problem I described is real, but too big and the pushback against attempts to change it too strong, for him to offer me guidance or support at the time. This edited version includes references from other correspondence with him, and alterations for style and flow. If you know where I should direct this kind of inquiry for more efficient consideration, please let me know.]
Update, October 2020: the individual I originally wrote to got back to me after a week or so and gave me some very thoughtful advice on how to avoid pitfalls in my career. He said, “Don’t go after the big money,” which I already knew. Shark tanks are not my thing. And “Don’t try to work on the big machine.” That was a new framing for me, and I really appreciated it. It also helps that he acknowledged the vastness of the bias that is baked into our representations of the world, including the language we use and the models we build from it: conceptual and semantic shortcuts that conceal huge swathes of reality and humanity from our consideration. It’s measurable, and not just a matter of opinion.
