NEW YORK – When Tumblr announced last month that it was banning porn from the platform, Tumblr CEO Jeff D’Onofrio was upfront about the fact the execution of the ban wasn’t expected to go entirely smoothly.
“Filtering this type of content versus say, a political protest with nudity or the statue of David, is not simple at scale,” D’Onofrio wrote at the time. “We’re relying on automated tools to identify adult content and humans to help train and keep our systems in check. We know there will be mistakes, but we’ve done our best to create and enforce a policy that acknowledges the breadth of expression we see in the community.”
I’m guessing one mistake D’Onofrio might not have anticipated was the possibility Tumblr’s porn filter would flag the company’s own examples of depictions of nudity that the site would still allow to be displayed.
Last week, the Tumblr staff posted some examples of “imagery that are still permitted within our policy.”
According to Gizmodo however, when they posted a copy of the same example gif, “it was immediately flagged as a potential violation and hidden by the platform’s filter.”
“When the images shown in the gif were uploaded individually, two of the four examples – which appeared to show a breast ultrasound and a pro-choice protest, respectively – were similarly flagged and hidden,” added Gizmodo’s Hudson Hongo in a post published Friday.
As of the time of this post’s writing, Tumblr had not responded to Gizmodo’s request for comment on the outcome of Gizmodo’s experiment uploading the purportedly still-permitted depictions. The fact the content was flagged by Tumblr’s filter probably shouldn’t come as a surprise though, given that over-inclusiveness is a well-known problem with image recognition software.
For example, right around this time last year, reports circulated about the Metropolitan Police in London realizing the image recognition software the MP used was flagging depictions of sand dunes as naked human bodies.
“Sometimes it comes up with a desert and it thinks it’s an indecent image or pornography,” Mark Stokes, the MP’s head of digital and electronic forensics, said at the time. “For some reason, lots of people have screen-savers of deserts and [the software] picks it up, thinking it is skin color.”
As Gizmodo notes, Tumblr’s filter hasn’t just struggled to recognize the example images as compliant with the platform’s new content policy – it has also (allegedly) flagged an image of a finger as a penis, as well as completely non-sexual images captured from the show Great British Bake Off.
While I’m sure parents around the world will be pleased to hear their children are now protected from depictions of British people preparing confections, it sounds as if D’Onofrio’s prediction that “there will be mistakes” was even more on-the-nose than he may have realized.
