This piece contains references to eating disorders. If you or someone you know needs help, support is available.
Human contractors hired to improve Microsoft’s Copilot AI chatbot are constantly bombarded with obscene or sexually explicit photo editing requests and images that users have uploaded, including photos of upskirts or putting women in sexual positions. These contractors are then asked to check whether the generated image successfully answered the prompt – for example, did the image generator make the woman’s AI-enlarged breasts large enough.
When a Copilot user uploads a photo of themselves or someone else, they likely expect that image to remain private between them and the AI tool. In reality, a workforce of at least hundreds of human reviewers sometimes reviews this prompt and the images they upload. And in many cases, these entrepreneurs are inundated with requests to create foot fetish images of children’s cartoon characters, shorten a real woman’s skirt, or put people in sexual positions.
The news, based on a cache of internal contractor documents viewed by 404 Media, shows that AI companies are using human workers to review not only text prompts from AI chatbot users, but also images they upload and wish to edit. Earlier this month, 404 Media revealed that OpenAI had thousands of entrepreneurs which in some cases review real prompts from ChatGPT users. This approach also extends to Microsoft and Copilot.
“Faces are still not censored, and many of the prompts are sexual in nature and questionably consensual,” a person who works on the prompts told 404 Media. 404 Media outlets granted the person anonymity because they were not authorized to speak to the press.
💡
Are you a rapid reviewer for OpenAI, Anthropic, or another AI company? I would like to hear from you. Using a non-professional device, you can message me securely on Signal at joseph.404 or email me at joseph@404media.co.
404 Media obtained a set of internal documents relating to contractors examining Copilot prompts and images, including instruction guides, prompts and actual images of Copilot users, as well as conversations between contractors on an internal discussion forum.
In a thread on the internal discussion forum, an entrepreneur discussed prompts asking Copilot to shorten a woman’s skirt, enlarge her breasts, or show more leg while wearing stiletto heels. Images of She-Hulk come up often, the person who works on the prompts told 404 Media. Other entrepreneurs also wrote that they were shown pro-anorexia content.
“I cringed,” one entrepreneur wrote about this content. Another person wrote that one of the prompts she reviewed asked Copilot to create an image of Ariana Grande suffering from anorexia. Another reported a prompt that seemed designed to create a lewd or suggestive image of a group of young girls.
These contractors are not engaged to report or review offensive or inappropriate content. They are paid to review the quality of Copilot’s production, including in these cases of sexual images.
“Who writes these prompts and who decides that Copilot now focuses on producing porn? It’s hard to take things seriously when I have to focus on the model who generated the appropriate bust size or the middle-aged woman who was placed in the appropriate sexually suggestive position,” one entrepreneur said in the thread.
Microsoft explicitly looks to entrepreneurs’ human intuition to determine which image looks better, according to the documents. “Trust your intuition: When you look at the two edited images side by side, which one immediately seems better? Your gut reaction as a human viewer matters,” reads a set of instructions given to entrepreneurs.
“When in doubt, trust your first impression,” the instructions continue. “Human intuition is good at detecting subtle and difficult-to-express differences in quality.”
Human entrepreneurs being exposed to gruesome, unpleasant, or traumatic content is, of course, nothing new. Big tech companies, and particularly social networks, have for years used armies of contractors to moderate user-generated content. AI companies have also used poorly paid workers abroad to train their AI models or, in the case of OpenAI, make them less toxic. What’s different in this latest episode of Copilot, and 404 Media reports on entrepreneurs in other AI companies like OpenAIis that the content that humans review are prompts and uploads that chatbot users may consider private, and that contractors do not view this material to train models to filter offensive images or for other security concerns. The training aims to make chatbot responses better: more informative, user-friendly and clearer.
The instructions seen by 404 Media focus on what contractors do when a Copilot user asks the AI to edit an image. The contractor sees the user’s original prompt, the uploaded image, and then two edits generated by Copilot. The contractor must choose which retouch is best, based on four elements: Does the retouched image correctly follow the retouching instructions? are elements of the image that should remain unchanged and preserved — for example, if the prompt is “add a cup to the table,” nothing else should be changed except the cup — does the edited image contain visible artifacts like distortions or unnatural textures; and the overall quality of the AI-generated edit.
Some Copilot contractors complained on the private forum that they had accepted “tasks” that unknowingly contained explicit, dangerous or even potentially illegal images. “I just came across an image set consisting of eight upskirt photos,” the entrepreneur wrote, while asking his superiors to add a dangerous content flag to the task. On the same thread, another entrepreneur said he saw images of “some sort of animal sacrifice,” and a third said he saw prompts about sexual movements and positions.
At least one of the companies hiring the contractors for this work is called Prolific. A prolific Mentions service on his website is “Human feedback from representative populations – for preference setting, security assessments, and benchmarks you can defend.” »
In another thread, an entrepreneur cites Prolific’s own guidelines, which point out that it can be difficult to ensure that trainers are not presented with sexual images: “Particular caution regarding disturbing or explicit content should be taken with generative AIs. This is because, unlike traditional content, researchers cannot fully control what is shown to participants.”
Prolific did not respond to a request for comment. A Microsoft spokesperson told 404 Media in an email: “Microsoft uses customer data as described in our Terms of Service, including to improve our products and enforce our code of conduct. »
Microsoft’s AI tools have long been misused by people to create non-consensual AI-generated images of people. Members of the pornography-focused Telegram channels 4chan and AI used Microsoft tools, e.g. generate Taylor Swift porn which then went viral on Twitter. Microsoft fixed the flaw these people were using in Microsoft Designer after the 404 Media report.
In 2024, 404 Documented media how Copilot would respond to, and then delete, responses to potentially controversial or sexual prompts in real time. In March, a top Senate official endorsed Copilot for use in the Senate, with ChatGPT and Gemini from Google. A memo states that Copilot “can assist with the routine work of the Senate, including drafting and editing documents, synthesizing information, preparing talking points and briefing papers, and conducting research and analysis.”
Gn tech
