The news: A recent study found that generative AI image-creation tool Midjourney was prompt-engineered into creating dozens of racist and conspiratorial images in violation of the company’s rules, per Bloomberg.
Reality distortion: Researchers from the Center for Countering Digital Hate revealed that Midjourney’s users can subvert the AI’s built-in guardrails by substituting prompts.
- Instead of requesting an image of a politician with blood on their hands, the researchers substituted the phrase “strawberry syrup” for blood.
- The study suggests that despite Midjourney automatically blocking some text inputs and having 68 content moderators, these defenses are easily circumvented.
- In many instances, Midjourney complies with requests for fabricated images of politicians, celebrities, and other public figures in compromising scenarios.
- The service created the most famous and widely-circulated AI-generated image of Pope Francis wearing a puffy jacket.
The problem: Online misinformation coupled with convincing imagery can become dangerous, especially at a time of heightened disinformation campaigns heading into elections.
- Midjourney, which is accessible via a $10 monthly subscription on the Discord app, reached more than 42 million monthly visitors when it peaked in popularity in April, per Similarweb.
- Far-right outlet Breitbart and YouTuber Jackson Hinkle have used Midjourney to promote racially-driven conspiracies, per Bloomberg.
- Failure to reign in generative AI’s misuse could accelerate stringent regulation or lawsuits.
Our take: As the 2024 elections loom, the potential misuse of AI tools to generate deceptive images depicting fictitious events warrants attention.
- Services like Google Images are responding by labeling AI-generated images in search results, but there’s more that the tools’ creators can do.
- Adding visible watermarks or exchangeable image file format (exif) data indicating the source or creator of the image could increase accountability and help determine the authenticity of AI-generated content.