“But I’m really good at detecting AI in pictures!”

  • It is currently popular to take online quizzes to detect AI

    Typically the quiz will show perhaps 10 photograph-like images, with the viewer challenged to detect “which images are generated with AI.”

    Many users report they get as many as 9 out of 10 correct.

  • But there are multiple problems with these quizzes

    • Being right only 90 percent of the time isn’t great
    Scoring “9 out of 10” means that for every 10 million fabricated images, fully 1 million would go undetected.

    • The quizzes allow too much time
    If the quizzes were realistic, quiz-takers would only be given 1 or 2 seconds to decide about each image (because that’s how long billions of online photos are seen by viewers as they scroll through their feed).

    • The quizzes pick easy targets
    The quizzes typically feature only “plaigis” (photo-like AI-generated images), in which the entire image is AI-generated and thus there are countless opportunities for an AI misstep.

    • The quizzes avoid hard targets
    There are many millions of difficult-to-detect “aigmented” images — normal photographs that have been augmented with a few AI-generated elements secretly inserted into them.

    • The quiz-takers are looking for trouble
    The people taking the quizzes are carefully looking for clues. But when people scroll through dozens of photos every minute, the small AI-generated elements that are now routinely added to photos are likely to go undetected.

    • One person’s skill at identifying AI is irrelevant
    Even if the reader of this page were the most talented AI-spotter in the world, that would not noticeably affect how many millions of people are duped by AI images every minute. The culture-wide affect of AI-imaging is what matters.

    • Check back in two or three years
    We are still in the very early stages of AI, which “trains” itself by learning from its own mistakes. As this cycle of self-learning is repeated billions of times, AI will continually get much more difficult to detect.

  • So what’s the solution?

    Any plan for combatting AI-generated content that relies on “visual detection” is doomed from the start.

    The only viable solution is the regular and frequent use of trust tests to identify trustworthy image sources.

     

    Any addition of AI-generated content — even a single pixel — disqualifies an image from TTG.