Here is a sentence the internet has been waiting to hear: science has now formally agreed that AI image generators are a black box, and the black box keeps getting darker. A new study in Nature Communications finds that for diffusion models, the technology behind Stable Diffusion, Midjourney, and countless other apps, it is effectively impossible to prove that a generated image came from any specific piece of training data.
That is fantastic news for the models. It is absolutely brutal news for the artists currently suing them.
An Astronaut, a Tree, and Your Artwork
The study comes from a pair of MIT CSAIL researchers: Zheng Dai, a fourth-year PhD student, and computer scientist David Gifford. They did not set out to argue about copyright. They wanted to answer a simpler question: when a diffusion model draws something, is it actually looking at a particular image, like a portrait painter glancing at a human subject?
To find out, the pair built two dozen custom models trained on datasets ranging from a few hundred images up to hundreds of thousands of images pulled from online databases. They also trained separate models on faces and on the work of specific artists. Then they ran the experiment that should make every plaintiff's lawyer wince: they removed individual images from the training data, one at a time, and measured how much the final output changed.
They called that difference the counterfactual radius. In a perfect world, removing an image that inspired an output would change that output noticeably. Dai explains the thinking nicely: in the alternative universe where that data never existed, the model's sample should look different.
It does not.
Attribution Decay Is Apparently a Thing
In all three test categories, the result was identical. The more images a model was trained on, the harder it became to figure out which training images had anything to do with the output. The researchers gave this phenomenon a wonderfully unsettling name: attribution decay.
As the study puts it, at large training set sizes it becomes infeasible to attribute generated images back to the training images, generated people back to real people, and generated artwork back to the artists who were trained on.
The team's favorite analogy captures the scale problem: imagine an astronaut on the International Space Station trying to spot a single tree in the Amazon rainforest with the naked eye. That is roughly the difficulty of connecting an output to a single input in a huge model.
And commercial models are many orders of magnitude bigger than anything the researchers tested. That means they are far less attributable, not more. For the big consumer models, an AI-generated image may have no single human inspiration at all. The output is essentially unattributable.
So Do We Burn the Lawsuits?
This is where the humor curdles into a migraine for the copyright economy. Artists have been suing AI companies over training data for years, and Stable Diffusion is basically the poster child of that entire fight. The Getty Images lawsuit, the artist class actions, the whole pile of litigation, all of it leans on the idea that a generated image can be traced back to stolen work.
This study does not say the copying did not happen. It says something arguably worse for the plaintiffs: the copying cannot be proved, at least not with the direct causal link the law tends to want.
Removing an artist's work from the training data, the study suggests, will not change the output, even when that output looks unmistakably like the artist's style. If proof is the price of entry into a courtroom, this closes the door rather firmly.
So what should artists walk away with, besides a sinking feeling?
- Proving theft in court just got harder, because unattributable is not a phrase lawyers love hearing.
- Resolution will likely have to come from legislation and licensing deals instead of forensic evidence.
- Artists should keep protecting work with watermarks and detection tools, but should not expect them to crack open the black box.
The researchers are not gloating. Dai points out that understanding these models matters precisely so we can study them and regulate them properly. The black box is not an excuse; it is the thing we still need to figure out.
In the meantime, the meme writes itself. Stable Diffusion cannot tell you where it got an idea, the science says it genuinely cannot remember, and a room full of lawyers just found out they are trying to prove a negative about a machine that dreams in statistical noise.
Somewhere out there, an astronaut is squinting at the Amazon, and an artist is squinting at a render, and both are learning the exact same lesson: some things are just too big to pin down.
Comments