Comment on AI-altered images on birdwatching forums putting research at risk

• Tiresia@slrpnk.net ⁨2⁩ ⁨weeks⁩ ago

  And that’s terrible. Imagine seeing a detail in a picture of a bird taken with off-the-shelf components and not being able to know whether that detail was hallucinated by denoising software. Unless that image is accompanied by the full weights of the specific AI version that did the operation as well as the random seed used for its stochastic elements, there is no way to double-check.

  There isn’t even a guarantee that these modifications will be uniform. What’s stopping an oh-so-helpful postprocessing software engineer from using AI vision to determine what the objects of the image are and then handing that information over to the upscaling algorithm to fill the detail with context-dependent hallucinations? Like, if the AI thinks it’s looking at a bird, it’ll fill the space with generic bird feather texture that might not fit to the bird in question at all.

  So you couldn’t even come up with an ad-hoc postprocessing step that replaces AI slop with noise because the AI slop is trying camouflage itself as realistic. The only scientific option is to nuke all data about how the image looks below a certain level of detail, effectively blurring it until the AI slop is invisible. This is way worse than whatever sort of blurriness the AI is trying to correct.

  And using AI to redraw part of the image is even worse. You can’t even blur the image to the point you’re sure there are no hallucinations, the entire image is a fiction inspired by the image that hit the camera, which is now lost. Features at any scale could be made up, the result of the AI hallucinating something in place of an obstruction.

  source