I think it needs depth buffer access and a UI mask, but not necessarily deep integration beyond that? At least for the FSR3 shader code I’ve looked at personally.
This is not out of the question for generic integration.
I’m less sure if FSR4 though, as I think that was a pretty big leap over 3.
thingsiplay@lemmy.ml 7 hours ago
I don’t know the exact process too, so we are both speculating. But its fact that Temporal upscaling (that is FSR2 and upwards for FSR related ones) need integration from the game. And these are Ai models that needs some kind of training data from the game as well. I don’t know what that exactly means, but my guess is that the integrated Ai model to upscale is trained with the data in the game and then delivered as a package. That makes it impossible to integrate a general purpose Temporal upscaler with Ai into a system, that could be utlized on any game (or program if you will). So I don’t know how this cannot be anything else “out of question”. I mean this is by concept not possible, as far as I understand (and why nobody did it too).
But an Ai system could be used as a Spatial upscaler, like a filter that is not integrated into the game. FSR 1 itself is not using any Ai, but DLSS 5 has such capabilities (think of the new lighting model, it could be tweaked to upscale the image instead).
brucethemoose@lemmy.world 5 hours ago
I don’t think it’s actually trained per game. Maybe per type of game.
I do remember Nvidia offering this as a service to game devs for a hot minute, but really, these models generalize well. A model they spent time tweaking and training on a big basket of games might work better than a finetune for one specific game, as it would just get overfit.