Enhancing Photorealism Enhancement

What Hacker News said

Recommends arduinomancer · 2021-05 · on “Neural Rendering: How Low Can You Go in Terms of Input?”

The video linked of applying a neural net to GTAV for photorealism is really impressive if you haven't seen it. [this video]

Recommends valgaze · 2022-04 · on ““Can it run Doom” will never be the same, thanks to new ray-tracing mod”

DLSS & ideas like it outside an idea are really interesting: [LINK] Ex. run the base game at an insanely crummy/low resolution. On low(er) it’s possible to run a model to kinda “upscale” in each frame in a performant way. Or even this CNN initiative from Intel where they took Grand Theft Auto & “painted” it with dashcam footage from rural Germany: [this video] The results are pretty interesting— all “brown” southern CA mountains were green like in Germany

Cites thih9 · 2021-05 · on “Enhancing Photorealism Enhancement”

From the "Enhancing Photorealism Enhancement" video [1]: "It is a convolutional network which produces images frame by frame and can be run at interactive rates.". [1]: [this video] (0:19)

Cites L_ · 2021-05 · on “Enhancing Photorealism Enhancement”

There's also a version trained on Mapillary Vistas, which has images from all around the world. Results from that are at the bottom of the page and at the end of the video ( [this video] )

Cites wruza · 2021-08 · on “Software Dev Can't Be Automated – It’s a Creative Process with an Unknown Goal”

Likely not, but that was not my point either, it was just an non-stretchable analogy. AI is not a single network usually, it is a pipeline of networks of different kinds, see e.g. [this video] (seek to 1:35 for beginning of an architecture overview). You can’t call it “just pattern matching” without fetching it too far.

Cites valgaze · 2022-12 · on “Stable Diffusion with Core ML on Apple Silicon”

Video is really a series of frames, the framerate for film/human can get away with 24 frames/second-- so maybe ~40ms/image for real-time at least? What's cool about the era in which we live is if you look at high-performance graphics for games or simulations, for instance, it may in fact be faster to a the model to "enhance" a low-resolution frame rather than trying to render it fully on the machine. ex. AMD's FSR vs NVIDIA DLSS - AMD FSR (Fidelity FX Super Resolution): [LINK] - NVIDIA DLSS (Deep Learning Super Sampling): j [LINK] AMD's approach renders the game at a crummy, low-detail resolution then each frame uses "upscales" Both FSR and DLSS aim to improve frames-per-second in games by r

Cites FoodWThrow · 2023-11 · on “How do I become a graphics programmer? – A guide from AMD Game Engineering team”

> AI will be doing graphics programming? I doubt that. I think the bigger concern (if you could call it that) is that certain AI techniques may be able to replace entire stacks once they get advanced/stable enough. Something along the lines of this perhaps, where you render simplified geometry and let the AI fill in the blanks: [this video] Even without experimental technologies, you can get a glimpse of how shipped tools like ray reconstruction can morph the field in the future, forever entrenching themselves into graphics programming one way or another. As far as AI writing graphics code? No way, at least not in the next couple of decades. Button snippets are a far cry from rendering virtualize

Cites joegibbs · 2024-02 · on “Sora: Creating video from text”

I think for the near future we’ll see something like this: [this video] From a few years ago, where the game is rendered traditionally and used as a ground truth, with a model on top of it that enhances the graphics. After maybe 10-15 years we will be past the point where the entire game can be generated without obvious mistakes in consistency. Realtime AI dialogue is already possible but still a bit primitive, I wrote a blog post about it here: [LINK]