AI is at the center of most of Adobe’s recently announced new capabilities and experiments, such as text-based picture production in Photoshop and object addition and removal for Premiere Pro. According to The Verge, the business has now revealed VideoGigaGAN, an experimental AI function that it claims can upscale video by eight times without introducing typical errors like flickering or distortion.
According to Adobe, VideoGigaGAN outperforms previous Video Super Resolution (VSR) techniques since it does not present the typical artifacts and flickering that GAN (General Adversarial Networks) introduced. In addition, it adds sharpness and detail, something that most other systems are unable to accomplish simultaneously.
Of course, the system is making up detail that doesn’t exist out of whole cloth, so this wouldn’t be suitable for things like forensic video enhancement, à la CSI-style crime shows. But the detail it does add looks impressively real, like skin textures, fine hairs, swan feather details and more.
The model builds on a large-scale image upsampler called GigaGAN, according to to Adobe’s researchers. Previous VSR models have had difficulty generating rich details in results, so Adobe married “temporal attention” (reducing artifacts that accumulate over time), feature propagation (adding detail where none exists), anti-aliasing and something called “HF shuttle” (shuttling high-frequency features) to create the final result.
If added to products like Premiere Pro or After Effects, it could allow video producers to make low-resolution shots look a lot better, though using AI too enhance people is a controversial practice. There’s no word yet on whether Adobe plans to do this (currently, the clips are short and only play at 12fps) but plenty of companies including NVIDIA, Microsoft, Blackmagic Design and others are working on AI upscaling technology as well.
