Image Restoration

pix2gestalt is able to estimate the shape and appearance of whole objects that are only partially visible behind occlusions.

DiffusionLight can estimate the lighting in a single input image and convert it into an HDR environment map. The technique is able to generate multiple chrome balls with varying exposures for HDR merging and can be used to seamlessly insert 3D objects into an existing photograph. Pretty cool.

DiffusionMat is a novel image matting framework that employs a diffusion model for the transition from coarse to refined alpha mattes. The key innovation of the framework is a correction module that adjusts the output at each denoising step, ensuring that the final result is consistent with the input image’s structures.

HyperHuman is a text-to-image model that focuses on generating hyper-realistic human images from text prompts and a pose image. The results are pretty impressive and the model is able to generate images in different styles and up to a resolution of 1024x1024.

ScaleCrafter can generate ultra-high-resolution images up to 4096x4096 and videos at 2048x1152 using pre-trained diffusion models. It reduces problems like object repetition and allows for custom aspect ratios, achieving excellent texture detail.

DA-CLIP is a method that can be used to restore images. Apart from inpainting, the method is able to restore images by dehazing, deblurring, denoising, derainining and desnowing them as well as removing unwanted shadows and raindrops or enhance lighting on low-light images.

PGDiff can restore and colorize faces from low-quality images by using details from high-quality images. It effectively fixes issues like scratches and blurriness.

RIP expensive low-light cameras? It’s amazing how AI is able to solve problems which so far was only possible with better hardware. In this example the novel LED model is able to denoise low-light images trained on only 6 pairs of images. The results are impressive, but the team is not done yet. They’re currently researching a method that works on a wide variety of scenarios trained on only 2 pairs.

LDMs are high-resolution image generators that can inpaint, generate images from text or bounding boxes, and do super-resolution.

GFPGAN can restore realistic facial details from low-quality images using a pretrained face GAN. It works well on both synthetic and real-world images, allowing for quick restoration with just one pass, unlike older methods.