AI Toolbox

VAR-CLIP creates detailed fantasy images that match text descriptions closely by combining Visual Auto-Regressive techniques with CLIP! It uses text embeddings to guide image creation, ensuring strong results by training on a large image-text dataset.

CityGaussian can render large-scale 3D scenes in real-time using a divide-and-conquer training approach and Level-of-Detail strategy. It achieves high-quality rendering at an average speed of 36 FPS on an A100 GPU.

Perm can generate and manipulate 3D hairstyles. It enables applications such as 3D hair parameterization, hairstyle interpolation, single-view hair reconstruction, and hair-conditioned image generation.

SV4D 2.0 can generate high-quality 4D models and videos from a reference video.

SEG improves image generation for SDXL by smoothing the self-attention energy landscape! This boosts quality without needing guidance scale, using a query blurring method that adjusts attention weights, leading to better results with fewer drawbacks.

SMooDi can generate stylized motion from text prompts and style motion sequences.

Interactive3D can generate high-quality 3D objects that users can easily modify. It allows for adding and removing parts, dragging objects, and changing shapes.

XHand can generate high-fidelity hand shapes and textures in real-time, enabling expressive hand avatars for virtual environments.

DreamMover can generate high-quality intermediate images and short videos from image pairs with large motion. It uses a flow estimator based on diffusion models to keep details and ensure consistency between frames and input images.

AniTalker is another talking head generator that can animate talking faces from a single portrait and input audio with naturally flowing movements and diverse outcomes.

Magic Clothing can generate customized characters wearing specific garments from diverse text prompts while preserving the details of the target garments and maintain faithfulness to the text prompts.

Audio-Synchronized Visual Animation can animate static images using audio clips to create synchronized visual animations. It uses the AVSync15 dataset and the AVSyncD diffusion model to produce high-quality animations across different audio types.

ClickDiff can generate controllable grasps for 3D objects. It employs a Dual Generation Framework to produce realistic grasps based on user-specified or algorithmically predicted contact maps.

ViPer can personalize image generation by capturing individual user preferences through a one-time commenting process on a selection of images. It utilizes these preferences to guide a text-to-image model, resulting in generated images that align closely with users’ visual tastes.

Adobe’s Magic Fixup lets you edit images with a cut-and-paste approach that fixes edits automatically. Can see this being super useful for generating animation frames for tools like AnimateDiff. But it’s not clear yet if or when this hits Photoshop.

SV4D can generate dynamic 3D content from a single video. It ensures that the new views are consistent across multiple frames and achieves high-quality results in video synthesis.

Artist stylizes images based on text prompts, preserving the original content while producing high aesthetic quality results. No finetuning, no ControlNets, it just works with your pretrained StableDiffusion model.

DreamCar can reconstruct 3D car models from just a few images or single-image inputs. It uses Score Distillation Sampling and pose optimization to enhance texture alignment and overall model quality, significantly outperforming existing methods.

Cinemo can generate consistent and controllable image animations from static images. It achieves enhanced temporal consistency and smoothness through strategies like learning motion residuals and employing noise refinement techniques, allowing for precise user control over motion intensity.

MasterWeaver can generate photo-realistic images from a single reference image while keeping the person’s identity and allowing for easy edits. It uses an encoder to capture identity features and a unique editing direction loss to improve text control, enabling changes to clothing, accessories, and facial features.