Video AI Tools

DisPose can generate high-quality human image animations from sparse skeleton pose guidance.

SynCamMaster can generate videos from different viewpoints while keeping the look and shape consistent. It improves text-to-video models for multi-camera use and allows re-rendering from new angles.

ObjCtrl-2.5D enables object control in image-to-video generation using 3D trajectories from 2D inputs with depth information.

3DTrajMaster can control the 3D motions of multiple objects in videos using user-defined 6DoF pose sequences.

On the other hand, Customizing Motion can learn and generalize input motion patterns from input videos and apply them to new and unseen contexts.

MEMO can generate talking videos from images and audio. It keeps the person’s identity consistent and matches lip movements to the audio, producing natural expressions.

CAVIS can do instance segmentation on videos. It’s able to better track objects and improve instance matching accuracy, resulting in more accurate and stable instance segmentation.

VideoRepair can improve text-to-video generation by finding and fixing small mismatches between text prompts and videos.

Inverse Painting can generate time-lapse videos of the painting process from a target artwork. It uses a diffusion-based renderer to learn from real artists’ techniques, producing realistic results across different artistic styles.

CAT4D can create dynamic 4D scenes from single videos. It uses a multi-view video diffusion model to generate videos from different angles, allowing for strong 4D reconstruction and high-quality images.

SAMURAI combines the SOTA visual video tracking of SAM 2 with motion-aware memory.

StableV2V can stabilize shape consistency in video-to-video editing by breaking down the editing process into steps that match user prompts. It handles text-based, image-based, and video inpainting.

CamI2V is a method which can generate videos from images with precise control over camera movements and text prompts.

JoyVASA can generate high-quality lip-sync videos of human and animal faces from a single image and speech clip.

CHANGER can integrate an actor’s head onto a target body in digital content. It uses chroma keying for clear backgrounds and enhances blending quality with Head shape and long Hair augmentation (H2 augmentation) and a Foreground Predictive Attention Transformer (FPAT).

DAWN can generate talking head videos from a single portrait and audio clip. It produces lip movements and head poses quickly, making it effective for creating long video sequences.

DimensionX can generate photorealistic 3D and 4D scenes from a single image using controllable video diffusion.

SG-I2V can control object and camera motion in image-to-video generation using bounding boxes and trajectories

GIMM is a new video interpolation method that uses motion modelling to predict motion between frames.

AutoVFX can automatically create realistic visual effects in videos from a single image and text instructions.