Text-to-Video

Phantom can generate videos that keep the subject’s identity from images while matching them with text prompts.

SkyReels-V2 can generate infinite-length videos by combining a Diffusion Forcing framework with Multi-modal Large Language Models and Reinforcement Learning.

FramePack aims to make video generation feel like image gen. It can generate single video frames in 1.5 seconds with 13B models on a RTX 4090. Also supports full fps-30 with 13B models using a 6GB laptop GPU, but obviously slower.

TTT-Video can create coherent one-minute videos from text storyboards. As the title of this paper says, this uses test-time training instead of self-attention layers to be able to produce consistent multi-context scenes, which is quite the achievement. The paper is worth a read.

AccVideo can speed up video diffusion models by reducing the number of steps needed for video creation. It achieves an 8.5x faster generation speed compared to HunyuanVideo, producing high-quality videos at 720x1280 resolution and 24fps, which makes text-to-video generation way more efficient.

MotionMatcher can customize text-to-video diffusion models using a reference video to transfer motion and camera framing to different scenes.

PP-VCtrl can turn text-to-video models into customizable video generators. It uses control signals like Canny edges and segmentation masks to improve video quality and control without retraining the models, making it great for character animation and video editing.

Mobius can generate seamlessly looping videos from text descriptions.

MovieAgent can generate long-form videos with multiple scenes and shots from a script and character bank. It ensures character consistency and synchronized subtitles while reducing the need for human input in movie production.

VideoMaker can generate personalized videos from a single subject reference image.

Step-Video-T2V can generate high-quality videos up to 204 frames long using a 30B parameter text-to-video model.

Magic 1-For-1 can generate one-minute video clips in just one minute.

Diffusion as Shader can generate high-quality videos from 3D tracking inputs.

Lumina-Video can generate high-quality videos with synchronized sound from text prompts.

FlashVideo can generate videos from text prompts and upscale them to 1080p.

VideoGuide can improve the quality of videos made by text-to-video models without needing extra training. It enhances the smoothness of motion and clarity of images, making the videos more coherent and visually appealing.

RepVideo can improve video generation by making visuals look better and ensuring smooth transitions.

Kinetic Typography Diffusion Model can generate kinetic typography videos with legible and artistic letter motions based on text prompts.

TransPixar can generate RGBA videos, enabling the creation of transparent elements like smoke and reflections that blend seamlessly into scenes.

DiTCtrl can generate multi-prompt videos with smooth transitions and consistent object motion.