Hello there, my fellow dreamers, and welcome to issue #38 of AI Art Weekly! 👋
Things are moving along nicely and we’re slowly creeping our way to the 2’000 subscribers milestone. If you have a minute, sharing the newsletter with your friends and colleagues would go a long way. Thank you 🙏. Some of the highlights of the week are:
- MusicGen – a new open-source music generation model
- MetaHuman Animator – animate MetaHuman’s in minutes
- DreamHuman – animatable 3D avatars from text
- TryOnDiffusion – transfer garments from one image onto the pose of another
- Rerender a Video and VidEdit – text-guided video-to-video style transfer
- Seeing the World through Your Eyes – reconstruct 3D scenes from eye reflections
- Interviews with Mielconejo D’Maecdo and Joshua Larson
- An essay by Marc Andressen on “Why AI Will Save the World”
- And yet another prompt generator
Cover Challenge 🎨
News & Papers
Last week, right after publishing issue 37, Meta released MusicGen, a simple and controllable model for music generation. Unlike MusicLM by Google, this one you can use Today. Open-source yaaay. There is also a Google Colab and a HuggingFace space demo.
Epic Games released MetaHuman Animator this week. The new feature lets you animate facial expressions of MetaHuman’s in literally minutes with only an iPhone and a PC. Extremely impressive.
DreamHuman: Animatable 3D Avatars from Text
Now, MetaHuman might be state of the art when it comes to 3D avatars, but they aren’t generated just from text. DreamHuman’s avatars on the other hand are. And the method is able to 3D avatars that are animatable. Existing methods produce fixed rather than animated 3D human models and DreamHuman has found a solution to that problem.
TryOnDiffusion: A Tale of Two UNets
RIP fashion models. TryOnDiffusion makes it possible to transfer garments from one image onto the pose of another with extremely impressive results. AI will take all our jobs? Maybe, or maybe not. Before you freak out, take a breath, scroll down, and read the essay by Marc Andressen below on “Why AI Will Save the World” in the Tools & Tutorials section.
Video Editing: Rerender a Video + VidEdit
Language-Guided Music Recommendation for Video via Prompt Analogies
Methods like Language-Guided Music Recommendation for Video via Prompt Analogies will soon make it possible to generate soundtracks from only a video clip or in combination with a text prompt. As this research is being done by Adobe, I can see this being implemented down the road in Adobe Premiere to generate matching audio for B-roll footage.
Seeing the World through Your Eyes
It’s said that our eyes hold the universe. When it comes to the method discussed in the paper Seeing the World through Your Eyes, they at least hold a 3D scene. The method discussed in the paper is able to reconstruct 3D scenes beyond the camera’s line-of-sight using portrait images containing eye reflections.
- AniFaceDrawing: Anime Portrait Exploration during Your Sketching
- BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping
- Face0: Instantaneously Conditioning a Text-to-Image Model on a Face
- intrinsicFlash: Computational Flash Photography through Intrinsics
- Local Deformation: Local Deformation for Interactive Shape Editing
Tools & Tutorials
These are some of the most interesting resources I’ve come across this week.
And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:
- Sharing it 🙏❤️
- Following me on Twitter: @dreamingtulpa
- Buying me a coffee (I could seriously use it, putting these issues together takes me 8-12 hours every Friday 😅)
Reply to this email if you have any feedback or ideas for this newsletter.
Thanks for reading and talk to you next week!