AI Art Weekly #38
Hello there, my fellow dreamers, and welcome to issue #38 of AI Art Weekly! 👋
Things are moving along nicely and we’re slowly creeping our way to the 2’000 subscribers milestone. If you have a minute, sharing the newsletter with your friends and colleagues would go a long way. Thank you 🙏. Some of the highlights of the week are:
- MusicGen – a new open-source music generation model
- MetaHuman Animator – animate MetaHuman’s in minutes
- DreamHuman – animatable 3D avatars from text
- TryOnDiffusion – transfer garments from one image onto the pose of another
- Rerender a Video and VidEdit – text-guided video-to-video style transfer
- Seeing the World through Your Eyes – reconstruct 3D scenes from eye reflections
- Interviews with Mielconejo D’Maecdo and Joshua Larson
- An essay by Marc Andressen on “Why AI Will Save the World”
- And yet another prompt generator
Cover Challenge 🎨
The theme for next weeks cover is censorship. The reward is another $50. Rulebook can be found here and images can be submitted here. Come join our Discord to talk challenges. I’m looking forward to your submissions 🙏
News & Papers
MusicGen
Last week, right after publishing issue 37, Meta released MusicGen, a simple and controllable model for music generation. Unlike MusicLM by Google, this one you can use Today. Open-source yaaay. There is also a Google Colab and a HuggingFace space demo.
MetaHuman Animator
Epic Games released MetaHuman Animator this week. The new feature lets you animate facial expressions of MetaHuman’s in literally minutes with only an iPhone and a PC. Extremely impressive.
MetaHuman Animator in action
DreamHuman: Animatable 3D Avatars from Text
Now, MetaHuman might be state of the art when it comes to 3D avatars, but they aren’t generated just from text. DreamHuman’s avatars on the other hand are. And the method is able to 3D avatars that are animatable. Existing methods produce fixed rather than animated 3D human models and DreamHuman has found a solution to that problem.
DreamHuman examples
TryOnDiffusion: A Tale of Two UNets
RIP fashion models. TryOnDiffusion makes it possible to transfer garments from one image onto the pose of another with extremely impressive results. AI will take all our jobs? Maybe, or maybe not. Before you freak out, take a breath, scroll down, and read the essay by Marc Andressen below on “Why AI Will Save the World” in the Tools & Tutorials section.
TryOnDiffusion examples
Video Editing: Rerender a Video + VidEdit
Rerender A Video and VidEdit are yet two other attempts at a text-guided video-to-video style transfer method. The temporal coherency has definitely come a long way since the “early days”.
Rerender a Video example
Language-Guided Music Recommendation for Video via Prompt Analogies
Methods like Language-Guided Music Recommendation for Video via Prompt Analogies will soon make it possible to generate soundtracks from only a video clip or in combination with a text prompt. As this research is being done by Adobe, I can see this being implemented down the road in Adobe Premiere to generate matching audio for B-roll footage.
Check out the project page above for audio examples
Seeing the World through Your Eyes
It’s said that our eyes hold the universe. When it comes to the method discussed in the paper Seeing the World through Your Eyes, they at least hold a 3D scene. The method discussed in the paper is able to reconstruct 3D scenes beyond the camera’s line-of-sight using portrait images containing eye reflections.
A reconstructed 3D scene from eye reflections
More gems
- AniFaceDrawing: Anime Portrait Exploration during Your Sketching
- BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping
- Face0: Instantaneously Conditioning a Text-to-Image Model on a Face
- intrinsicFlash: Computational Flash Photography through Intrinsics
- Local Deformation: Local Deformation for Interactive Shape Editing
@Martin_Haerlin produced a super cool short with Gen-1, Elevenlabs & Reface. The coherency of the video is phenomenal.
Interviews
This week Anna Dart and me got another two AI Surrealism interviews for you. This week with Mielconejo D’Macedo and Joshua Larson.
Tools & Tutorials
These are some of the most interesting resources I’ve come across this week.
@pmarca wrote a great piece on why AI will save the world – if we let it. The article brings up some interesting points against the top 5 circulating AI “risks”. A must read in my opinion.
I came across @pactalom’s ultimate prompt generator this week (thank you @aisetmefree 🙏) and had the most fun with its randomization feature. Definitely worth to give this a try.
Matting Anything can estimate the alpha matte of any instance in an image with flexible and interactive visual or linguistic user prompt guidance. This is useful for separating a subject from a background. @camenduru put together a Google Colab and @fffiloni created a HuggingFace demo for video matting.
I haven’t shared a Stable Diffusion checkpoint in a while. Here is an experimental one by one of our Discord members @YamerOfficial that I found interesting. A merge of 600+ models that can, as he describes it, create “the pinnacle of perfection or a delightful chaos of visuals”.
@huggingface put together a QR Code AI Art Generator that uses a ControlNet model trained on a large dataset of 150’000 QR code and QR code artwork couples by @diontimmermusic. And yes, 60% of the time, they work every time.
alien lounge band performing in a las vegas casino bar, associated press photo, UHD, 8k resolution, sharp focus. Made by hazmat008#6887 in our Discord.
And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:
- Sharing it 🙏❤️
- Following me on Twitter: @dreamingtulpa
- Buying me a coffee (I could seriously use it, putting these issues together takes me 8-12 hours every Friday 😅)
Reply to this email if you have any feedback or ideas for this newsletter.
Thanks for reading and talk to you next week!
– dreamingtulpa