AI Art Weekly #38

Hello there, my fellow dreamers, and welcome to issue #38 of AI Art Weekly! 👋

Things are moving along nicely and we’re slowly creeping our way to the 2’000 subscribers milestone. If you have a minute, sharing the newsletter with your friends and colleagues would go a long way. Thank you 🙏. Some of the highlights of the week are:

  • MusicGen – a new open-source music generation model
  • MetaHuman Animator – animate MetaHuman’s in minutes
  • DreamHuman – animatable 3D avatars from text
  • TryOnDiffusion – transfer garments from one image onto the pose of another
  • Rerender a Video and VidEdit – text-guided video-to-video style transfer
  • Seeing the World through Your Eyes – reconstruct 3D scenes from eye reflections
  • Interviews with Mielconejo D’Maecdo and Joshua Larson
  • An essay by Marc Andressen on “Why AI Will Save the World”
  • And yet another prompt generator

Cover Challenge 🎨

Theme: summer vibes
84 submissions by 45 artists
AI Art Weekly Cover Art Challenge summer vibes submission by mamaralic
🏆 1st: @mamaralic
AI Art Weekly Cover Art Challenge summer vibes submission by spiritform
🥈 2nd: @spiritform
AI Art Weekly Cover Art Challenge summer vibes submission by Annabell_art
🥈 2nd: @Annabell_art
AI Art Weekly Cover Art Challenge summer vibes submission by mikadomikado13
🥉 3rd: @mikadomikado13

News & Papers

MusicGen

Last week, right after publishing issue 37, Meta released MusicGen, a simple and controllable model for music generation. Unlike MusicLM by Google, this one you can use Today. Open-source yaaay. There is also a Google Colab and a HuggingFace space demo.

MetaHuman Animator

Epic Games released MetaHuman Animator this week. The new feature lets you animate facial expressions of MetaHuman’s in literally minutes with only an iPhone and a PC. Extremely impressive.

MetaHuman Animator in action

DreamHuman: Animatable 3D Avatars from Text

Now, MetaHuman might be state of the art when it comes to 3D avatars, but they aren’t generated just from text. DreamHuman’s avatars on the other hand are. And the method is able to 3D avatars that are animatable. Existing methods produce fixed rather than animated 3D human models and DreamHuman has found a solution to that problem.

DreamHuman examples

TryOnDiffusion: A Tale of Two UNets

RIP fashion models. TryOnDiffusion makes it possible to transfer garments from one image onto the pose of another with extremely impressive results. AI will take all our jobs? Maybe, or maybe not. Before you freak out, take a breath, scroll down, and read the essay by Marc Andressen below on “Why AI Will Save the World” in the Tools & Tutorials section.

TryOnDiffusion examples

Video Editing: Rerender a Video + VidEdit

Rerender A Video and VidEdit are yet two other attempts at a text-guided video-to-video style transfer method. The temporal coherency has definitely come a long way since the “early days”.

Rerender a Video example

Language-Guided Music Recommendation for Video via Prompt Analogies

Methods like Language-Guided Music Recommendation for Video via Prompt Analogies will soon make it possible to generate soundtracks from only a video clip or in combination with a text prompt. As this research is being done by Adobe, I can see this being implemented down the road in Adobe Premiere to generate matching audio for B-roll footage.

Check out the project page above for audio examples

Seeing the World through Your Eyes

It’s said that our eyes hold the universe. When it comes to the method discussed in the paper Seeing the World through Your Eyes, they at least hold a 3D scene. The method discussed in the paper is able to reconstruct 3D scenes beyond the camera’s line-of-sight using portrait images containing eye reflections.

A reconstructed 3D scene from eye reflections

More gems

  • AniFaceDrawing: Anime Portrait Exploration during Your Sketching
  • BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping
  • Face0: Instantaneously Conditioning a Text-to-Image Model on a Face
  • intrinsicFlash: Computational Flash Photography through Intrinsics
  • Local Deformation: Local Deformation for Interactive Shape Editing

Interviews

This week Anna Dart and me got another two AI Surrealism interviews for you. This week with Mielconejo D’Macedo and Joshua Larson.


Tools & Tutorials

These are some of the most interesting resources I’ve come across this week.

alien lounge band performing in a las vegas casino bar, associated press photo, UHD, 8k resolution, sharp focus. Made by hazmat008#6887 in our Discord.

And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:

  • Sharing it 🙏❤️
  • Following me on Twitter: @dreamingtulpa
  • Buying me a coffee (I could seriously use it, putting these issues together takes me 8-12 hours every Friday 😅)

Reply to this email if you have any feedback or ideas for this newsletter.

Thanks for reading and talk to you next week!

– dreamingtulpa

by @dreamingtulpa