AI Art Weekly #51
Hello there, my fellow dreamers, and welcome to issue #51 of AI Art Weekly! 👋
How time flies! Tomorrow, one year ago, I published the first issue of AI Art Weekly. A lot has happened since then. AI has grown from incoherent low-resolution image generators to fully fledged AI companions that will soon be integrated into our everyday operating systems. Whether you think these changes are exciting, scary, or both, I think we can all agree that it at least hasn’t been boring. And from the looks of it, the future will only get wilder. A bit more than 2500 of you have decided to join me on this wild ride into the future, and I’m more than grateful to have you all on board. Let us explore together what the future of AI holds for us. Thank you for your support 🙏
In Today’s issue we cover:
- DALL·E 3 gets integrated into ChatGPT
- ProPainter can inpaint videos: remove watermarks and objects
- ControlNet-XS – the lightweight version of ControlNet
- Repainting 3D Assets with text prompts
- SVGCustomization – text-guided vector graphics customization
- PGDiff – guiding diffusion models for versatile face restoration
- and more tutorials, tools and gems!
After publishing issue #50 last week, one of my original spiral art posts went viral and gathered 1.2 million views on X. I created a refurbished version since, which I offer on Etsy to support the newsletter. If you would like to own a specific artwork of mine that is not available yet, let me know!
Cover Challenge 🎨
For next weeks cover I’m looking for subliminal art. The reward is $50 and the Challenge Winner role within our Discord community. This rare role earns you the exclusive right to cast a vote in the selection of future winners. Rulebook can be found here and images can be submitted here. I’m looking forward to your submissions 🙏
News & Papers
DALL·E 3 x ChatGPT
OpenAI announced this week that DALL·E 3 will be built natively on ChatGPT. This will let you use ChatGPT as a brainstorming partner for generating images. Release will be early October and Microsoft announcent this week that it’ll be freely available in Bing as well. Will this already be the start of the end for prompt engineers? Depending on how good the images are that DALL·E 3 will produce, I can see this being a shift for big players like Midjourney.
DALL·E 3 will enable iterative image generation through natural conversations
ProPainter: Improving Propagation and Transformer for Video Inpainting
ProPainter is a new video inpainting method that is able to remove objects, complete masked videos, remove watermarks and even expand the view of a video. Code can be found on GitHub.
ProPainter lets you remove watermarks from videos
ControlNet-XS
ControlNet-XS shows that it’s able to achieve state-of-the-art results with as little as 1% of the parameters of the base model, considerably better than ControlNet in terms of FID score. Code hasn’t been released yet, but this will probably make ControlNet extremely more lightweight.
ControlNet-XS results
Repainting 3D Assets
As we move more and more into 3D, texturing 3D assets will become an important part. Repainting 3D Assets is a new method that can take any 3D asset and paint it with a given text prompt. The results, while low-resolution, are pretty impressive.
Repainting 3D Assets example
SVGCustomization: Text-Guided Vector Graphics Customization
Love this one! SVGCustomization is a novel pipeline that is able to edit existing vector images with text prompts while preserving the properties and layer information vector images are made of.
SVGCustomization examples
PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance
PGDiff is a new method that can be used for a broad range of face restoration tasks, including blind restoration, colorization, inpainting, reference-based restoration and old photo restoration. The method is able to restore images with partial guidance, meaning you can guide the model with a rough sketch or a few strokes instead of a fully colored image. Especially older but still popular models like Stable Diffusion are still plagued from deformed faces, a pipeline combined with face recognition and PGDiff could improve those.
PGDiff pipeline and examples
More papers & gems
- FreeU: Free Lunch in Diffusion U-Net
- DreamLLM: Synergistic Multimodal Comprehension and Creation
- SPHP: Sparse and Privacy-enhanced Representation for Human Pose Estimation
- MOVIN: Real-time Motion Capture using a Single LiDAR
- JDD: Dual-Camera Joint Deblurring-Denoising
- CartoonDiff: Training-Free Cartoon Image Generation with Diffusion Transformer Models
- LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Models
RIP Fernando Botero (1932-2023). To honour Fernando, @gorkemyurt trained a LoRA on his art style which captures the whimsical and accessible nature of his artwork.
@MatthieuGB created a stunning photorealistic AI video with images made with Midjourney and aniimated using RunwayML’s Gen-2.
Tools & Tutorials
These are some of the most interesting resources I’ve come across this week.
After my last weeks viral spiral art hit which gathered 1.2 million views, a lot of you asked me how I did it. So, here is a tutorial with the fundamentals.
The code for the paper Rerender A Video (issue 38) got released! Style transfer a video with temporal consistency (no flickering), no training and combine it with existing ControlNet and LoRA models.
@ThomasSimonini wrote a tutorial about how to integrate AI models into Unity. The tutorial uses Sentis, a Unity plugin that allows you to use Hugging Face models directly within Unity.
Do you remember those old upside down illusion images? Based on the Google Colab code by @alexcarliera, @angrypenguinPNG put together a HuggingFace space to generate images like that with two different prompts.
OBEY 👁️ by me. Explorations of the QRcode monster technique with subliminal text messages. More examples here.
And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:
- Sharing it 🙏❤️
- Following me on Twitter: @dreamingtulpa
- Buying me a coffee (I could seriously use it, putting these issues together takes me 8-12 hours every Friday 😅)
- Buy a physical art print to hang onto your wall
Reply to this email if you have any feedback or ideas for this newsletter.
Thanks for reading and talk to you next week!
– dreamingtulpa