AI Art Weekly #44

Hello there, my fellow dreamers, and welcome to issue #44 of AI Art Weekly! 👋

Exciting update. My piece LIVE.DIE.REPEAT. ranked 2nd place in Claire Silver’s motion contest 🥳

Apart from that, there are a lot more exciting things that happened this week:

  • Stable Diffusion XL 1.0 released
  • Gen-2 now supports Image to Video
  • WavJourney is a new system that uses LLMs to generate audio
  • Dyn-E is a method for making local edits to NeRFs
  • A new image Interpolating technique for diffusion models
  • Interview with artist CrazyP̆̈ĕ̈p̆̈ĕ̈🫀🧠👁️
  • And some more cool gems and papers

Cover Challenge 🎨

Theme: idioms
88 submissions by 52 artists
AI Art Weekly Cover Art Challenge idioms submission by moelucio
🏆 1st: @moelucio
AI Art Weekly Cover Art Challenge idioms submission by VikitoruFelipe
🥈 2nd: @VikitoruFelipe
AI Art Weekly Cover Art Challenge idioms submission by CurlyP139
🥉 3rd: @CurlyP139
AI Art Weekly Cover Art Challenge idioms submission by mrtelevoid
🧡 4th: @mrtelevoid

News & Papers

Stable Diffusion XL 1.0 released

The weights for SDXL 1.0 finally got released! The new model is using a new pipeline (base to refiner) which is able to generate more vibrant and accurate colors, with better contrast, lighting, and shadows than its predecessor and raises the image resolution from 768x768 to a full 1-megapixel (1024x1024) with multiple aspect ratios. You can use the model either with common web interfaces like ComfyUI or give it a try on Clipdrop. It works with Automatic1111 as well, but support for the refiner pipeline is lacking i think. Can’t wait to see what the community and the fine-tuners will come up in the next few days and weeks. Just remember, it’s called XL, not XXL.

glimpses of summer sadness created with SDXL 1.0 on Clipdrop and outpainted with the uncrop feature.

Gen-2 Image-to-Video released

Gen-2 now supports Image to Video without the need for a text prompt. The new mode is a great way to explore consistent characters, locations and blocking from generation to generation. Most generations have not a lot of movement yet and apparently it works better with simpler compositions. To use it, just add an image to the Gen-2 interface without a prompt and hit Generate.

A piece for an upcoming collection turned into a video with Gen-2.

WavJourney: Compositional Audio Creation with Large Language Models

WavJourney is a system that uses large language models to generate audio content with storylines encompassing speech, music, and sound effects guided from text instructions. The demo results, while not perfect, sound great.

Visual representation of a WavJourney example. Checkout the project page for audio examples.

Dyn-E: Local Appearance Editing of Dynamic Neural Radiance Fields

Similar to the scribbles feature of INVE last week, Dyn-E is a method for making local edits to NeRFs by manipulating pixels in a single frame of a training video. I’ve a use case in mind for this where educational video footage might be converted to 3D and certain spots get highlighted with this technique.

Dyn-E example

Interpolating between Images with Diffusion Models

This paper introduces a new method based on ControlNet for generating interpolation videos between two images. Might be a fun experiment to generate input videos for video-to-video models with it.

Stable Diffusion Interpolation examples

More papers & gems


This week I finally got the chance to interview one of our Discord OG’s @CrazyPepeEth whose work is currently exhibited in France in Villa Malzac. CrazyPepe is a programmer who during the pandemic started to create art with his son @ErrorSkull and never looked back. Truly inspiring. Enjoy!

Tools & Tutorials

These are some of the most interesting resources I’ve come across this week.

Social media Hsiao-Ron Cheng abstract tiktok oil painting by yum yum gang gang detailed crowded modernity scene --c 15 --style raw --w 100 --ar 3:2. Made by me in our Discord.

And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:

  • Sharing it 🙏❤️
  • Following me on Twitter: @dreamingtulpa
  • Buying me a coffee (I could seriously use it, putting these issues together takes me 8-12 hours every Friday 😅)

Reply to this email if you have any feedback or ideas for this newsletter.

Thanks for reading and talk to you next week!

– dreamingtulpa

by @dreamingtulpa