AI Art Weekly #44
Hello there, my fellow dreamers, and welcome to issue #44 of AI Art Weekly! 👋
Exciting update. My piece LIVE.DIE.REPEAT. ranked 2nd place in Claire Silver’s motion contest 🥳
Apart from that, there are a lot more exciting things that happened this week:
- Stable Diffusion XL 1.0 released
- Gen-2 now supports Image to Video
- WavJourney is a new system that uses LLMs to generate audio
- Dyn-E is a method for making local edits to NeRFs
- A new image Interpolating technique for diffusion models
- Interview with artist CrazyP̆̈ĕ̈p̆̈ĕ̈🫀🧠👁️
- And some more cool gems and papers
Twitter recently shut down free API access which puts our weekly cover challenges at risk. By becoming a supporter, you can help me make AI Art Weekly and its community efforts more sustainable by supporting its development & growth! 31/100% reached so far 🙏
Cover Challenge 🎨
For next weeks cover I’m looking for art inspired by our modern times. The reward is $50. Rulebook can be found here and images can be submitted here. Come join our Discord to talk challenges. I’m looking forward to your submissions 🙏
News & Papers
Stable Diffusion XL 1.0 released
The weights for SDXL 1.0 finally got released! The new model is using a new pipeline (base to refiner) which is able to generate more vibrant and accurate colors, with better contrast, lighting, and shadows than its predecessor and raises the image resolution from 768x768 to a full 1-megapixel (1024x1024) with multiple aspect ratios. You can use the model either with common web interfaces like ComfyUI or give it a try on Clipdrop. It works with Automatic1111 as well, but support for the refiner pipeline is lacking i think. Can’t wait to see what the community and the fine-tuners will come up in the next few days and weeks. Just remember, it’s called XL, not XXL.
glimpses of summer sadness created with SDXL 1.0 on Clipdrop and outpainted with the uncrop feature.
Gen-2 Image-to-Video released
Gen-2 now supports Image to Video without the need for a text prompt. The new mode is a great way to explore consistent characters, locations and blocking from generation to generation. Most generations have not a lot of movement yet and apparently it works better with simpler compositions. To use it, just add an image to the Gen-2 interface without a prompt and hit Generate.
A piece for an upcoming collection turned into a video with Gen-2.
WavJourney: Compositional Audio Creation with Large Language Models
WavJourney is a system that uses large language models to generate audio content with storylines encompassing speech, music, and sound effects guided from text instructions. The demo results, while not perfect, sound great.
Visual representation of a WavJourney example. Checkout the project page for audio examples.
Dyn-E: Local Appearance Editing of Dynamic Neural Radiance Fields
Similar to the scribbles feature of INVE last week, Dyn-E is a method for making local edits to NeRFs by manipulating pixels in a single frame of a training video. I’ve a use case in mind for this where educational video footage might be converted to 3D and certain spots get highlighted with this technique.
Dyn-E example
Interpolating between Images with Diffusion Models
This paper introduces a new method based on ControlNet for generating interpolation videos between two images. Might be a fun experiment to generate input videos for video-to-video models with it.
Stable Diffusion Interpolation examples
More papers & gems
- Photo-Sketch-Correspondence: Learning Dense Correspondences between Photos and Sketches
- Visual Instruction Inversion: Image Editing via Visual Prompting
- TEDi: Temporally-Entangled Diffusion for Motion Synthesis
@opticnerd is the winner of the Claire Silver ‘motion’ contest. His piece Inference is an homage to past analogue media as a young woman moves along the radio and TV dial.
@ranetas publishd a short film created with Pika Labs called “Momentos” depicting diverse artistic styles going through human emotions. A must watch.
@petravoice made a 12-second short for Claire Silver’s motion contest called ‘bliss’ which is a love letter to her visit to Bali.
Interviews
This week I finally got the chance to interview one of our Discord OG’s @CrazyPepeEth whose work is currently exhibited in France in Villa Malzac. CrazyPepe is a programmer who during the pandemic started to create art with his son @ErrorSkull and never looked back. Truly inspiring. Enjoy!
Tools & Tutorials
These are some of the most interesting resources I’ve come across this week.
NoGPU-Webui is a freemium service by OmniInfer that lets you use the Automatic1111 interface with SDXL 1.0 and a ton of others model for free using their cloud infrastructure in the backend.
ResShift is a new upscaler model that uses residual shifting and can achieve image super-resolution faster compared to other methods.
The image to 3D model One-2-3-45 from issue 40 now has a demo on HuggingFace Spaces.
Yamer, our Stable Diffusion fine-tuning expert within our Discord has put together a new model for SD 1.5. Can’t wait with what he comes up for SDXL.
Social media Hsiao-Ron Cheng abstract tiktok oil painting by yum yum gang gang detailed crowded modernity scene --c 15 --style raw --w 100 --ar 3:2. Made by me in our Discord.
And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:
- Sharing it 🙏❤️
- Following me on Twitter: @dreamingtulpa
- Buying me a coffee (I could seriously use it, putting these issues together takes me 8-12 hours every Friday 😅)
Reply to this email if you have any feedback or ideas for this newsletter.
Thanks for reading and talk to you next week!
– dreamingtulpa