Hello there, my fellow dreamers, and welcome to issue #44 of AI Art Weekly! 👋
Apart from that, there are a lot more exciting things that happened this week:
- Stable Diffusion XL 1.0 released
- Gen-2 now supports Image to Video
- WavJourney is a new system that uses LLMs to generate audio
- Dyn-E is a method for making local edits to NeRFs
- A new image Interpolating technique for diffusion models
- Interview with artist CrazyP̆̈ĕ̈p̆̈ĕ̈🫀🧠👁️
- And some more cool gems and papers
Cover Challenge 🎨
News & Papers
Stable Diffusion XL 1.0 released
The weights for SDXL 1.0 finally got released! The new model is using a new pipeline (base to refiner) which is able to generate more vibrant and accurate colors, with better contrast, lighting, and shadows than its predecessor and raises the image resolution from 768x768 to a full 1-megapixel (1024x1024) with multiple aspect ratios. You can use the model either with common web interfaces like ComfyUI or give it a try on Clipdrop. It works with Automatic1111 as well, but support for the refiner pipeline is lacking i think. Can’t wait to see what the community and the fine-tuners will come up in the next few days and weeks. Just remember, it’s called XL, not XXL.
Gen-2 Image-to-Video released
Gen-2 now supports Image to Video without the need for a text prompt. The new mode is a great way to explore consistent characters, locations and blocking from generation to generation. Most generations have not a lot of movement yet and apparently it works better with simpler compositions. To use it, just add an image to the Gen-2 interface without a prompt and hit Generate.
WavJourney: Compositional Audio Creation with Large Language Models
WavJourney is a system that uses large language models to generate audio content with storylines encompassing speech, music, and sound effects guided from text instructions. The demo results, while not perfect, sound great.
Dyn-E: Local Appearance Editing of Dynamic Neural Radiance Fields
Similar to the scribbles feature of INVE last week, Dyn-E is a method for making local edits to NeRFs by manipulating pixels in a single frame of a training video. I’ve a use case in mind for this where educational video footage might be converted to 3D and certain spots get highlighted with this technique.
Interpolating between Images with Diffusion Models
This paper introduces a new method based on ControlNet for generating interpolation videos between two images. Might be a fun experiment to generate input videos for video-to-video models with it.
More papers & gems
- Photo-Sketch-Correspondence: Learning Dense Correspondences between Photos and Sketches
- Visual Instruction Inversion: Image Editing via Visual Prompting
- TEDi: Temporally-Entangled Diffusion for Motion Synthesis
This week I finally got the chance to interview one of our Discord OG’s @CrazyPepeEth whose work is currently exhibited in France in Villa Malzac. CrazyPepe is a programmer who during the pandemic started to create art with his son @ErrorSkull and never looked back. Truly inspiring. Enjoy!
Tools & Tutorials
These are some of the most interesting resources I’ve come across this week.
And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:
- Sharing it 🙏❤️
- Following me on Twitter: @dreamingtulpa
- Buying me a coffee (I could seriously use it, putting these issues together takes me 8-12 hours every Friday 😅)
Reply to this email if you have any feedback or ideas for this newsletter.
Thanks for reading and talk to you next week!