AI Art Weekly #58
Hello there, my fellow dreamers, and welcome to issue #58 of AI Art Weekly! 👋
We didn’t get to see OpenAI’s androids on DevDay this week, but the new API capabilities, custom GPTs as well as xAI’s Grok are still pretty exciting. These changes might not look like much on first sight, but they mark the beginning of the end for search engines and standalone software. It’s gonna take a few more years, but in the future we’ll be talking to an AI assistant to get things done.
But before that is going to happen, let’s dive into this week’s highlights:
- OpenAI releases custom GPTs
- LRM: Adobe’s new image-to-3D model
- I2VGen-XL: A new image-to-video model
- Consistent4D: 360° dynamic object generation from a single video
- MeshNCA: Dynamic textures on 3D meshes
- Interview with artist ORGNLPLN
- and more tutorials, tools and gems!
Putting these weekly issues together takes me between 8-12 hours every Friday. If you like what I do, please consider buying me a coffee so I can stay awake 🙏
Cover Challenge 🎨
For next weeks cover I’m looking for “mythology” submissions. The reward is $50 and the Challenge Winner for the winner and the Challenge Finalist role for all finalists within our Discord community. These rare roles earn you the exclusive right to cast a vote in the selection of future winners. Rulebook can be found here and images can be submitted here. I’m looking forward to your submissions 🙏
News & Papers
Custom GPTs
OpenAI released the ability to craft your own GPTs.
Custom GPTs are basically ChatGPT with some additional instructions. The difference to simple prompt engineering comes from the ability to add custom data and connect it to external services through APIs. To summarize:
- Low entry barrier: Everyone can create a customized GPT through natural language, no coding required.
- Extendable knowledge: GPTs can be extended with external data through files and databases.
- Custom Actions: GPTs can fetch and send data to external tools through APIs.
- Multi-modality support: All GPTs can write and run code with code-interpreter, create art with Dalle-3, search the internet with web browsing.
- Revenue share: OpenAI announced that they will launch a GPT store later this month and that they will pay out revenue shares to GPT creators.
I predict that at some point in the near future the default ChatGPT will be able to access all the custom GPTs and their data. This will be our first taste of a single AI assistant that can do almost everything we’ve done with standalone software so far. Instead of having to learn and handle multiple apps, we’ll get things done through a simple chat interface.
I’ll deep diving custom GPTs over the next few weeks and I’m especially interested in building GPTs that can connect to external services. The first one I built is called CineTulpa. It’s a movie recommendation GPT based on my personal taste. If you create GPT yourself, please share it with me!
Sam Altman reveiling custom GPTs
I2VGen-XL
AI video generation has made some incredible progress this year. Semantic accuracy and temporal continuity are still a challenge though. I2VGen-XL is a new model that generates videos from images while trying to solve these issues.
I2VGen-XL example: A lonely tiger walks by the sea, at sunset, 2D culture.
LRM: Large Reconstruction Model for Single Image to 3D
Adobe is entering the image-to-3D game. LRM can create high-fidelity 3D object meshes from a single image in just 5 seconds. The model is trained on massive multi-view data containing around 1 million objects. The results are pretty impressive and the method is able to generalize well to real-world pictures and images from generative models.
LRM examples
Consistent4D: Consistent 360° Dynamic Object Generation from Monocular Video
Consistent4D is an approach for generating 4D dynamic objects from uncalibrated monocular videos. With the speed we’re progressing, it looks like dynamic 3D scenes from single-cam videos will be here sooner than I’ve expected the last few weeks.
Consistent4D examples
Mesh Neural Cellular Automata
Mesh Neural Cellular Automata (MeshNCA) is a method for directly synthesizing dynamic textures on 3D meshes without requiring any UV maps. The model can be trained using different targets such as images, text prompts, and motion vector fields. Additionally, MeshNCA allows several user interactions including texture density/orientation control, a grafting brush, and motion speed/direction control.
MeshNCA example
More papers & gems
- Cross-Image Attention for Zero-Shot Appearance Transfer
- Sewformer: Towards Garment Sewing Pattern Reconstruction from a Single Image
- LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills
@neonglitch86 built a complete 3D scene from prompt assets generated with Luma Labs new Genie text-to-3D model and used Mixamo to rig some of the characters.
@Yosun built an AR 3D app that lets you rip any product image into 3D printable parts using open-source image-to-3D AI models.
@MartinNebelong showcases the power of real-time AI transformation with LCM by connecting it with Photoshop.
@CreativeAIgency put together an AI movie trailer they always wanted to tell using Runway’s updated text-to-image feature.
Interview
Over the course of the next few issues, Anna Dart and I are bringing back some #AISurrealism interviews. Starting with ORGNLPLN
Tools & Tutorials
These are some of the most interesting resources I’ve come across this week.
@PurzBeats created a speedrun video tutorial for ComfyUI, multiple image IPAdapter, AnimateDiff-Evolved and ControlNet (QRCode).
YouTune is a CLI tool to fine-tune SDXL on images and MusicGen on audio from YouTube videos.
@radamar put together a HuggingFace space with the LCM model and ControlNet canny edge support.
CollageRL or “Neural Collage Transfer: Artistic Reconstruction via Material Manipulation” is a method to create collage style images based on a target image and training materials (like a dataset of newspaper articles).
And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:
- Sharing it 🙏❤️
- Following me on Twitter: @dreamingtulpa
- Buying me a coffee (I could seriously use it, putting these issues together takes me 8-12 hours every Friday 😅)
- Buy a physical art print to hang onto your wall
Reply to this email if you have any feedback or ideas for this newsletter.
Thanks for reading and talk to you next week!
– dreamingtulpa