AI Art Weekly #83

Hello there, my fellow dreamers, and welcome to issue #83 of AI Art Weekly! 👋

While strolling through the Turia River park this week, OpenAI and Google were propping up ChatGPT and Gemini to become our personal AI assistants, tutors, and even companions for everyday life. Looking at the insane capabilities these models are gaining, it’s hard to not feel dread about the future of work.

But it’s also hard to not feel excited about the possibilities they bring. Surrounded by the beauty of the park, I imagined an utopian world where bullshit jobs are a thing of the past. In that moment, the dream felt real — I was creating art, doing what I love, and feeling truly happy. Although it still seems distant, this future may be closer than we think.

In this issue:

• GPT-4o is more than a virtual voice assistant
• 3D: CAT3D, Dual3D, Coin3D, Toon3D, LayGA
• Image: T2V-NPR, Analogist, LogoMotion, BlobGEN, SOEDiff
• and more!

---

Cover Challenge 🎨

---

News & Papers

GPT-4o

Besides becoming a virtual voice assistant replica of Samantha from the movie Her, GPT-4o has also some other remarkable capabilities almost no one talks about. Let’s check them out!

Visual narratives

GPT-4o can generate visually coherent stories either from only text prompts or from a combination of text and images.

Image creation and editing

It is also able to generate and edit design work based on a combination of text and images. For instance movie posters.

Poetic typography

Its abilities to generate text are mind blowing.

Image stylization

And it can stylize existing images.

3D reconstruction

It can generate multi-view images and turn them into 3D objects.

Image inpainting

And inpaint logos onto images.

And this is just one small subset of the things it can do. You can check out more of these capabilities on the announcement blog post.

3D

CAT3D: Create Anything in 3D with Multi-View Diffusion Models

Besides Gemini, Google presented CAT3D this week. It can turn any number of images into a 3D scene. The resulting scenes can be rendered interactively and the total processing time, including both view generation and 3D reconstruction, runs in as little as one minute.

Dual3D: Efficient and Consistent Text-to-3D Generation with Dual-mode Multi-view Latent Diffusion

Dual3D is yet another text-to-3D method that can generate high-quality 3D assets from text prompts in only 1 minute.

Coin3D: Controllable and Interactive 3D Assets Generation with Proxy-Guided Conditioning

Coin3D can generate and edit 3D assets from a basic input shape. Similar to ControlNet, this enables precise part editing and responsive 3D object previewing within a few seconds.

Toon3D: Seeing Cartoons from a New Perspective

Toon3D can generate 3D scenes from two or more cartoon drawings. It’s far from perfect, but still pretty cool!

LayGA: Layered Gaussian Avatars for Animatable Clothing Transfer

LayGA is a new Gaussian Avatar representation that can separate body and clothes into separate layers from multiview videos and can transfer garments to different bodies.

Image

T2V-NPR: Text-to-Vector Generation with Neural Path Representation

T2V-NPR can generate vector graphics from text or images. The method is also able to optimize the generated SVGs with adjustable levels of details and different styles, as well as animate them based on a text prompt describing the desired motion.

Analogist: Out-of-the-box Visual In-Context Learning with Image Diffusion Model

Instead of training a model for each specific task like deblurring or colorization, Analogist uses a flexible in-context learning approach with pre-trained diffusion models. This method only needs a few example pairs to handle various visual tasks, including denoising, low-light enhancement, image translation, style transfer, motion transfer, pose transfer, inpainting, and more.

LogoMotion: Visually-Grounded Code Generation for Content Aware Animation

LogoMotion can turn logos from layered PDF files into content-aware animated HTML canvas animations. Very cool!

BlobGEN: Compositional Text-to-Image Generation with Dense Blob Representations

BlobGEN is a new text-to-image model by NVIDIA that is able to generate images based on blob representations. These blob representations can be automatically extracted from a scene and then be used to guide the image generation process.

SOEDiff: Efficient Distillation for Small Object Editing

You ever tried to inpaint smaller objects and details into an image? Can be kind of a hit or miss. SOEDiff has been specifically trained to handle these cases and can do a pretty good job at it.

Also interesting

• Text Scene Motion: Generating Human Motion in 3D Scenes from Text Descriptions
• FastSAG: Towards Fast Non-Autoregressive Singing Accompaniment Generation

And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:

• Sharing it 🙏❤️
• Following me on Twitter: @dreamingtulpa
• Buying me a coffee (I could seriously use it, putting these issues together takes me 8-12 hours every Friday 😅)
• Buying a physical art print to hang onto your wall

Reply to this email if you have any feedback or ideas for this newsletter.

Thanks for reading and talk to you next week!

– dreamingtulpa