AI Art Weekly #24

Hello there my fellow dreamers and welcome to issue #24 of AI Art Weekly! 👋

I openend up a Discord server. If you’re looking to chat with some like-minded creators about AI art, animation, games, movies or whatever peaks your interest, come join us. In there we also brainstorm about the weekly challenge themes, give each other feedback on art pieces and share the latest shiny papers and tools that make their way into this newsletter. Don’t be a stranger!

This weeks highlights include:

  • GigaGAN: Large-scale GAN for Text-to-Image Synthesis
  • Word-As-Image turns letters into semantic visuals (perfect for logo design?)
  • Interview with artist and Cannes short film winner Glenn Marshall
  • New ControlNet models for style and palette guidance
  • ControlNet models for Stable Diffusion v2.1

Cover Challenge 🎨

The challenge for this weeks cover was “fashion” which received 82 submissions from 44 different artists. The community decided on the final winner:

  1. @trevorforrestm1 🏆
  2. @KingGregree116 🥈
  3. @K_o_S 🥈
  4. @NathanBoey 🥉

Congratulations to @trevorforrestm1 for winning the cover art challenge 🎉🎉🎉 and a big thank you to everyone who found the time to contribute!

The next challenges theme is “horror”, I don’t think a lot of explanations are needed for this one, just make sure it’s not too gory. The reward is another $50. Rulebook can be found here and images can be submitted here.

The new Discord is also a great place to ask and receive feedback to your challenge submissions.

I’m looking forward to all of your artwork 🙏

Reflection: News & Gems

GigaGAN: Large-scale GAN for Text-to-Image Synthesis

GANs are back with a bang baby! Training GANs with large datasets like LAION didn’t work well so far, GigaGAN aims to solve this by introducing a new architecture that far exceeds the old limits, bringing back GANs as a viable option for text-to-image synthesis. There are three major advantages: It is orders of magnitude faster at inference time, taking only 0.13 seconds to synthesize a 512px image. It can synthesize high-resolution images, for example, 16-megapixel pixels in 3.66 seconds. Finally, GigaGAN supports various latent space editing applications such as latent interpolation, style mixing, and vector arithmetic operations. Super exciting! Always wanted to get into StyleGAN interpolations, I guess now I’ll finally get to do that, no more excuses, except one, someone needs to open source some code 😅. So fingers crossed 🤞

Disentangled GigaGAN prompt interpolation example

T2I-Adapter updates

Remember last week when I fanboyed about Composer being able to let you control the color palette of images? Well, it seems right after that issue went out, TencentArc published two new T2I-Adapters. Style Adapter lets you use a reference image and Color Adapter a spatial palette (basically a 2d map color palette) for image generation guidance. If you want to try them out, there is a HuggingFace demo. You can also download the models and use them with the Automatic1111 ControlNet plugin. There are also canny, depth and pose models for Stable Diffusion v2.1 available as of late. Checkout the resources section below.

T2I Color Adapter (Spatial Palette)

Word-As-Image for Semantic Typography

I sometimes design logos (either for clients or friends) and Word-As-Image seems like a great way to find inspiration for designing an icon that goes along well with a specific font. The model is capable of transforming letters of a written word to semantically illustrate its meaning. For example transforming the shape of the letter “C” in the word cat, well, into a cat. To take this even further one could take that shape and stylize it with Stable Diffusion using ControlNet or the SDv2 depth2img model as illustrated on the project page.

Word-As-Image examples

Video-P2P: Video Editing with Cross-attention Control

We’ve already seen a few video editing models in the past few months. Video-P2P is the newest addition to this category. Quality for video production maybe isn’t there yet, but it’s always interesting to see specific tasks get better over time.

Video-P2P example: A Spider-Man ~~man~~ is driving a motorbike in the forest

X-Avatar: Expressive Human Avatars

Who wants to be a 3D VTuber? X-Avatar is a novel avatar model that captures the full expressiveness of digital humans to bring about life-like experiences in telepresence and AR/VR. The method models bodies, hands, facial expressions and appearance in a holistic fashion and can be learned from either full 3D scans or RGB-D data. If VTubing isn’t what you’re looking for (I wouldn’t blame you), X-Avatar might also be a good tool for just creating animations for movies or shorts.

X-Avatar examples


Last year, ANGIE was one of my most favourite papers, presenting the ability to generate videos of talking and gesturing people with only an input image and audio file. Unfortunately, to this date, the code stays behind closed doors. But there seems to be a promising alternative called SadTalker, although this one focuses only on animating faces without the gesturing part, the results look smooth and good on a wide array of different styles.

SadTalker examples

Imagination: Interview & Inspiration

In Today’s issue of AI Art Weekly, we talk to AI artist Glenn Marshall. Glenn is a 2 Prix Ars Electronica awards and Cannes Short Film winner. While I do enjoy still image art, there is something that pumps my heart when it comes to animation, and Glenn definitely delivers on that front, so let’s jump in.

[AI Art Weekly] Glenn, what’s your background and how did you get into AI art?

I first started working with 3D animation about 25 years ago. Around 15 years ago, I shifted my focus entirely to generative art using code. This field was much more exciting and innovative to me. I became fascinated with the abstract and metaphysical aspects of art and how to convey philosophical ideas using numbers and algorithms.

Soon after, the development of text-to-image synthesis with AI began, and I was an early adopter of “Big Sleep,” an open-source Google Colab project created by Ryan Murdock. Although rudimentary, it was revolutionary. I could articulate my imagination and watch as it was visualized by a machine, which sparked new creative areas of my brain.

Fight!” by Glenn Marshall

[AI Art Weekly] Do you have a specific project you’re currently working on? What is it?

Yes, it’s quite exciting, actually. I currently have short and feature films in development, and I’m using AI and Virtual Production (The Volume). I will be shooting in a few weeks and using AI for production design, as well as stylizing and rotoscoping live action elements. Unfortunately, I can’t provide many more details at the moment!

[AI Art Weekly] What drives you to create?

This new technology allows for the creation of things that have never been seen before and enables storytelling in ways that were previously impossible.

Standing forever” by Glenn Marshall

[AI Art Weekly] What does your workflow look like?

Browsing YouTube for videos that I could take and transform. Creating videos and animations has always been my main thing with AI. With Automatic1111, fine tuned stable diffusion models, ControlNet and other tools, the creative possibilities are getting crazier by the day and we’ve definitely reached the stage where AI is a serious creative tool for mainstream TV, film and Animation productions.

[AI Art Weekly] What is your favourite prompt when creating art?

I have never been a prompt wizard. Instead, I rely on transforming videos and stills into something unique using simple styling prompts. For instance, I have enjoyed applying the Giuseppe Arcimboldo, art nouveau style to some mindfulness coloring pages, using ControlNet to work on a piece by Escher or simply applying the Pre-Raphaelite painting style to a Robert Palmer music video.

“The end of the world is nigh.” by Glenn Marshall

[AI Art Weekly] How do you imagine AI (art) will be impacting society in the near future?

To say that this will cause disruption would be an understatement. Ironically, I believe human artists will become more valued. We’ve just gone through the phase of people saying “wow, can you believe a computer made this amazing art?”. But now, it’s so easy for anyone to create amazing AI art, that one day soon we might see a “real” painting and say “wow, can you believe a human made this amazing art?”.

[AI Art Weekly] Who is your favourite artist?

The Surrealists, namely Dali, Magritte, and Escher, are the obvious ones. I believe they would have been blown away by AI.

“Landscape of Cadaques” by Salvador Dali

[AI Art Weekly] Anything else you would like to share?

Thierry Frémaux, director of the Cannes Film Festival, wrote to congratulate me on the success of my AI film, ‘The Crow.’ He also informed me that his friend, Mylene Farmer, France’s top female pop star, was interested in using it during her concerts this summer. So I gave it to her to use!

Creation: Tools & Tutorials

These are some of the most interesting resources I’ve come across this week.

A blend of some old work by @moelucio and me, created by @moelucio

And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:

Reply to this email if you have any feedback or ideas for this newsletter.

Thanks for reading and talk to you next week!

– dreamingtulpa

by @dreamingtulpa