AI Art Weekly #31

Hello there my fellow dreamers and welcome to issue #31 of AI Art Weekly! 👋

We finally managed to reach the milestone of 1500 subscribers. Thank you all for your support and for being part of this community! I deeply appreciate it 🙏

This weeks highlights are:

• MCDiff - animate frames by drawing arrows onto them
• IconShop – text-to-SVG icons
• Interview with AI artist DocT 🟡
• ClaireSilver’s “Collaborative” AI Contest Finalists and Winners
• MuseAI – a AI chatbot that acts as your art muse

---

Cover Challenge 🎨

---

Reflection: News & Gems

MCDiff: Motion-Conditioned Diffusion Model for Controllable Video Synthesis

What if you could animate an image by just drawing arrows onto it to determine how a scenery should shift or a character should move? That sounds pretty awesome to me and MCDiff makes it possible. It’s a model that generates a video from a starting image frame and a set of strokes, allowing you to specify the intended content and dynamics for synthesis.

IconShop: Text-Based Vector Icon Synthesis with Autoregressive Transformers

If you’ve been developing a user interface you might know the pain, finding and picking the right icons… IconShop is a method to generate, well, SVG icons. As simple as that. The icons might not be perfect to ome from a design perspective, but this is one puzzle piece for automated UI generation.

Sin3DGen: Patch-based 3D Natural Scene Generation from a Single Example

As the title says, Sin3DGen is a new method for 3D scene generation from a single example. The method uses a patch based approach to synthesize high-quality general natural 3D scenes with both realistic geometric structure and visual appearance, in large quantities and varieties. Among other things, the model is also able to edit scenes by removing, duplicating and modifying objects within it.

Piggyback-Color: Improved Diffusion-based Image Colorization via Piggybacked Models

Piggyback-Color presents a new method for image colorization using pre-trained text-to-image diffusion models. The method can achieve conditional colorization with additional inputs such as user hints and texts and is able to generate pixel-perfect alignment to the input (grayscale) image.

Ray Conditioning: Trading Photo-Consistency for Photo-realism in multi-view image generation

Ray Conditioning is a lightweight and geometry-free technique for multi-view image generation. You have that perfect portrait shot of a face but the angle is not right? No problem, just use that shot as an input image and generate the portrait from a another angle. Done.

Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesis

Okay, this one is pretty wild. What if you could record a video of someone or something and then rerender another video from 1st or 3rd person perspective? That’s exactly what Total-Recon does. The method also enables 3D video filters that attach virtual 3D assets to an actor or make it possible to remove objects within a scene.

GTN-Bailando: Genre Consistent Long-Term 3D Dance Generation based on Pre-trained Genre Token Network

Here’s a new one: GTN-Bailando is a pre-trained Genre Token Network (GTN) which is able to generate long-term 3D dance motions conditioned on music. Soon you’ll be able to ask your AI assistant to go dance in VR, yay… just kidding, but maybe not 😅

More papers and gems

• Photoconsistent-NVS: Long-Term Photometric Consistent Novel View Synthesis with Diffusion Models
• SA3D: Segment Anything in 3D with NeRFs
• TextMesh: Generation of Realistic 3D Meshes From Text Prompts
• LumiGAN: Unconditional Generation of Relightable 3D Human Faces
• Neucam: Inverting the Imaging Process by Learning an Implicit Camera Model
• PoseVocab: Learning Joint-structured Pose Embeddings for Human Avatar Modeling
• Make It So: Steering StyleGAN for Any Image Inversion and Editing
• Putting People in Their Place: Affordance-Aware Human Insertion into Scenes
• ActorsNeRF: Animatable Few-shot Human Rendering with Generalizable NeRFs

---

Imagination: Interview & Inspiration

I only discovered this weeks artist two weeks ago when talking to Rebel Without Applause, and boy am I glad I did. DocT is one of a kind and his story is as well, so without further ado and in Doc’s own words: tallyho and let us dive in!

What’s your background and how did you get into AI art?

Art has been a significant part of my life since an early age, owing to my mother’s background in painting and curation. As my father worked in the development cooperation sector, I was fortunate enough to grow up in Germany, Kenya, Senegal, and Bangladesch, being exposed to a range of highly distinct art styles, concepts, and understandings.

As an adolescent, I became heavily captivated by street art, with the legendary Blek le Rat and King Robbo serving as my idols. Their creativity, audacity, and social critique inspired me to more candidly express my thoughts and opinions through the medium of graffiti, poetry, and hip hop. Those were the days when a good night meant that you had spray paint on your fingertips the next morning.

My aspiration was to become an author and journalist, and I even managed to publish my first articles in nationwide newspapers at the age of 16. However, a visit to Afghanistan in 2002, just a few months after the fall of the Taliban, altered my life’s course dramatically. After visiting my father, who was working there at the time, I determined that writing about the world’s events was insufficient. I wanted to be more involved. Upon my return to Germany, I enrolled in medical studies and studied art history in parallel, thanks to Germany’s nearly free university education. Although I did not obtain a complete art history degree, I was fortunate enough to receive a quite thorough education.

After graduation and training, I worked for years in the humanitarian and transitional aid sector in the Afghan-Pakistan border region and then later in Iraq. During this time I rediscovered my passion for writing and started to learn about photography to capture and document what I was encountering. I believe this is when I began searching for a more visual outlet to express my sentiments, mainly for the sake of maintaining my sanity.

However, it was not until mid-2021 that I became acquainted with NFTs before stumbling upon AI art tools in early 2022. I began with vqgan+clip before progressing to DALL-E and eventually Midjourney and Stable Diffusion.

Working with AI felt like receiving an additional pair of vocal cords that enabled me to visualize my ideas and concepts precisely the way I desired.

Do you have a specific project you’re currently working on? What is it?

I am currently working on two projects. One is a small collection titled “Art Underground“, which is a diverse 1/1 collection that spans my own artistic journey. I am placing iconic artworks representing milestones in art history in a street art setting using the post-photography style. “Art Underground“ represents the merging of diverse influences and techniques to create something new that is both familiar and unfamiliar. It also represents the power of art to transcend traditional boundaries and to break down barriers between seemingly disparate genres.

Secondly, I am also engaged in a more unconventional undertaking, the “Neurodigital Odyssey”, which takes a more radical approach. This series endeavors to offer an immersive experience to the viewer, to allow one to explore an artist’s mind and the pandemonium it houses within its dreamlike space.

The “Neurodigital Odyssey” delves into a range of scenes that encapsulate a mind teetering on the brink of collapse, where disorder has become commonplace and is harnessed as a tool for novel modes of creative expression. It is also the first series for which I am including AI generated sound atmospheres to create a more captivating audio-visual experience.

What does the color yellow mean to you?

I like to include the color yellow in many of my works, because I like it’s ambiguity. It can have a variety of meanings and interpretations, depending on cultural, personal, and contextual factors. It can represent joy and happiness, but can also be a sign of warning and caution. In some cultures, yellow is associated with cowardice or fear, whereas in others with wealth and prosperity. The meaning of the color yellow can vary widely making it very interesting to play with.

What drives you to create?

Being a dedicated pediatrician with an emphasis on neonatal and intensive care as well as my previous work in the humanitarian and development sector, I find myself regularly gazing deep into the abyss of human suffering, which can be emotionally challenging to say the least.

As a result, I have sought refuge in the realm of art, where I am able to translate these experiences and emotions into something tangible that - in a way - can continue to exist outside of me. This, for me, is both a passion and an escape, a means of expressing myself to the wider world.

What does your workflow look like?

I usually choose a conceptual approach for my work. I start by simply thinking about the theme and how it could best be visualized. Revisiting my diary is an essential component of this procedure, for it allows me to collect my thoughts and contemplate upon the subject matter. I do and write in it almost on a daily basis. I then decide on a topic, scene, style and color palette and perhaps the acoustic atmosphere for my audiovisual pieces. The rest is prompting across different tools, partial blending and quite excessive post-editing in PhotoShop.

Do you still keep a diary?

I do and write in it almost on a daily basis. I can highly recommend it to everyone. It helps a great deal in structuring thought (btw. this works best when done by hand, but I don’t want to bore you with the neurophysiology behind this). It’s also a great way to remember the little moments of enlightenment we so easily forget when throwing ourselves into that machine that makes you forget everything and which we call ‘every day life‘.

Editorial note: I can highly recommend getting into the habit of writing morning pages popularized by Julia Cameran’s The Artist’s Way.

What is your favourite prompt when creating art?

I can’t really say if I have a favorite prompt. What I do have are repeating themes. One would be “just because you should be going insane doesn’t mean you will go insane“. I also like including perspectives such as overhead view. If things are not working out the way I want to, I sometimes also include --chaos 50 to generate something with more variation.

How do you imagine AI (art) will be impacting society in the near future?

I consider AI art the greatest democratization of access to artistic expression since our ancestors were pressing their hands against some wall of a cave. It has unlocked a whole new world of artistic expression, one that is accessible to anyone with a creative impulse and a willingness to explore.

This aligns nicely with the core principles of web3 and its paradigm in which technology and creativity converge to empower people to realize their own artistic vision.

Who is your favourite artist?

The first name that comes to mind would be BLAC.ai who has been a tremendously helpful force in this space. I would also like to include DeltaSauce and not only because he picked up the first piece I ever minted. I greatly enjoy art across a range of genres, so others certainly would include Michael Hafftka who has really defined for me what art can be in the context of web3. Furthermore the incredibly talented Ren AI, Indi Björnsson and DOPEMIND who introduced me to audio-visual art. For the post-photography genre it would be Pop Punk as the creator of the iconic County Fair collection, Andy Schwetz, Oliver Dahl and my long term web3 friend Historic Crypto. This list is terribly incomplete of course.

As for non-blockchain art, I would start with my aforementioned street art idols Blek le Rat and King Robbo, but also Max Beckmann and Caspar David Friedrich. Max Beckmann’s work is both bold and haunting. I am particularly drawn to his self-portraits, which capture the intense emotional turmoil of his life. His ability to convey complex emotions through his paintings is absolutely remarkable. As for Caspar David Friedrich, the deep sense of melancholy and introspection resonates with me. His landscapes are often desolate and empty, yet I feel that they possess a quiet beauty and evoke a sense of profound contemplation.

Anything else you would like to share?

Art is in the eye of the beholder, not the definer. You don’t need anyone’s approval to exercise your freedom of artistic expression. Be bold. Dare to break traditional concepts. You are the artist.

Also, I can only recommend to reach out to your fellow artists. In my estimation, our AI art niche is amongst the most genuinely kind and helpful corners of this entire space.

---

Creation: Tools & Tutorials

These are some of the most interesting resources I’ve come across this week.

And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:

• Sharing it 🙏❤️
• Following me on Twitter: @dreamingtulpa
• Buying me a coffee (I could seriously use it, putting these issues together takes me 8-12 hours every Friday 😅)

Reply to this email if you have any feedback or ideas for this newsletter.

Thanks for reading and talk to you next week!

– dreamingtulpa