AI Art Weekly #31
Hello there my fellow dreamers and welcome to issue #31 of AI Art Weekly! 👋
We finally managed to reach the milestone of 1500 subscribers. Thank you all for your support and for being part of this community! I deeply appreciate it 🙏
This weeks highlights are:
- MCDiff - animate frames by drawing arrows onto them
- IconShop – text-to-SVG icons
- Interview with AI artist DocT 🟡
- ClaireSilver’s “Collaborative” AI Contest Finalists and Winners
- MuseAI – a AI chatbot that acts as your art muse
Cover Challenge 🎨
For the next cover I’m looking for forgotten dreams, repressed trauma and idealized nostalgia because we’re exploring the theme of memories. The reward is another $50. Rulebook can be found here and images can be submitted here. Come join our Discord to receive feedback and talk future challenges. I’m looking forward to all of your artwork 🙏
Reflection: News & Gems
MCDiff: Motion-Conditioned Diffusion Model for Controllable Video Synthesis
What if you could animate an image by just drawing arrows onto it to determine how a scenery should shift or a character should move? That sounds pretty awesome to me and MCDiff makes it possible. It’s a model that generates a video from a starting image frame and a set of strokes, allowing you to specify the intended content and dynamics for synthesis.
IconShop: Text-Based Vector Icon Synthesis with Autoregressive Transformers
If you’ve been developing a user interface you might know the pain, finding and picking the right icons… IconShop is a method to generate, well, SVG icons. As simple as that. The icons might not be perfect to ome from a design perspective, but this is one puzzle piece for automated UI generation.
Sin3DGen: Patch-based 3D Natural Scene Generation from a Single Example
As the title says, Sin3DGen is a new method for 3D scene generation from a single example. The method uses a patch based approach to synthesize high-quality general natural 3D scenes with both realistic geometric structure and visual appearance, in large quantities and varieties. Among other things, the model is also able to edit scenes by removing, duplicating and modifying objects within it.
Piggyback-Color: Improved Diffusion-based Image Colorization via Piggybacked Models
Piggyback-Color presents a new method for image colorization using pre-trained text-to-image diffusion models. The method can achieve conditional colorization with additional inputs such as user hints and texts and is able to generate pixel-perfect alignment to the input (grayscale) image.
Ray Conditioning: Trading Photo-Consistency for Photo-realism in multi-view image generation
Ray Conditioning is a lightweight and geometry-free technique for multi-view image generation. You have that perfect portrait shot of a face but the angle is not right? No problem, just use that shot as an input image and generate the portrait from a another angle. Done.
Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesis
Okay, this one is pretty wild. What if you could record a video of someone or something and then rerender another video from 1st or 3rd person perspective? That’s exactly what Total-Recon does. The method also enables 3D video filters that attach virtual 3D assets to an actor or make it possible to remove objects within a scene.
GTN-Bailando: Genre Consistent Long-Term 3D Dance Generation based on Pre-trained Genre Token Network
Here’s a new one: GTN-Bailando is a pre-trained Genre Token Network (GTN) which is able to generate long-term 3D dance motions conditioned on music. Soon you’ll be able to ask your AI assistant to go dance in VR, yay… just kidding, but maybe not 😅
More papers and gems
- Photoconsistent-NVS: Long-Term Photometric Consistent Novel View Synthesis with Diffusion Models
- SA3D: Segment Anything in 3D with NeRFs
- TextMesh: Generation of Realistic 3D Meshes From Text Prompts
- LumiGAN: Unconditional Generation of Relightable 3D Human Faces
- Neucam: Inverting the Imaging Process by Learning an Implicit Camera Model
- PoseVocab: Learning Joint-structured Pose Embeddings for Human Avatar Modeling
- Make It So: Steering StyleGAN for Any Image Inversion and Editing
- Putting People in Their Place: Affordance-Aware Human Insertion into Scenes
- ActorsNeRF: Animatable Few-shot Human Rendering with Generalizable NeRFs
@ClaireSilver12 shared her “Collaborative” AI Contest finalists. There are some insanely cool projects in that thread worth checking out! Congratulation to the winners @alxalxalx, @dragos_badita, @PhoenixFawkes25, @MutagenSamurai and @bl_artcult!
@Grimezsz announced this week that she’ll split 50% royalties on any successful AI generated song that uses her voice. If you want to give this a shot, take a look at SoftVC VITS which I shared in issue 27 alongside a tutorial from @NerdyRodent on how to train a custom voice.
YouTuber Art from the Machine is working on a Skyrim VR mod that lets you talk to NPCs powered by ChatGPT and xVASynth. The voices still sound a bit mechanic and it takes a while to generate the responses, but remember, this will be the worst this tech will ever be. I at least am super excited for the future of gaming!
@Merzmensch created a trippy AI short film called ALICE using Gen-2 by @runwayml. I’m sure we’re going to see a lot of those in the near future and I’m all in for it.
@kenakennedy is hosting an explorative AI “hackathon/festival” in Berlin during May 11th-29th 2023 featuring 19 days of extended coliving with a vibehack finale weekend festival.
This hasn’t anything to do with art (except for the engineering part), but I wanted to share this just for the sheer fun of it. First robot soccer league wen? ⚽️🤖
Imagination: Interview & Inspiration
I only discovered this weeks artist two weeks ago when talking to Rebel Without Applause, and boy am I glad I did. DocT is one of a kind and his story is as well, so without further ado and in Doc’s own words: tallyho and let us dive in!
What’s your background and how did you get into AI art?
Art has been a significant part of my life since an early age, owing to my mother’s background in painting and curation. As my father worked in the development cooperation sector, I was fortunate enough to grow up in Germany, Kenya, Senegal, and Bangladesch, being exposed to a range of highly distinct art styles, concepts, and understandings.
As an adolescent, I became heavily captivated by street art, with the legendary Blek le Rat and King Robbo serving as my idols. Their creativity, audacity, and social critique inspired me to more candidly express my thoughts and opinions through the medium of graffiti, poetry, and hip hop. Those were the days when a good night meant that you had spray paint on your fingertips the next morning.
My aspiration was to become an author and journalist, and I even managed to publish my first articles in nationwide newspapers at the age of 16. However, a visit to Afghanistan in 2002, just a few months after the fall of the Taliban, altered my life’s course dramatically. After visiting my father, who was working there at the time, I determined that writing about the world’s events was insufficient. I wanted to be more involved. Upon my return to Germany, I enrolled in medical studies and studied art history in parallel, thanks to Germany’s nearly free university education. Although I did not obtain a complete art history degree, I was fortunate enough to receive a quite thorough education.
After graduation and training, I worked for years in the humanitarian and transitional aid sector in the Afghan-Pakistan border region and then later in Iraq. During this time I rediscovered my passion for writing and started to learn about photography to capture and document what I was encountering. I believe this is when I began searching for a more visual outlet to express my sentiments, mainly for the sake of maintaining my sanity.
However, it was not until mid-2021 that I became acquainted with NFTs before stumbling upon AI art tools in early 2022. I began with vqgan+clip before progressing to DALL-E and eventually Midjourney and Stable Diffusion.
Working with AI felt like receiving an additional pair of vocal cords that enabled me to visualize my ideas and concepts precisely the way I desired.
Do you have a specific project you’re currently working on? What is it?
I am currently working on two projects. One is a small collection titled “Art Underground“, which is a diverse 1/1 collection that spans my own artistic journey. I am placing iconic artworks representing milestones in art history in a street art setting using the post-photography style. “Art Underground“ represents the merging of diverse influences and techniques to create something new that is both familiar and unfamiliar. It also represents the power of art to transcend traditional boundaries and to break down barriers between seemingly disparate genres.
Secondly, I am also engaged in a more unconventional undertaking, the “Neurodigital Odyssey”, which takes a more radical approach. This series endeavors to offer an immersive experience to the viewer, to allow one to explore an artist’s mind and the pandemonium it houses within its dreamlike space.
The “Neurodigital Odyssey” delves into a range of scenes that encapsulate a mind teetering on the brink of collapse, where disorder has become commonplace and is harnessed as a tool for novel modes of creative expression. It is also the first series for which I am including AI generated sound atmospheres to create a more captivating audio-visual experience.
What does the color yellow mean to you?
I like to include the color yellow in many of my works, because I like it’s ambiguity. It can have a variety of meanings and interpretations, depending on cultural, personal, and contextual factors. It can represent joy and happiness, but can also be a sign of warning and caution. In some cultures, yellow is associated with cowardice or fear, whereas in others with wealth and prosperity. The meaning of the color yellow can vary widely making it very interesting to play with.
What drives you to create?
Being a dedicated pediatrician with an emphasis on neonatal and intensive care as well as my previous work in the humanitarian and development sector, I find myself regularly gazing deep into the abyss of human suffering, which can be emotionally challenging to say the least.
As a result, I have sought refuge in the realm of art, where I am able to translate these experiences and emotions into something tangible that - in a way - can continue to exist outside of me. This, for me, is both a passion and an escape, a means of expressing myself to the wider world.
What does your workflow look like?
I usually choose a conceptual approach for my work. I start by simply thinking about the theme and how it could best be visualized. Revisiting my diary is an essential component of this procedure, for it allows me to collect my thoughts and contemplate upon the subject matter. I do and write in it almost on a daily basis. I then decide on a topic, scene, style and color palette and perhaps the acoustic atmosphere for my audiovisual pieces. The rest is prompting across different tools, partial blending and quite excessive post-editing in PhotoShop.
Do you still keep a diary?
I do and write in it almost on a daily basis. I can highly recommend it to everyone. It helps a great deal in structuring thought (btw. this works best when done by hand, but I don’t want to bore you with the neurophysiology behind this). It’s also a great way to remember the little moments of enlightenment we so easily forget when throwing ourselves into that machine that makes you forget everything and which we call ‘every day life‘.
Editorial note: I can highly recommend getting into the habit of writing morning pages popularized by Julia Cameran’s The Artist’s Way.
What is your favourite prompt when creating art?
I can’t really say if I have a favorite prompt. What I do have are repeating themes. One would be “just because you should be going insane doesn’t mean you will go insane“. I also like including perspectives such as overhead view
. If things are not working out the way I want to, I sometimes also include --chaos 50
to generate something with more variation.
How do you imagine AI (art) will be impacting society in the near future?
I consider AI art the greatest democratization of access to artistic expression since our ancestors were pressing their hands against some wall of a cave. It has unlocked a whole new world of artistic expression, one that is accessible to anyone with a creative impulse and a willingness to explore.
This aligns nicely with the core principles of web3 and its paradigm in which technology and creativity converge to empower people to realize their own artistic vision.
Who is your favourite artist?
The first name that comes to mind would be BLAC.ai who has been a tremendously helpful force in this space. I would also like to include DeltaSauce and not only because he picked up the first piece I ever minted. I greatly enjoy art across a range of genres, so others certainly would include Michael Hafftka who has really defined for me what art can be in the context of web3. Furthermore the incredibly talented Ren AI, Indi Björnsson and DOPEMIND who introduced me to audio-visual art. For the post-photography genre it would be Pop Punk as the creator of the iconic County Fair collection, Andy Schwetz, Oliver Dahl and my long term web3 friend Historic Crypto. This list is terribly incomplete of course.
As for non-blockchain art, I would start with my aforementioned street art idols Blek le Rat and King Robbo, but also Max Beckmann and Caspar David Friedrich. Max Beckmann’s work is both bold and haunting. I am particularly drawn to his self-portraits, which capture the intense emotional turmoil of his life. His ability to convey complex emotions through his paintings is absolutely remarkable. As for Caspar David Friedrich, the deep sense of melancholy and introspection resonates with me. His landscapes are often desolate and empty, yet I feel that they possess a quiet beauty and evoke a sense of profound contemplation.
Anything else you would like to share?
Art is in the eye of the beholder, not the definer. You don’t need anyone’s approval to exercise your freedom of artistic expression. Be bold. Dare to break traditional concepts. You are the artist.
Also, I can only recommend to reach out to your fellow artists. In my estimation, our AI art niche is amongst the most genuinely kind and helpful corners of this entire space.
Creation: Tools & Tutorials
These are some of the most interesting resources I’ve come across this week.
@bl_artcult won the “Collaborative” AI Contest with his Poe chatbot “MuseAI”. The idea is to build an AI that could act as a ‘Muse’ for artists; to inspire, help create & add value. Absolutely love it. Used it to come up with this weeks cover art challenge. @bl_artcult if there is a way to get involved, let me know 😉✌️
Track-Anything is a flexible and interactive tool for video object tracking and segmentation, based on Segment Anything and XMem.
Prompt Reader is a simple standalone viewer for reading prompts from Stable Diffusion generated images outside the WebUI.
And that my fellow dreamers, concludes yet another AI Art weekly issue. Please consider supporting this newsletter by:
- Sharing it 🙏❤️
- Following me on Twitter: @dreamingtulpa
- Buying me a coffee (I could seriously use it, putting these issues together takes me 8-12 hours every Friday 😅)
Reply to this email if you have any feedback or ideas for this newsletter.
Thanks for reading and talk to you next week!
– dreamingtulpa