CoffeeVectors
[AI Art Weekly] CoffeeVectors, who is your favourite artist?
I donāt have a favourite artist when it comes to aesthetics. I draw inspiration from hundreds of sources across all kinds of mediums from novels, to renaissance paintings, fashion photography, anime and video games. Also Iām more drawn to individual pieces of an artist rather than entire bodies of work. Iām in love with the paintings of Tamara de Lempicka for example, but Iām not inspired by every piece.Ā
Painting by Tamara de Lempicka
But I do have a favourite artist in terms of an overall perspective and thatās Leonardo Da Vinci. After reading Walter Isaacsonās giant biography on him, I was deeply inspired by something counter-intuitiveāDa Vinci practically gave up painting towards the end of his life. The art seemed to have become incapable of sating his deep curiosity, perhaps ironically because of his success. So it faded in importance for him. And Iām strangely drawn to that because I think it shows that his art was in service to him rather than he a slave to it. You donāt have to define yourself by only the things youāre good at; by only what people know you for.Ā
Also Da Vinci was someone who did not see art as a separate thing from engineering and science, but it was all part of a continuum of beauty and mystery. Thatās how I tend to view things. I see art as an extension of communication, no different than coding, consoling a friend in difficult times, a birthday party, mathematics, comedy, or trying to organize and lead a team. Thereās obviously major differences between all these things, and thatās usually what we see, but if youāre someone that can find the unity in all of it, the hidden bridges and tunnels that connect what is obviously different on the surface, I think thatās something really special. I love how Da Vinci seemed equally concerned about being seen as an engineer as much as an artist. And he was also a set designer, an event planner, all manner of things. Itās like he was trying to explore an underlying structure that was unaffected by categories and semantics. Thereās something magical to me about people who can see things that way. They can create art outside of art.
[AI Art Weekly] Whatās your workflow look like?
I come from a photography/video background so my approach is more about considering selections from a set of options. Iām usually running a local instance of Stable Diffusion. I like that because I have more control and since I have 24GB of VRAM, I can generate sets and āsearchā the latent space relatively fast (I can generate something like four images in 10 seconds with my initial settings). But if Iām not getting results I like, Iāll head over to MidJourney and Dall-E 2.
Starting in a local instance of Stable Diffusion Iāll come up with some medium sized prompts. Usually a specific concept followed by some fairly generic modifiers like 3d rendering, oil painting, photo etc. From there Iāll improvise playing with shorter/longer prompts and see whatās getting me closer to something interesting.Ā
Once I have something thatās like 80% there, I start looking at ānearbyā images in the latent space by walking through the different variables that are available to me in Stable Diffusion. Using something called an X/Y Plot script, I can generate a grid where the axis correspond to different values of whatever variable I want to walk through. So X could be steps and Y could be different sampler methods. Iāll do different X/Y Plots and see if thereās a different configuration of settings that gets my prompt closer to something more cohesive, with less artifacts, or thatās more interesting.Ā
X/Y plot example by CoffeeVectors
I think one of the good things about taking a searching approach like this is that it helps alleviate some of the FOMO around wondering if youāve really generated the best image. I know with MidJourney we sometimes can get into what a few AI Artists call ā doom prompting ā where weāre just endlessly trying tiny variations with the vague hope that one more round of image synthesis will get us that EXACT image we want. I donāt know about you but that makes me feel like a mouse in a lab experiment lol.
Once I have something Iām happy with, I think of it like a RAW image file or clip. Iāll usually bring that into Photoshop or another program and do additional work on it depending on what Iām going for. Could be generating depth maps for an animation, simple adjustments to fix remaining artifacts, or just upscaling and sharpening. Right now Iām mostly interested in seeing how AI tools integrate with other digital tools, but occasionally Iāll post something RAW right out of the AI.
[AI Art Weekly] How do you think AI Art tools will evolve in the future? What possibilities can you image?
Thereās a short-term and a long-term answer.Ā
In the short-term, say the next year or so, I think where weāre going is using AI to create a more direct relationship between 2D and 3D pipelines. That is, use AI to synthesize an image with prompts and then have that easily transfer into a 3-dimensional representation, either with some kind of AI that can generate meshes and project the 2D image onto textures (filling in missing information with inpainting), or it will generate some kind of NeRF (neural radiance field). Obviously weād need to design these systems so that artists can build up, refine, and art direct the results as needed. The purpose would be getting to a starting point without having to build EVERYTHING from scratch while also having the controls to make deep changes to that automated starting point. If we can do that we open up an entire field of new process for animation, for being able to art direct 2D images more precisely and coherently, and dramatically speed up existing 3D workflows.
āAI + 3D. Scene made in #midjourney Characters from @daz3d #daz Post in PS and animation and rack focus in AE using a depth map made in a colab.ā
For the long-term, I think weāre looking at developing the next generation of entirely new mediums, or variations on existing mediums, that are AI-first and that specifically require artists familiar with AI tools to create the content. Itās hard to say what these mediums might look like as theyāll have to emerge from how the community grows and what it discovers over the next several years, not to mention how the markets react and intersect with these new forms. They could revolve around complex world building on a scale that would take a human-only trad-art team too long to create. It could be something extremely dynamic that changes at a frequency, speed, and scale that would need a persistent creative intelligence to generate (so more than what procedural generation without an AI can do). For example a narrative-based video game that writes new dialog or character interactions as you go, or that even generates new game mechanics depending on player choices.
Editorial note: Checkout Character.AI in the Creation section below, I feel thatās a good example on how character dialogue could potentially be generated in games utilizing AI.
We could also end up with something like a Holodeck-style interface from Star Trek connected to a VR/AR experience where youāre able to create and control the environment directly through speech, without the need for detailed knowledge of computer graphics.
The holodeck is the ultimate evolution of virtual environment creation; large rooms capable of re-creating vistas, landscapes and environments for the purposes of training and recreation and even holographic characters with whom the user can interact and role play with.
[AI Art Weekly] Anything else you want to share?
If you ever get the chance to search the latent space for images āaroundā a prompt you like, you can see how small changes in variables can get you a totally different image. What this means is, you can have the ārightā prompt, but because the other settings arenāt in place youāre not getting stuff you like. Put another way, a prompt is still latent space, just a smaller version. To explore that space requires going into the other controls available to you.
Or you can think of it in the opposite wayāif you have the āwrongā prompt, but other settings are configured a certain way, you might end up with what you were originally going for. Sometimes the path to an image is straight. Sometimes the only path is to become lost. In truth both sets of paths probably exist at the same time, it just depends on chance which ones you happen to start closer to.
While I can imagine that knowing this might be a bit disheartening for some people, I think it counter-intuitively brings AI Art towards a more analog space. Thereās a kind of chaos (or extremely complex, almost fractal order) present in the system. Depending on the configuration of tools youāre using, you can make a factory (which can have value in a lot of situations), or you can make a dynamic, unique experience of creation where youāre in relationship with a machine and the library of human creation. Or it can be a mix of both in different degrees. Youāll have to search to find what works best for you.