CoffeeVectors
[AI Art Weekly] CoffeeVectors, who is your favourite artist?
I don’t have a favourite artist when it comes to aesthetics. I draw inspiration from hundreds of sources across all kinds of mediums from novels, to renaissance paintings, fashion photography, anime and video games. Also I’m more drawn to individual pieces of an artist rather than entire bodies of work. I’m in love with the paintings of Tamara de Lempicka for example, but I’m not inspired by every piece.
Painting by Tamara de Lempicka
But I do have a favourite artist in terms of an overall perspective and that’s Leonardo Da Vinci. After reading Walter Isaacson’s giant biography on him, I was deeply inspired by something counter-intuitive—Da Vinci practically gave up painting towards the end of his life. The art seemed to have become incapable of sating his deep curiosity, perhaps ironically because of his success. So it faded in importance for him. And I’m strangely drawn to that because I think it shows that his art was in service to him rather than he a slave to it. You don’t have to define yourself by only the things you’re good at; by only what people know you for.
Also Da Vinci was someone who did not see art as a separate thing from engineering and science, but it was all part of a continuum of beauty and mystery. That’s how I tend to view things. I see art as an extension of communication, no different than coding, consoling a friend in difficult times, a birthday party, mathematics, comedy, or trying to organize and lead a team. There’s obviously major differences between all these things, and that’s usually what we see, but if you’re someone that can find the unity in all of it, the hidden bridges and tunnels that connect what is obviously different on the surface, I think that’s something really special. I love how Da Vinci seemed equally concerned about being seen as an engineer as much as an artist. And he was also a set designer, an event planner, all manner of things. It’s like he was trying to explore an underlying structure that was unaffected by categories and semantics. There’s something magical to me about people who can see things that way. They can create art outside of art.
[AI Art Weekly] What’s your workflow look like?
I come from a photography/video background so my approach is more about considering selections from a set of options. I’m usually running a local instance of Stable Diffusion. I like that because I have more control and since I have 24GB of VRAM, I can generate sets and “search” the latent space relatively fast (I can generate something like four images in 10 seconds with my initial settings). But if I’m not getting results I like, I’ll head over to MidJourney and Dall-E 2.
Starting in a local instance of Stable Diffusion I’ll come up with some medium sized prompts. Usually a specific concept followed by some fairly generic modifiers like 3d rendering, oil painting, photo etc. From there I’ll improvise playing with shorter/longer prompts and see what’s getting me closer to something interesting.
Once I have something that’s like 80% there, I start looking at “nearby” images in the latent space by walking through the different variables that are available to me in Stable Diffusion. Using something called an X/Y Plot script, I can generate a grid where the axis correspond to different values of whatever variable I want to walk through. So X could be steps and Y could be different sampler methods. I’ll do different X/Y Plots and see if there’s a different configuration of settings that gets my prompt closer to something more cohesive, with less artifacts, or that’s more interesting.
X/Y plot example by CoffeeVectors
I think one of the good things about taking a searching approach like this is that it helps alleviate some of the FOMO around wondering if you’ve really generated the best image. I know with MidJourney we sometimes can get into what a few AI Artists call “ doom prompting ” where we’re just endlessly trying tiny variations with the vague hope that one more round of image synthesis will get us that EXACT image we want. I don’t know about you but that makes me feel like a mouse in a lab experiment lol.
Once I have something I’m happy with, I think of it like a RAW image file or clip. I’ll usually bring that into Photoshop or another program and do additional work on it depending on what I’m going for. Could be generating depth maps for an animation, simple adjustments to fix remaining artifacts, or just upscaling and sharpening. Right now I’m mostly interested in seeing how AI tools integrate with other digital tools, but occasionally I’ll post something RAW right out of the AI.
[AI Art Weekly] How do you think AI Art tools will evolve in the future? What possibilities can you image?
There’s a short-term and a long-term answer.
In the short-term, say the next year or so, I think where we’re going is using AI to create a more direct relationship between 2D and 3D pipelines. That is, use AI to synthesize an image with prompts and then have that easily transfer into a 3-dimensional representation, either with some kind of AI that can generate meshes and project the 2D image onto textures (filling in missing information with inpainting), or it will generate some kind of NeRF (neural radiance field). Obviously we’d need to design these systems so that artists can build up, refine, and art direct the results as needed. The purpose would be getting to a starting point without having to build EVERYTHING from scratch while also having the controls to make deep changes to that automated starting point. If we can do that we open up an entire field of new process for animation, for being able to art direct 2D images more precisely and coherently, and dramatically speed up existing 3D workflows.
“AI + 3D. Scene made in #midjourney Characters from @daz3d #daz Post in PS and animation and rack focus in AE using a depth map made in a colab.”
For the long-term, I think we’re looking at developing the next generation of entirely new mediums, or variations on existing mediums, that are AI-first and that specifically require artists familiar with AI tools to create the content. It’s hard to say what these mediums might look like as they’ll have to emerge from how the community grows and what it discovers over the next several years, not to mention how the markets react and intersect with these new forms. They could revolve around complex world building on a scale that would take a human-only trad-art team too long to create. It could be something extremely dynamic that changes at a frequency, speed, and scale that would need a persistent creative intelligence to generate (so more than what procedural generation without an AI can do). For example a narrative-based video game that writes new dialog or character interactions as you go, or that even generates new game mechanics depending on player choices.
Editorial note: Checkout Character.AI in the Creation section below, I feel that’s a good example on how character dialogue could potentially be generated in games utilizing AI.
We could also end up with something like a Holodeck-style interface from Star Trek connected to a VR/AR experience where you’re able to create and control the environment directly through speech, without the need for detailed knowledge of computer graphics.
The holodeck is the ultimate evolution of virtual environment creation; large rooms capable of re-creating vistas, landscapes and environments for the purposes of training and recreation and even holographic characters with whom the user can interact and role play with.
[AI Art Weekly] Anything else you want to share?
If you ever get the chance to search the latent space for images “around” a prompt you like, you can see how small changes in variables can get you a totally different image. What this means is, you can have the “right” prompt, but because the other settings aren’t in place you’re not getting stuff you like. Put another way, a prompt is still latent space, just a smaller version. To explore that space requires going into the other controls available to you.
Or you can think of it in the opposite way—if you have the “wrong” prompt, but other settings are configured a certain way, you might end up with what you were originally going for. Sometimes the path to an image is straight. Sometimes the only path is to become lost. In truth both sets of paths probably exist at the same time, it just depends on chance which ones you happen to start closer to.
While I can imagine that knowing this might be a bit disheartening for some people, I think it counter-intuitively brings AI Art towards a more analog space. There’s a kind of chaos (or extremely complex, almost fractal order) present in the system. Depending on the configuration of tools you’re using, you can make a factory (which can have value in a lot of situations), or you can make a dynamic, unique experience of creation where you’re in relationship with a machine and the library of human creation. Or it can be a mix of both in different degrees. You’ll have to search to find what works best for you.