Ok. So far we've seen Stable Diffusion (SD) create the most fantastic portraits and visual compositions, from just a few word prompts and descriptions. But I asked, "How does that help me, as an artist? I mean, it can't all be portraits, right? What if I had a specific action scene in mind? Could I somehow harness the power of Ai to create the scenes that I wanted t to create?" That was challenge that I tackled this time around.
I began, as usual by uploading my own artwork for the reference image I wanted Ai to use. Then, I matched it with the most descriptive and specific prompts I could make, hit "Create Image", and hoped SD picked up on the character pose. The results came back as super awesome variations. Sadly though, none came close to the pose in my drawing. I changed my prompts multiple times, hoping that if I grouped words between commas, or just make them as simple as possible, the 'stupid' Ai would pick up on that. But as you can tell from my frustration, it all ended in failure. I could have settled for the composition SD randomly spewed out. But, I was determined to tame this beast, and after dozens of YouTube Ai videos, I found it. It was all in the settings.
There should be no surprise that things eventually boiled down to the technicalities. This is Ai after all. So I found that by manipulating the level of 'Prompt Strength' and 'Guidance Scale' (for SD version Easy Diffusion) I could control how much Ai influenced my reference image. My settings were set on default. As a result, Ai did what it pleased. I found that by lowering these settings, more of my art came through in the images produced. That's when I felt a ray of heavenly light escaped from between divine clouds and shone on my like a spotlight on a melodramatic thespian on stage.
Here are the different setting strengths. We start with the Ai images from the far left, to my own drawing and sketch.
Now, the Ai images here are not perfect. The hands in particular are terrible, and this is a known Ai challenge. But my point here is not to showcase finished pieces, rather I wanted to illustrate that Ai can be trained to capture your art's composition.
This got me all excited and I wasted no time making new Ai images from my own art and slapping them into Photoshop. I was gonna create a collage. So I organized my composition and cropped the the characters to the best of my ability and put them together. But when I done, something looked very odd. Something that no amount of blurring or layer correction could make right. Then it dawned on me. The problem was that each image had its own lighting (I sighed). And with no consistent light source/s, the finished collage just looked weird. So I bit my lower lip and decided to wrestle creating consistent lighting next. Stay tuned.