Fun with Stable Diffusion

Good Morning Friends and Happy Juneteenth! Yesterday was my Birthday in addition to being Father’s Day, and I am super thankful for all the assorted well wishes. This morning is going to be a bit of a departure from my regularly scheduled ARPG nonsense. Over the weekend I spent some time messing around with Stable Diffusion running locally, and I thought I would talk a bit about it. Let’s get some stuff out of the way first. I do not condone Art Generation models as a method of replacing the work of actual artists. You will see a bunch of images adorning this blog, and they are all for the most part the paid creation of my good friend Ammo. In fact, as we speak she is working on yet another one of my hair-brained ideas and I have no clue what the final bill will even be, but will pay it happily as I always do. For me, the “AI Art” landscape is more a toy than a tool and in the past, I have enjoyed feeding it nonsense and seeing what it comes up with out of that chaos. For example… I have no clue who the fuck this dude is but any time I feed it a prompt with the name “Gideon” in it, I end up with this visage.

I’ve known for a while that you could run various generation tools locally off your graphics cards, but I always assumed it would be a tedious process. I started down this present rabbit hole when I found out that there was literally a one-click installer that set everything that you needed up for you. There is a distribution of Stable Diffusion called “Easy Diffusion” that offers a quick install for Windows, Mac, and Linux, and within about 15 minutes I was up and running and generating nonsense. I am sure there is probably something inferior about the path that I took to get to this destination, but I honestly don’t really care. I wanted to see if it could be done, and have gotten sufficient enjoyment from this digital version of wooly willy.

The end result is a web server running on your local machine, that you can then access from any other machine on your network. By default, this runs on port 9000 and gives you a fairly detailed interface to control the process. You are going to need some prompts and honestly… the best guide that I have found about this process is this one. More specifically it was helpful in understanding the concept of negative prompts… aka the things that you want to steer the engine away from creating. This is going to be really important if you are working with anything that could be considered the human form because like a pre-teen boy… it seems to be drawn to replicating boobs in the strangest ways. There are negative prompts that you can supply to the process in order to sort of steer it away from that particular uncanny valley.

The other thing you are probably going to want is some more models to play with. You could get really deep in the weeds in trying to explain exactly what a model is… but effectively think of it as encoded data that tells Stable Diffusion how it should produce images. The best place for these seems to be CivitAI.com and on the front page, you will find a number of the more popular options. I’ve played with several of these and after fiddling around a bit, I think I probably like DreamShaper the best because it tends to lean towards more imaginative imagery rather than attempting to replicate reality. Wherever you installed Easy Diffusion, you should find a directory along the lines of “EasyDiffusion\models\stable-diffusion”. When you download a model from CivitAI it will be a “.safetensors” file and you just need to drag it into this directory in order to use it. Something important to note… each model is roughly 2 gigabytes in size so they can rapidly fill your drive if you download too many.

Let’s walk through a multi-hour deep dive that I did yesterday in trying to get something interesting out of the generator. I wanted a Kaiju attacking a city, but I assumed that maybe the model wouldn’t understand the term Kaiju. Essentially when writing a prompt I find it best to sort of use simple language. So for example this is the prompt that I supplied to get the above image.

A giant monster attacking a modern city with the military fighting back against it

This was maybe a little too on the nose and had a distinct Godzilla feel to it. Since I tend to generate five images at a time, you can often see it going in a bunch of directions. Some of these were more akin to something that you would find in a Doom video game, but I mostly liked the general direction it was going. Essentially what I usually do is feed the image that was just generated back into the prompt in order to iterate on that idea. I find that over the course of what ends up being a few hundred generations, it slowly narrows down the focus to get closer to what you were actually wanting.

I wanted to go in a different direction, so I tweaked my prompt a little bit and fed it the previous image.

A giant monster that looks like Cthulhu attacking a modern city with the military fighting back against it

Basically, I supplied that I wanted it to look more like Cthulhu to steer it away from Godzilla… which worked like a charm. However, remember that bit where I said that the models seem to really want to draw boobs? I included this specific image just to show that point.

I took a bit of a pivot because I didn’t want this to end up being a pretty boring drab scene. So for the next prompt, I started adding some style elements to it. I also wanted the city to look a bit more ruinous.

a vaporwave scene featuring A giant monster that looks like Cthulhu attacking a modern city knocking down buildings some of the rubble on fire

I noticed that some of the elements of the monster how were mechanical looking, which made me start shifting in a slightly different direction. Could I get this to make it give me a scene of a Kaiju fighting a Robot over a ruined landscape? Unfortunately at this point going forward… I don’t have exact prompts. The images that generate are named based on your prompt, and my phrase got too long for the file character limit.

I kept the prompt above but instead added that it was fighting a Giant Robot that was wielding a laser sword. The generator got confused as it often does… and just started making the monster look more robotic. While extremely cool looking… this was not really what I was going for, but it was a key step in the process nonetheless.

I tried a few more rounds of generation, feeding my favorite from the previous round into the next round… but no matter what I tried it seemed to be hung up on a single “monster” figure. I am including this one mostly just because it looked pretty freaking badass. I have no clue what is going on with that building but I am on board with the bio-organic mech with a giant laser sword thing that is going on here.

It was around this point that I decided that I needed to tweak things up a bit further. So I specifically called out that there were two figures, the monster, and the giant robot and this finally began to produce paydirt. It started out a bit ephemeral at first, with this pseudo-robot-building thing in the background. However, that gave me a thread to expand on, again feeding one generation into the next round of five generations.

After a few more rounds of generation, the idea was starting to finally take root. I say idea because in fiddling with this nonsense, it does seem like the generator gets something stuck in its head and you have to sort of forcibly dislodge it at times. I was getting somewhere though, but knew it would take many more instances of taking an image that was the closest to what I wanted, and then feeding it into the next loop… and doing this over and over until the results started to turn in the direction I wanted it to.

Then after legitimately two or three hundred rounds of this nonsense and the course of an entire afternoon wasted… we have this glorious piece of nonsense. I think what I find so interesting about Stable Diffusion so far, is that it can serve as this rapid ideation platform. If you want to quickly iterate on some ideas you had in your head, you can come up with something that is still… very visually wrong at times but contains the flavor of what you wanted. I could see this being honestly an amazing tool for an “actual artist” to test out some ideas and have the machine keep iterating on something until they get a layout and subject matter that they wanted… that could then serve as the scaffolding to build something interesting. Even for wordsmiths, this could be super interesting because I can already wrap a story in my head around what is going on in this picture.

It is also sometimes just super interesting to feed it a prompt and see where it goes. This is a hundred or so generations off of the prompt “Belghast”. No clue why but it seemed to really latch onto a military and zombies theme when I used my chosen moniker as the prompt.

This delightfully ominous scene was generated with this prompt:

a skinwalker stalking a group of hunters in the woods

So while I would not at all consider that to be a successful prompt as far as subject matter goes… it still looks freaking cool nonetheless. Again this image has a story to it that is just begging to be told. I feel like trying to make one of these image generators create exactly the image that you were wanting… is a path to madness. However, if you sort of go with the flow and iterate on the patterns that you are seeing emerging… it can produce some really interesting things. While I don’t exactly consider this art on the same level as the things being produced for me by Ammo… there is definitely an art form that is emerging from guiding the machine. It somewhat reminds me of carving a woodcut block, and allowing yourself to lean into the imperfections of the material… rather than trying to fight against them.

Anyways I thought this was interesting. No clue if anyone else cares about it, and I have no clue if I will ever walk through one of my generative steps again in the future. The cool thing about this blog is it is a “me” blog more than it is a blog devoted to any one particular topic. I’m enjoying creating nonsense with Stable Diffusion and thought I would share that with you all.

The Wi-Fi Miracle

Aging with Technology

high-contrast-keyboard

It has been a really weird morning so far, and as a result I am getting an extremely late start getting a blog out today.  For starters I was up pretty late last night watching episodes of Lost Girl on Netflix.  Neither of us really seemed to want to get out of bed this morning, seeing as for both of us it was a holiday.  The original plan was to go up to my mother-in-laws and set up a printer.  However there were several other issues that needed to be dealt with at all.  She is in her 70s, and the fact that she is on Facebook is pretty damned impressive to me.  However technology as a whole is this black box for her, and she seems to be unable to decouple the existence of the internet with the existence of her laptop.

For weeks she has complained that her laptop isn’t working, and in truth it works completely fine.  What is at fault however is her internet connection.  Quite simply put she doesn’t have one.  That is not to say she does not sometimes have one, but internet is not a thing she pays for.  Instead years ago I installed a wifi router at my nieces house that lives adjacent to her.  The intent was to provide me internet access when I was up there during a family illness so I could continue to work while still being available.  By some freak if science the wifi signal was usable over at my wife’s mothers house. 

The Wi-Fi Miracle

Using Google maps to measure the approximate distance it is well over 200 ft, and passes through lots of walls between the access point at and the location where my mother-in-law has her desk.  It was absolutely amazing that it ever worked, let alone has continued to work until recently with a over seven year old wireless router.  But she cannot grasp or does not seem to believe that it is simply an issue of not enough signal strength.  At this point it could be so many different things, her new laptop could simply have a weaker wireless card, since good wi-fi coverage is just an assumed fact these days.  It could be that she piled more stuff in the back bedrooms, which the signal has to pass through.  Or it could simply be that the wireless access point is slowly dying.

Whatever the case she feels there is a  grand conspiracy to deny her access to Facebook.  In truth I was really not ready to deal with this issue today as I have never attempted to set up a wireless repeater, let alone tried to make two different access points work together.  We have a graveyard of old wireless routers laying around the house as we seem to burn through them every couple of years.  The last thing I wanted to do was try and install a semi-functional technology in the home of someone completely technologically confused…  and end up having to do house calls two hours away.  So we rescheduled for next weekend, and this week I need to research some options and get something ordered from Amazon.

Gauntlet Revival

Hammerwatch 2014-01-19 16-29-59-38 I am not really sure if yesterdays Steampowered Sunday post adequately covered how much I really enjoyed playing Hammerwatch.  By the time I stopped to write my blog post, I had played roughly 66 minutes.  By the end of the day I had logged well over 4 hours.  Shortly after posting my blog entry, I hooked up with 3 other friends and we discovered a bunch of new things about the game.  Firstly when doing multi-player you have the ability to add crutches.  Since we were still obviously learning the game, we added two really important ones.  Firstly we gave ourselves unlimited lives, which allowed us to brute force a few horrible areas of the game by simply respawning back into combat and bashing our faces against a wall until we got through it.  Additionally we added hit point regeneration… which was extremely slow… but just enough to help take the edge off things a bit.

With these added bonuses… we were able to push through to the third set of stages and down two world bosses.  I have to say the game just keeps getting more enjoyable and trickier.  It seems like each of the character classes has some really strong abilities.  The paladin that I was playing eventually got the ability to proc a stun on mobs, as well as the ability to deflect missile attacks from an ever widening frontal arc.  The warlock began getting some life drain abilities which greatly improved its sustain.  While I didn’t pay that close of attention to the Ranger or Wizard they both also seemed to get more and more lethal over time.  I have to say sale or no, this game is very much worth picking up.  The above image if from a really cool Gauntlet bonus level that we unlocked.  So much nostalgia. so many feels.