Introduction

Sandman: Deeper Dreams Still is a fan comic, created with the assistance of AI, that follows Dream of the Endless as he seeks out a new creature in the universe that aspires to dream, and makes the judgement on whether it is worthy of that gift, or not.

This project was made with two goals in mind: to give us the opportunity to make a Sandman story of our own, and to test the limits of what AI art is capable of. The Sandman by Neil Gaiman is our favorite comic book, but making our own issue of Sandman, or any comic for that matter, would have been too large a task for just Luis and I. The debate on AI Art is tricky but, though the jury is still out on whether AI Art is art at all, we hope that this counts. We hope that through this project we have made something that is art by using AI as one of the tools in helping us realize our vision.

We chose to make a Sandman comic because Sandman is all about dreams. It's about the art we make, the stories we tell, the people we are, and the people we hope to be. It is the perfect world to tell this story, a story about dreams, and how they'll carry on after we're gone. I hope you think it’s cool! We certainly do.

Without further ado, here is Sandman: Deeper Dreams Still. You can read it by going through the pictures in the carousel, or by clicking this link to download the full resolution PDF from Google Drive. After the story itself, there's going to be a section explaining our methodology (for those interested). You can read the script and some notes in our Google Doc over here.

-- Juan Lam & Luis Larios, 2022.

Sandman: Deeper Dreams Still

Tap on the pictures to enlarge

Background & Methodology

Summary

1. Brainstormed story idea
2. Spent a long time experimenting (bulk of project)
3. Then wrote the script with limitations in mind
4. Spent a lot of time figuring out what AI could and couldn't do and then generated backgrounds according to script
5. Luis began laying out the panels
6. Luis drew inits, or put them together in Photoshop, which we fed to the AI so that it could paint over it.
7. Done! (After a lot of Photoshop).

Background

Creating this project was a lot of work (and a lot of fun). I discovered AI Art in the beginning of 2022, and was amazed (and somewhat terrified) of what it was capable of. I introduced my roommate Ethan to VQGAN + Clip, and eventually Disco Diffusion, and he quickly became an expert in AI Art, outpacing me and most everyone involved with AI Art in short order. Over that time, he helped to uncover everything Diffusion Models (like Disco Diffusion) could do. Inspired by this, and seeing the lack of narratives made with this tech, I set out to create a narrative project using AI Art. This is that project.

Of course, despite the increasing quantity and quality of documentation and instruction concerning AI Art, like The Traveler's Guide to The Latent Space (which is an incredible resource), AI Art is still fickle, limited, and very much novel. As such, Luis and I had to employ a lot of work arounds. In that sense, we had a creative partnership with the code that generated the images you saw in those panels. We had to "work with it", as it were.

The two main AIs we used were Disco Diffusion v5.2, and DALL-E 2, though we truly mostly used Disco.

Most of the project's development time was spent figuring out what the AI was capable of doing well, and what was out of question. We learned a lot of lessons.

Techniques & Experiments

Firstly, we decided to use Disco Diffusion to generate the backdrops and scenes. Disco has an artistic quality to it, and a surrealness that is very charming. And besides, Disco was the expertise of my roommate who pitched in with tips every now and then on "prompting" the AI, setting certain parameters, and choosing our models + visual transformers.

But what Disco is not good at is getting humans right. It's just hard, and lacking some of the innovations Ethan's made with Disco in recent weeks, wasn't worth the effort of trying to salvage via Photoshop.

We used DALL-E to help out with creating Morpheus and Matthew because DALL-E is better at coherent things, at the cost of being less artistic. But even with the might of DALL-E (which is great at making people), we couldn't just ask for Morpheus of the Endless, or Dream, or even "Sandman". Asking for any one of these got us either Morpheus from The Matrix or Sand Man from Spider-Man. Or a mix of both. Because the models only know what it's seen, it doesn't know how to draw the Morpheus we wanted. It hasn't been shown enough images of the DC Character "Sandman" AKA Morpheus AKA Dream of the Endless, to know what that really is, or even how to draw it.

You can imagine that was a headache to figure out on a comic that's supposed to be AI generated (it was a headache). But Luis had the idea to, instead of asking for Morpheus, ask for a digital painting or comicbook style painting of someone who inspired or looks like Morpheus. So, we ended up asking for a lot of goth Robert Smiths, goth Neil Gaimans, goth Brandon Lees, and goth Tom Sturridge's.

Just a few of the hundred or so images of Morpheus' face we tried to generate. Made with Majesty Diffusion. We used none of these.

That more or less did the trick, but we weren't too happy with it. No close-ups with Diffusion models. We needed DALL-E. That, with some help from Disco, was the solution. Though ultimately we still needed the help of Photoshop, as is evidenced by this picture.

Luis had to do a lot of photoshopping.

We also tried asking for the silhoutte of a goth man in, say, a field. Or a hall. But Disco Diffusion couldn't quite get that composition down and at the time we weren't really using DALL-E. We ended up having to do the backgrounds and foreground separately. Sometimes, what we would do to get a coherent but artistic picture is create an output with DALL-E (or a sketch by Luis), and feed it to Disco to "paint" over it.

Init image vs Disco output

Where style and prompt are concerned, we spent a lot of time deciding on the look and feel we wanted. I spent about two weeks just generating test renders of a meadow at dawn with a white pedestal (in the Dreaming), to figure out what prompt combination worked. We tried to keep it feeling a lot like Sandman. As such, when prompting, we settled on asking for backgrounds to be made in the style of Charles Vess and Mike Dringenberg -- that turned out incredibly.

Writing & Prototyping

While I was experimenting (generations with Disco took ~45 minutes per picture), I worked on the script. It was helpful to be able to "write around" things the AI couldn't do at all. Originally, Dream was meant to go on this adventure with Delirium, but there was really just no shot the AI could pull that off. We also played around with the idea of having Death join in, but that was only slightly more plausible. I was rereading Sandman when I thought to myself -- you know, I really like Matthew. So Matthew it was.

Once we had the complete script and had settled on what prompts to use (for consistency), as well as what we could do, I started generating the images according to the script, and Luis did the layouts.

Pictured above are some of Luis' sketches for the layouts

After all the images were generated, it was just a matter of handing it over to Luis for color correcting, laying it out, making the different elements, and making a complete PDF.

Conclusion

I want to thank everyone who was involved in Sandman, beginning with Neil Gaiman himself, as well as Charles Vess, Mike Dringenberg, and Dave McKean (who we used as reference for the cover). This comic is meant to be an homage to their work, and the work of all the other great artists we imitated for the dream sequence. Where this particular comic is concerned, I want to thank Luis Larios who was my partner in making this. It could not have been done without him. I also want to thank Ethan for his advice concerning Disco Diffusion, and Nikki Kershner for looking over the script.

If you've read Deeper Dreams Still and are worried about AI affecting the world of art, I just have this to say: I can't promise that AI won't change anything. It will change everything. But in making this project I learned very quickly that while AI can mimic the aesthetics of art, it can not mimic its substance. Substance comes from the experience, the soul, of the artist. That cannot be forged. Not by an AI, and not by anyone.

What I hope I achieved with Sandman: Deeper Dreams Still is to take the meaningless output of the AI and arrange it in such a way that it may have meaning, to make a relationship with a very human technology such that it may give me a seed, and I give it the soil it needs to grow.

Here's some extra generations that we never got to use. Hope you enjoyed, and thank you for your time :)