Ryan K. Hunter Ryan K. Hunter Ryan K. Hunter

Running ComfyUI and Stable Diffusion XL

One of the coolest innovations in recent days has been image generation AI. Stable Diffusion XL is the gold standard in locally-hosted image generation. It was created by StabilityAI, a visual AI startup that releases their work open source. So it's free, as in beer. In fact, unless otherwise noted, all the tools below should be free and open source.

Below you'll find a collection of guides, in a recommended order, for getting started with ComfyUI. The people who have written these guides have put a tremendous amount of work into them, so I see no reason to replicate it, aside from to offer a guiding hand on which step to take next. Happy generating!

ComfyUI

ComfyUI is a frontend for creating images with Stable Diffusion. While there are others out there, I've found that it is one of the best-supported and easiest to use out of the box.

It sould be noted that ComfyUI was developed by an employee at StabilityAI, which I've found has made it far superior to other frontends on the market at the moment. And once again.

ComfyUI GitHub Page: Link

ComfyUI Installer Documentation: Link

If you're feeling lost, here's a helpful installer guide: Stable Diffusion Art's ComfyUI Install Guide


Stable Diffusion XL

Now that you have your copy of ComfyUI installed, you're going to need a model to use with it. That would be Stable Diffusion XL. You'll need two files.

First, download the actual Stable Diffusion XL model here.

Next, download the Stable Diffusion VAE file here.

The main model (called a 'checkpoint') will go in your COMFYUI_DIR/ComfyUI/models/checkpoints/ folder, where COMFYUI_DIR is wherever you placed your ComfyUI installation.

The VAE file will go in your COMFYUI_DIR/ComfyUI/models/vae/ folder.


How the hell do I use this thing?

Well, I'm not going to pretend there isn't a learning curve. But ComfyUI is far more intimidating to look at than it is to use.

First, it's best to just get comfortable generating an image. Follow this section of the ComfyUI Install Guide from stable-diffusion-art.com. It should get you oriented a bit.

Once you have your feet wet, a StableAI QA employee by name of Scott Detweiler has put together a killer playlist on YouTube to help you along your journey. He's very straight and to the point, and shows his whole workflow. Many a YouTuber would be wise to take notes on how he does it.