DreamBooth: fine-tuning

How to train a Stable Diffusion model with DreamVault?

Welcome to the exciting world of diffusion model training, where you can create unique concepts using DreamVault. In this guide, we learn how to customize the Stable Diffusion model to represent a specific concept, such as your own face. With the right tools, you will be able to do this effectively, optimizing resources and getting exceptional results. Let's dive into the customization process.

What do you need to start training?

To train a diffusion model, a number of technical requirements are essential. These can be:

• Adequate RAM: A minimum of 10 GB of VRAM is needed to perform the training on your laptop or system.
• Suitable tool: We will use Automatic 11.11 for the training, or you can opt for Google Colab if you do not have the minimum hardware requirements.
• Educational resources: A detailed blog and a YouTube video are available to guide you step-by-step in running with Google Colab.

Make sure you comply with these points before proceeding.

How to set up Automatic 11.11 for training?

The setup process is simple if you follow these steps:

1. Extensions Installation: Login to Automatic 11.11, go to "Extensions", select "Available", and from there search for DreamVoot. Click "Install" to start the download from GitHub.
2. Restarting the instance: Once the installation is complete, restart Automatic 11.11. This will allow the DreamVoot tab to appear in the interface.

How to start model training.

Now that the environment is configured, continue with the following steps to train the model:

1. Model creation: go to DreamVoot and create your model; name it, for example, "Test". Make sure you are using Stable Diffusion version 1.5 and configure "Ancestral Euler" as scheduler.

   # Model configurationmodel_name = "Test"version = "1.5"scheduler = "Euler Ancestral"create_model(model_name, version, scheduler)

2. Parameter settings: Go to "Concepts" and set the directory of images you will use for training. Set the number of epochs and other parameters like learning rate and accuracy.

   # Training parametersepochs = 100learning_rate = 0.001precision = "FP16".

3. PROM key: Define a phrase or vector that will associate your face with the concept of "person", making sure it is not an existing word in English.

   # Custom PROMprom_vector = "Photo of Alarcon C Person".

4. Start training: Once everything is set up, proceed to train the model. The time it will take will depend on your GPU, but will vary between 40 to 45 minutes approximately.

   # Start trainingtrain_model()

How to evaluate the results and generate images?

Once the training is finished, you will be able to select the created model from the models section and generate images of your custom concept.

1. Model selection: Choose the model you have just trained (e.g. "Test").

2. Image generation: Try different art styles like "Portrait", "Digital Art" 4K, 8K, or "Pencil Sketch".

   # Image generationgenerate_image("Portrait of Alarcon C Person",  style="Digital Art 8K")

What's the next step?

Now that you've mastered the process, it's time for a challenge! Try training different diffusion models: it could be your face, a pet's face or any object you want to represent. Then, share your results and observations in the comments section - keep exploring and having fun on this path of learning with diffusion models!