I trained a neural network on my drawings

12 November, 2022

Warning: this article is a bit outdated.

• • •

Download the model here: https://huggingface.co/netsvetaev/netsvetaev-free. It’s great for seamless patterns, abstract drawings, and watercolor-styled images. How to use it and train a neural network on your own pictures?

How to use .ckpt model

You download the .ckpt file, put it in your models folder. Then it all depends on what fork you're using. In InvokeAI, which I use, the models are added by usual loading invoke.py and executing !import_model command in console.

How to train neural network on your images if you don't have nvidia 4090?

Google has a wonderful service called Colab: Google gives you free server for about an hour. That's enough to build a model from 15-30-50 images.

First go here and get acquainted with colab (but don't touch anything): https://colab.research.google.com/github/ShivamShrirao/diffusers/blob/main/examples/dreambooth/DreamBooth_Stable_Diffusion.ipynb

All that is required from you, except for clicking through each code window and uploading pictures, is a hugging face token.

Register at huggingface.co, and then create a token in the settings: https://huggingface.co/settings/tokens

Insert the token here:

Check the box to save the model to google drive and name a folder, if you want (it will be created automatically)

Next, specify how we will call your style or object

This is not so important, but you need to remember this token to invoke your object.

Then upload the pictures:

You need to use the size of 512x512 pixels, and it is better to prepare them in advance. It's up to you to decide what to cut, but I recommend to cut it so that the composition and significant objects were preserved. You can also make several separate zoomed in and cropped pictures.

Depending on the number of images, you can change the number of training steps. A reasonable number is 100 steps * number of pictures. For 8 images (at least 8 is enough, btw) — 800 steps is enough. Too many steps is also bad: the generation will become too contrast and you will get artifacts on images (overtrained model).

Next, simply start each block in the colab with the Play button in the top left corner and wait for the completion of each code. When you get to the section shown above and start it, you will see the learning process. It will take about 10 minutes to prepare + another 15-30 minutes, depending on the number of pictures and steps.

After training you can test the model on test generation and convert it to .ckpt format:

Now you have the file on your Google disk: download it and add it to your favorite fork.

Additionally you can delete unnecessary files and finish the process.

Some examples from my model:

———
About me

My name is Artur Netsvetaev, I am a product manager, entrepreneur and UI/UX designer. I help with the development of the InvokeAI interface and have been using it myself since the beginning of this project.