The RPGMaker sprites were fairly easy to do, 90% of the effort was just creating the dataset.
-
Collect a silly amount of assets. In order to get good results, you will need a lot of images. The main reason I got decent results with the RPGMaker sprites is because all the sprites are very similar so it was easier to train with. I just went into the folder I store all my RPGMaker games, copied all the folders called “Characters” or “Faces” depending on what I was working on and shoved them all in one place.
-
Download a program called ImageMagick - Download . Makes preparing the data much easier and it super nifty.
-
I recommend doing a quick pass on cleaning up your dataset before you start modifying. You can do a group by dimensions to get rid of all the files that don’t match the dimensions of what you are trying to do. In the case of tall sprites, all the 3x4 sheets are 360x480 so I just deleted all the sprites that didn’t fit those dimensions.
-
Start preparing the dataset. I’d recommend making a copy of everything before you start doing commands because the commands will change the originals (You could do different commands to keep the originals but I’m lazy and it easier to just use the mogrify command). First step is to remove the transparency, which you can do by shift+right clicking on the folder all of your assets are in, open PowerShell or whatever command line thing you use and type
“magick mogrify -background white -alpha remove -alpha off *.png” which should result in all of your images starting to look like this.
-
Convert all the images to .jpg, it much faster and easier for the AI to work with by doing the command “magick mogrify -format jpg *.png”
-
Start cropping out all the individual sprites. The tall sprites are 120x120 px so I’ll use the command “magick mogrify -crop 120x120 *.jpg”. Note this will just cut out all the sprites. If you just want one particular sprite then you can do something like “magick mogrify -crop 120x120+120+0 *.jpg” which will crop out just the section that is 120px from the left, 0px from the top which is just the front facing sprite. Zip up all the results and you have your dataset.
-
It easiest to train on datasets that are a power of 2 so I recommend scaling up to the nearest power of 2. In the case of the RPGMaker sprites, they are 120x120 which is convenient because the nearest power of 2 is 128x128 so I’ll use the command
“magick mogrify -extent 128x128 -gravity center -background white *.jpg” if I want the sprite itself to stay the same size or
“magick mogrify -resize 128x128 -extent 128x128 -gravity center -background white *.jpg”
if I want to scale up the sprite as well. -
Get rid of any images that are far too different than the rest. Stuff like props, one off sprites like bad ends and such will muddy up the AI a bit
-
It much easier to train on Google Colab but in theory you can train locally. I’m going to explain the Google Colab way. First upload your dataset to your Google Drive. If you actually use your google drive I might recommend making a new account and starting there because AI stuff will start clogging it up.
-
After that, it mostly just following the instructions from the Google Colab here https://www.reddit.com/r/MachineLearning/comments/j7nwya/p_stylegan2ada_google_colab_transfer_learning/ . You can skip the “Download Weights for Transfer Learning” and “Do some surgery when weights don’t exist for your specific resolution”. All you have to do is change the file paths for a few things and it should just work. Training can take a VERY long time. You can see the results in your Google Drive as they come in.
Creating the anime dataset is a bit more involved and transfer learning from the “This Anime Does Not Exist” requires some other methods. I might write up how to do that if there is demand for it but it requires a good bit more know how.
That should be about it. If anyone that tries runs into any issues, feel free to contact me.
Edit: Forgot a step