How to set up DALLE-mini on your own hardware

By arka

10 June 2022

The replication of the DALLE known as DALLE-mini has grown exponentially in popularity. As a result, the servers at Hugging Face have struggled to keep up. So here I will tell you you how to run this on your own computer. The tutorial assumes you have the following:

• An Nvidia GPU with atleast 4GB of Vram
• A graphics card with at LEAST 4GB of Vram. 4GB is enough to run the mini model. To run the mega model you would need atleast 12 Gigabyes of VRAM
• The latest Nvidia video card drivers, cuda, and cuDNN installed
• Preferrably Linux, but it is possible to adapt this tutorial for Windows
• Python >= 3.8 installed and pip

Install tensorflow for cuda with the command:
pip install tensorflow-cuda

For Jax, do the following:

If you are using Cuda >= 11.4 and cudnn >= 8.2
pip install "jax[cuda11_cudnn82]" -f https://storage.googleapis.com/jax-releases/jax_releases.html

If you are using Cuda >= 11.1 and cudnn >= 8.0.5
pip install "jax[cuda11_cudnn805]" -f https://storage.googleapis.com/jax-releases/jax_releases.html

You can check cuda version with nvidia-smi

Check jax is installed by running python3 and typing import jax in the interpreter. If this fails, check to see what package is causing an issue in the stack trace. It may be numpy or scipy. If any package is causing problems, uninstall it with pip and then reinstall it using:
pip uninstall [PACKAGE]
pip install [PACKAGE]

Install dalle-mini using pip, as well as vqgan-jax.
pip install dalle-mini
pip install git+https://github.com/patil-suraj/vqgan-jax.git

Create a directory to work in and and enter it. Create a directory for the model and enter it.
mkdir dalle
cd dalle
mkdir mini-1
cd mini-1

Download the model. You can view models here, but to make it easy, I put the I put the models in a tar file to download.
wget https://arka.li/archives/mini-1.tar.xz

If you want the mega-model which requires a GPU with atleast 12GB Vram, you can get it from that site.

Extract the model: tar -xvf mini-1.tar.xz

Once the model is extracted, go up a dir and download this script (thanks /g/):
cd ..
curl https://pastebin.com/raw/zcZrztjF > dalle.py

Edit the script in your favorite text editor, and go to line 17 where it says DALLE_MODEL, and change:
DALLE_MODEL = "dalle-mini/dalle-mini/mini-1:v0"
to
DALLE_MODEL = "mini-1"

If you have gotten this far, you are now finished (assuming theres no fuck ups with python libraries).

Now you can run the model. It is easy:
python dalle.py [prompt]

If you are using the mini model, it will not be as good as what you will find on the other demo on huggingface, you will get similar results to what I had in the banner.

For those curious, going from left to right, starting from the top right corner, the prompts in the header were: "Lain Iwakura", "Recep Tayyip Erdogan", "Makise Kurisu", "Tsar Nicholas II", "Islam Karimov", and "Troll face".

If you have any questions, feel free to email me arkadag [AT] studiogherila.com!

Jupyter Notebook for the project
DALLE-mini Github repo
Models and where you can get the mega model
Original Demo
DALLE Playground