Immich and Unraid
Immich and Unraid
I have a large amount of photos and videos from my past that I’ve been lucky enough to never lose, even before I had good backup solutions in place. However they are distributed through many nested folders and there’s duplicates everywhere. Its a complicated beast that I’ve never wanted to touch. Not to mention the nostalgia trip of looking at photos from times past that stops any thought of productivity in its tracks.
But then there’s Immich - https://immich.app/. Immich promises to bring Google photos to your home server, with ML included for face and object recognition as well as de-duplication. The project looks amazing and I’ve always wanted to try it. But the idea of getting GPU pass through to work with it and the fact it has multiple containers to setup meant that I have been putting it off. To me it screams docker compose managed through a GIT repo, that would be the way to keep these containers all at the right versions and limit problems. But I had never got GPU pass-through to work with Compose on Unraid and it felt like a mountain to tall to climb. Even googling the problem felt like there was no guides available. However I’ve finally spent a few days figuring it out and now have a working compose file that uses the GPU for all the machine learning tasks. I will outline the steps required and give you my compose example below.
Firstly Immich only officially supports the docker compose method for getting the app working on Unraid and they have a (mostly) helpful guide here at https://immich.app/docs/install/unraid/. However there are two problems with this guide, it misses setting a variable for the PostGres container volume and also there is no mention of GPU pass-through.
First steps and .env
You can find my current .env and doocker comppose yaml here: https://github.com/abl030/DotFiles/tree/master/Compose
Follow the official guide, pulling down the current compose and environment files, commenting out the two start_interval and start_period variables as per. When it comes to editing the .env file there are two things to remember.
Firstly the upload location is the actual place your photos will be stored, the way Immich works is that you wil upload photos to it either through the CLI or web interface and it will copy all those files into it’s own directory structure. This is your “upload location”.
Secondly don’t forget to edit in your DB location. Think of this as the ‘appdata’ for your PostGres container. This is the actual meat of your Immich instance. All your configs and metadata and everything will be in this folder, the other containers do not have any mounted volumes.
My config below for posterity and yes, sure, I should update the PostGres password…… :
# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables
# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/user/data/Life/Photos
# The location where your database files are stored
DB_DATA_LOCATION=/mnt/user/appdata/immichPG
# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=Australia/Perth
# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release
# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=postgres
# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich
GPU Passthrough for ML and Transcode
Now is the part that took me a good few hours to do which is passing through the GPU for machine learning and video transcoding.
Firstly lets get transcoding working, this was quite easy in comparison. If you’ve ever got an NVIDIA gpu to work with Plex on Unraid you would know to give the container the correct runtime and 2 environment variables, namely the GPU ID and it’s capabilities. Below is my compose file I came up with, this works fine. Just note the removal of the extends portion and also the addition of the runtime and environment tags. If you copy this verbatim note I have limited the container to 6 CPU cores, change this to suit your environment. More information from Immich is here https://immich.app/docs/features/hardware-transcoding.
services:
immich-server:
container_name: immich_server
image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
# extends:
# file: hwaccel.transcoding.yml
# service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
runtime: nvidia
environment:
- NVIDIA_VISIBLE_DEVICES=GPU-85eff454-7a75-fa3c-7732-b385fd62723f
- NVIDIA_DRIVER_CAPABILITIES=all
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
- /etc/localtime:/etc/localtime:ro
env_file:
- .env
ports:
- 2283:3001
cpuset: "0-5"
depends_on:
- redis
- database
restart: always
Secondly lets work on GPU passthrough for the ML container; again because we are on unraid we need to inline our Immich machine learning yaml file into the docker compose file. More info here https://immich.app/docs/features/ml-hardware-acceleration. We adjust the image tag to -cuda while also passing through the runtime and environment flags and add in the deploy yaml example from the Immich docs site. You actually do not need to set the gpu capabilities flag, in fact this will break the container and it will revert to CPU inferencing. I assume its to do with how Immich use the deploy section but I don’t know enough about docker compose. Note I’ve also limited the CPU cores here too, if you get something wrong and it starts inferencing on CPU it may make your server unresponsive if lef to consume everything.
Lastly I changed the cache volume to a bind mount instead of a docker volume, this is just to do with how Unraid works, the docker volumes are actually mounted inside the docker image and are generally not very accessible. This lets us use a cache drive or something as model storage that we can access externally and add in models at will.
immich-machine-learning:
container_name: immich_machine_learning
# For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
# Example tag: ${IMMICH_VERSION:-release}-cuda
image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
runtime: nvidia
environment:
- NVIDIA_VISIBLE_DEVICES=GPU-85eff454-7a75-fa3c-7732-b385fd62723f
# - NVIDIA_DRIVER_CAPABILITIES=all
cpuset: "0-5"
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities:
- gpu
volumes:
- /mnt/user/AI/immich:/cache
env_file:
- .env
restart: always
And now it works?
If we’ve done everything right then the docker compose stack should start in the Unraid gui, check the logs of the server and ML container to see if there are any problems. Run NVTOP on the IUnraid host while importing some content and you’ll get a good idea if the passthrough is working.
I also edited the ML models to the largest one I could get to work: immich-app/ViT-H-14-378-quickgelu__dfn5b
The end result is glorious, the larger model maxes out my GPU when inferencing but only took a few hours to go through 30k objects. Video transcoding works perfectly and due to the size of the videos we are using they simply blink through the GPU on nvtop (GTX 1080). CPU load is minimal even when importing huge amounts of files and the server remains responsive.
The image search with the larger model is also amazing, I would highly recommend people give this a go, it’s just like Google photos! Hats off to the developers for a wonderful piece of software.