Installing Frigate OCI images on Incus using TensorRT

Now that Incus has OCI support it’s nice to pull in images that could previously only work with docker. Unfortunately as there’s no equivalent to docker-compose (yet) you have to do it manually.

In addition, although incus has the nvidia.driver setting to map nvidia files into the container, it is restricted to only work in unprivileged containers.. and Frigate only works in privileged containers, so you have to do the mapping manually.

Firstly, add OCI repos if not already done

  incus remote add --protocol oci docker https://docker.io
  incus remote add --protocol oci ghcr https://ghcr.io

My server is ipv6 only and ghcr for some reason has no v6 address, so additionally I had to add a redirect in hosts:

  echo >>/etc/hosts  64:ff9b::20.26.156.211   ghcr.io

Now create the image - modify for your network/cluster confiuration, this is basically the equivalent of docker-compose

  incus create ghcr:blakeblackshear/frigate:stable-tensorrt frigate -c security.privileged=true --network=br0 --target=goliath
  incus config set frigate boot.autostart=true
  incus config set frigate boot.autorestart=true
  incus config set frigate limits.memory=2048MB
  sudo mkdir -p /opt/frigate/config
  sudo mkdir -p /opt/frigate/media
  incus config device add frigate config disk source=/opt/frigate/config path=/config shift=true
  incus config device add frigate media disk source=/opt/frigate/media path=/media/frigate shift=true
  incus config device add frigate nvidia gpu

(Note here.. be generous with memory limit, or leave it out altogether.. cuda and especially tensorrt tend to spike memory usage during initialisation and you’ll find errors on cude/ffmpeg initialisation appearing with seeming no cause if it runs out).

You can’t use the nvidia.runtime option because it requires an unprivileged container, which does not work, so map the runtime directly.

  incus config device add frigate nvidia_1 disk source=/usr/lib/x86_64-linux-gnu/libcuda.so.1 path=/usr/lib/x86_64-linux-gnu/libcuda.so.1
  incus config device add frigate nvidia_2 disk source=/usr/lib/x86_64-linux-gnu/libcuda.so.1 path=/usr/lib/x86_64-linux-gnu/libcuda.so

  for i in /usr/lib/x86_64-linux-gnu/libnv*; do
    incus config device add frigate nvidia_$(basename $i) disk source=$i path=$i
  done

  for i in /usr/lib/x86_64-linux-gnu/libGL*; do
    incus config device add frigate nvidia_$(basename $i) disk source=$i path=$i
  done

  for i in /usr/bin/nvidia*; do
    incus config device add frigate nvidia_$(basename $i) disk source=$i path=$i
  done

Then start frigate with a valid nvidia/tensorrt config (left as an exercise to the reader) and enjoy.

It’s recommended you start simple with just video acceleration (no detection) then add tensorrt later, so you know the card is functional first.