As of 12 Nov 2023, ollama in nixpkgs fails to utilize CUDA-enabled devices.

This repo contains a nix flake that defines:
- ollama package with CUDA support
- NixOS module that may be helpful if you plan on using ollama as a system-wide
  LLM runner

To simply run the CUDA-enabled ollama binary, make sure you have nvidia-smi
available in the PATH and run the following command:
nix run github:havaker/ollama-nix