NVIDIA GPU fix
Fix Odysseus Docker GPU not detected
If Odysseus runs but local inference is slow, the app may not be the problem. Docker, WSL2, the NVIDIA driver, or the container runtime may be hiding the GPU from the service that actually runs your model.
1. Confirm the host can see the GPU
On Windows, run this in PowerShell. On Linux, run it in your shell. If this fails, fix the driver before checking Docker.
nvidia-smi2. Confirm WSL2 can see the GPU
If you use Windows WSL2, open your WSL distro and run the same command. A mismatch between Windows and WSL is common after driver updates.
nvidia-smi3. Test GPU access from Docker
Use a small NVIDIA CUDA container to verify that Docker can pass the GPU through. If this fails, reinstall or reconfigure NVIDIA Container Toolkit or Docker Desktop GPU integration.
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi4. Check the compose/runtime layer
Only after the test container works should you inspect the Odysseus compose file, model service, or Ollama deployment. The component serving the model needs GPU access; the web UI alone may not need it.
Symptoms and likely cause
nvidia-smifails on host: driver problem.- Host works, WSL fails: WSL2 GPU integration problem.
- WSL works, Docker test fails: container runtime problem.
- Docker test works, model is slow: model server is probably running CPU-only.