ConfigGenerator

Ollama Docker Generator

Generate Docker Compose setups for local LLM inference using Ollama with volume persistence.

The primary model to pull on startup (e.g., llama3.1, mistral, qwen2.5).

Default is 11434.

- Pass GPU devices into the Docker container.
- Save downloaded models to a volume so they aren't lost on restart.

Where to map the volume on the host.

- Add an OpenAI-compatible web interface to the compose stack.

Comma-separated domains or '*' to allow web apps to hit the API.

How long to keep models in VRAM (e.g., 5m, 1h, -1 for forever).

e.g., '4.0' to restrict to 4 CPU cores.

e.g., '16g' to restrict to 16GB RAM.

ollama-docker-generator.yaml
1

Overview

Enterprise Ollama Config Generator

The Ollama Docker Generator creates optimized docker-compose.yml files for running large language models locally. It handles complex Docker GPU passthrough settings, port bindings, and persistent volumes automatically.

Short Answer: Use this generator to quickly spin up Llama 3, Mistral, or Qwen models on your own hardware without exposing your API to the internet.

How It Works

  1. Select Hardware: Choose whether to enable NVIDIA, AMD, or Apple Silicon GPU acceleration.
  2. Configure Networking: Bind to 127.0.0.1 for local-only access or 0.0.0.0 for LAN access.
  3. Add UI: Toggle the inclusion of Open WebUI for a ChatGPT-like interface.
  4. Generate: Get the docker-compose file and a setup script to automatically pull your desired model.

Best Practices

  • Always use persistent volumes so you don't have to re-download multi-gigabyte models on container restart.
  • Bind to 127.0.0.1 (localhost) unless you specifically need other devices on your network to access the API.
  • Use resource limits (CPU/Mem) if you are running other critical services on the same host.
  • Combine Ollama with Open WebUI for a fully featured local AI workspace.

Common Mistakes

  • Forgetting to install the NVIDIA Container Toolkit on Linux before attempting GPU passthrough.
  • Binding to 0.0.0.0 on a public cloud server without a reverse proxy (exposes your compute to attackers).
  • Not setting OLLAMA_KEEP_ALIVE, causing models to unload from VRAM too quickly during chat sessions.

Security Recommendations

  • Ollama does NOT have built-in authentication. If you bind to 0.0.0.0, you MUST put it behind Nginx or Traefik with Basic Auth or OAuth.
  • Do not run the container as privileged unless strictly necessary.
  • Regularly update the Ollama image to patch vulnerabilities in its internal dependencies.

Frequently Asked Questions

Why is my Ollama container using CPU instead of my GPU?
You likely haven't passed the GPU into Docker correctly. For NVIDIA on Linux, you must install the nvidia-container-toolkit and add the `deploy.resources.reservations.devices` block to your compose file. This generator handles the compose file part automatically if you select NVIDIA.
How do I download a model into the container?
The generator outputs a `model-pull.sh` script. Run it after starting the container. It uses `docker exec` to trigger the download inside the running instance.