10  Week 9 (27-Nov-2025)

---

10.1 Create a Large Language Model Cluster

10.1.1 Create the LLM Engine Container

• We are going to use the Ollama software from here.

• Create a network called llm_network

docker network create llm_network

• Create a volume called ollama-data

docker volume create ollama-data

• Create the docker container with the ollama LLM engine.

docker run -d -v ollama-data:/root/.ollama -p 11434:11434 --network llm_network --name ollama ollama/ollama

10.1.2 Create the LLM Web UI

• Create a volume called ollama-webui

docker volume create ollama-webui

• Create the docker container with the ollama web graphical user interface.

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v ollama-webui:/app/backend/data --network llm_network --name llmui --restart always ghcr.io/open-webui/open-webui:main

10.1.3 Update hosts file

• Update your hosts file to point llmui.mylab.com to your own machine IP address.

  • ex: 192.168.1.100 llmui.mylab.com

• Check that you can access llmui.mylab.com using your browser.

• Traefik should now route your traffic to the llmui container.

10.1.4 Download your first LLM model

• Connect to your ollama container with this command:

docker exec -it ollama /bin/bash

• Run the following command to download the LLM model llama3.2.

ollama pull llama3.2

10.1.5 Ask Questions with the Web Interface

• Without Traefik, you can access your web interface in http://localhost:3000.

• If you have Traefik running, access your web interface in http://llmui.mylab.com.

10.1.6 How to download new LLM models on multiple hosts ?

• What can you use ?
  • Yes, a good method is by using our “friend” …… Ansible.
• We will do this in the next class.