oneuptime/docker-compose.llm.yml

services:

  # Redis for caching and cluster state
  redis:
    extends:
        file: ./docker-compose.base.yml
        service: redis

  llm: 
    extends:
        file: ./docker-compose.base.yml
        service: llm
    ports:
        - '8547:8547'
    build:
        network: host
        context: ./LLM
        dockerfile: ./Dockerfile
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

networks:
  oneuptime:
    driver: bridge
refactor: Update Llama app to use local model path instead of model ID This commit updates the Llama app to use a local model path instead of a model ID. The model path is set to "/app/Models/Meta-Llama-3-8B-Instruct". This change improves the reliability and performance of the app by directly referencing the model file instead of relying on an external model ID. 2024-06-18 20:41:29 +00:00			`services:`
chore: Add Redis service to docker-compose.llm.yml The Redis service has been added to the docker-compose.llm.yml file. This service is responsible for caching and cluster state. It extends the configuration from the docker-compose.base.yml file and uses the "redis" service defined there. 2024-06-28 16:42:43 +00:00
			`# Redis for caching and cluster state`
			`redis:`
			`extends:`
			`file: ./docker-compose.base.yml`
			`service: redis`

chore: Update Llama Models directory name to LLM The directory name for Llama Models has been updated from "Llama/Models" to "LLM/Models". This change ensures consistency and aligns with the updated file paths in the codebase. 2024-06-28 10:32:44 +00:00			`llm:`
refactor: Update Llama app to use local model path instead of model ID This commit updates the Llama app to use a local model path instead of a model ID. The model path is set to "/app/Models/Meta-Llama-3-8B-Instruct". This change improves the reliability and performance of the app by directly referencing the model file instead of relying on an external model ID. 2024-06-18 20:41:29 +00:00			`extends:`
			`file: ./docker-compose.base.yml`
chore: Update Llama Models directory name to LLM The directory name for Llama Models has been updated from "Llama/Models" to "LLM/Models". This change ensures consistency and aligns with the updated file paths in the codebase. 2024-06-28 10:32:44 +00:00			`service: llm`
refactor: Update Llama app to use local model path instead of model ID This commit updates the Llama app to use a local model path instead of a model ID. The model path is set to "/app/Models/Meta-Llama-3-8B-Instruct". This change improves the reliability and performance of the app by directly referencing the model file instead of relying on an external model ID. 2024-06-18 20:41:29 +00:00			`ports:`
			`- '8547:8547'`
			`build:`
			`network: host`
chore: Update Dockerfile and docker-compose.llm.yml for LLM models This commit updates the Dockerfile and docker-compose.llm.yml files to include the LLM Models directory. The Dockerfile now uses the `ADD .` command to copy the entire repository into the container at `/app`, and the docker-compose.llm.yml file has been modified to set the context to the `./LLM` directory. These changes ensure that the LLM Models directory is included in the Docker image and that the correct context is used for building the image. 2024-06-28 17:59:43 +00:00			`context: ./LLM`
			`dockerfile: ./Dockerfile`
refactor: Add GPU support to Llama app in docker-compose.ai.yml This commit adds GPU support to the Llama app in the docker-compose.ai.yml file. It includes a new deploy section with reservations for GPU devices, specifying the driver, count, and capabilities. This change enables the Llama app to utilize GPU resources for improved performance in AI processing. 2024-06-18 21:11:53 +00:00			`deploy:`
			`resources:`
			`reservations:`
			`devices:`
			`- driver: nvidia`
feat: Update count property in docker-compose.llm.yml to use 'all' This commit updates the count property in the docker-compose.llm.yml file to use the value 'all' instead of the previous value of '1'. This change allows for the allocation of all available devices with the 'gpu' capability, improving the utilization of resources. 2024-07-03 16:48:43 +00:00			`count: all`
refactor: Update Dockerfile.tpl to expose port 8547 instead of port 80 2024-06-19 12:18:25 +00:00			`capabilities: [gpu]`

			`networks:`
			`oneuptime:`
			`driver: bridge`