oneuptime/docker-compose.ai.yml

services:
  llama: 
    extends:
        file: ./docker-compose.base.yml
        service: llama
    ports:
        - '8547:8547'
    build:
        network: host
        context: .
        dockerfile: ./Llama/Dockerfile
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

networks:
  oneuptime:
    driver: bridge
refactor: Update Llama app to use local model path instead of model ID This commit updates the Llama app to use a local model path instead of a model ID. The model path is set to "/app/Models/Meta-Llama-3-8B-Instruct". This change improves the reliability and performance of the app by directly referencing the model file instead of relying on an external model ID. 2024-06-18 20:41:29 +00:00			`services:`
			`llama:`
			`extends:`
			`file: ./docker-compose.base.yml`
			`service: llama`
			`ports:`
			`- '8547:8547'`
			`build:`
			`network: host`
			`context: .`
refactor: Add GPU support to Llama app in docker-compose.ai.yml This commit adds GPU support to the Llama app in the docker-compose.ai.yml file. It includes a new deploy section with reservations for GPU devices, specifying the driver, count, and capabilities. This change enables the Llama app to utilize GPU resources for improved performance in AI processing. 2024-06-18 21:11:53 +00:00			`dockerfile: ./Llama/Dockerfile`
			`deploy:`
			`resources:`
			`reservations:`
			`devices:`
			`- driver: nvidia`
refactor: Update Dockerfile.tpl to use huggingface/transformers-pytorch-gpu image This commit updates the Dockerfile.tpl to use the huggingface/transformers-pytorch-gpu image instead of the continuumio/anaconda3 image. This change allows the Llama app to utilize GPU resources for improved performance in AI processing. Additionally, the unnecessary installation of the transformers and accelerate libraries is removed as they are already included in the huggingface/transformers-pytorch-gpu image. 2024-06-19 13:06:23 +00:00			`count: 1`
refactor: Update Dockerfile.tpl to expose port 8547 instead of port 80 2024-06-19 12:18:25 +00:00			`capabilities: [gpu]`

			`networks:`
			`oneuptime:`
			`driver: bridge`