mirrors/tabby
1
0
Fork 0
mirror of https://github.com/TabbyML/tabby synced 2024-11-22 17:41:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

Improve documentations. (#20)

Browse Source 
* Improve help message of model preload

* Update development/scripts/triton.sh

* Improve documents

* Update deployment.md

* Update deployment.md
This commit is contained in:
Meng Zhang 2023-03-27 11:46:18 +08:00 committed by GitHub
parent 9d92821cf5
commit d5d58fbbec
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 34 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -10,4 +10,4 @@ An opensource / on-prem alternative to GitHub Copilot
## Deployment
See [deployment](./deployment/README.md)
See [deployment](./docs/deployment.md)

13
deployment/README.md
View File

@ -1,13 +0,0 @@
## Prerequisites
You need install following dependencies
* docker `>= 17.06`
* An NVIDIA GPU with enough VRAM to run the model you want.
* [NVIDIA Docker Driver](https://docs.nvidia.com/datacenter/tesla/tesla-installation-notes/index.html)
## Setup
`docker-compose up`
Open Admin Panel [http://localhost:8501](http://localhost:8501)
![image](https://user-images.githubusercontent.com/388154/227792390-ec19e9b9-ebbb-4a94-99ca-8a142ffb5e46.png)

4
development/scripts/triton.sh
View File

@ -1,6 +1,9 @@
#!/bin/bash
set -e
if [ -d "$MODEL_NAME" ]; then
MODEL_DIR="$MODEL_NAME"
else
# Get model dir.
MODEL_DIR=$(python3 <<EOF
from huggingface_hub import snapshot_download
@ -8,6 +11,7 @@ from huggingface_hub import snapshot_download
print(snapshot_download(repo_id='$MODEL_NAME', allow_patterns='triton/**/*', local_files_only=True))
EOF
)
fi
# Set model dir in triton config.
sed -i 's@${MODEL_DIR}@'$MODEL_DIR'@g' $MODEL_DIR/triton/fastertransformer/config.pbtxt

23
docs/deployment.md Normal file
View File

@ -0,0 +1,23 @@
## Prerequisites
You need install following dependencies
* docker `>= 17.06`
* An NVIDIA GPU with enough VRAM to run the model you want.
* [NVIDIA Docker Driver](https://docs.nvidia.com/datacenter/tesla/tesla-installation-notes/index.html)
## Setup Tabby Server with `docker-compose`.
1. Goto [`deployment`](../deployment) directory
2. Execute `docker-compose up`.
## Tabby Client
There're several ways to talk to the Tabby Server.
### Tabby Admin Panel [http://localhost:8501](http://localhost:8501)
![image](https://user-images.githubusercontent.com/388154/227792390-ec19e9b9-ebbb-4a94-99ca-8a142ffb5e46.png)
### OpenAPI [http://localhost:5000](http://localhost:5000)
![image](https://user-images.githubusercontent.com/388154/227835790-29e21eb5-6e9c-45ab-aa0f-c4c7ce399ad7.png)

7
tabby/tools/model_preload.py
View File

@ -9,7 +9,12 @@ class Arguments:
repo_id: str = field(
metadata={"help": "Huggingface model repository id, e.g TabbyML/NeoX-160M"}
)
prefer_local_files: bool = True
prefer_local_files: bool = field(
metadata={
"help": "Whether prefer loading local files (skip remote version check if local files are valid)."
},
default=True,
)
def parse_args():