mirrors/tabby
1
0
Fork 0
mirror of https://github.com/TabbyML/tabby synced 2024-11-22 17:41:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: Update MODEL_SPEC.md

Browse Source
This commit is contained in:
Meng Zhang 2023-11-01 09:37:38 -07:00 committed by GitHub
parent 36ffeb63f1
commit 90e446bfba
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
MODEL_SPEC.md
View File

@ -7,7 +7,6 @@ The minimal Tabby model directory should include the following contents:
```
ggml/
tabby.json
tokenizer.json
```
### tabby.json
@ -27,10 +26,8 @@ One example for the **prompt_template** is `<PRE>{prefix}<SUF>{suffix}<MID>`. In
The **chat_template** field is optional. When it is present, it is assumed that the model supports an instruct/chat-style interaction, and can be passed to `--chat-model`.
### tokenizer.json
This is the standard fast tokenizer file created using [Hugging Face Tokenizers](https://github.com/huggingface/tokenizers). Most Hugging Face models already come with it in repository.
### ggml/
This directory contains binary files used by the [llama.cpp](https://github.com/ggerganov/llama.cpp) inference engine. Tabby utilizes ggml for inference on `cpu`, `cuda` and `metal` devices.
Currently, only `q8_0.v2.gguf` in this directory is in use. You can refer to the instructions in llama.cpp to learn how to acquire it.