mirrors/tabby
1
0
Fork 0
mirror of https://github.com/TabbyML/tabby synced 2024-11-22 08:21:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore: remove multi-node integration docs as we're moving to a http based implementation

Browse Source
This commit is contained in:
Meng Zhang 2024-05-30 21:26:44 +08:00
parent d42fb0571e
commit c4abeff74e
5 changed files with 2 additions and 81 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
website/blog/2024-03-26-tabby-with-replicas-behind-reverse-proxy/index.mdx vendored
View File

@ -84,5 +84,5 @@ For those interested in securing their setup, consider using Caddy directives li
And there you have it! You've successfully set up Tabby with Caddy as a reverse proxy. Happy coding with your new AI assistant!
As an additional note, since the release of v0.9.0, Tabby enterprise edition now includes the built-in ability to handle replicas and load balancing, with a integrate account management system.
For more information, refer to the [official documentation](/docs/administration/distributed/) for details.
As an additional note, since the release of v0.9.0, Tabby enterprise edition now includes the built-in account management system.
For more information, refer to the [official documentation](/) for details.

BIN
website/docs/administration/distributed/chat-playground.png (Stored with Git LFS) vendored
View File

Binary file not shown.

BIN
website/docs/administration/distributed/cluster-information.png (Stored with Git LFS) vendored
View File

Binary file not shown.

BIN
website/docs/administration/distributed/completion-worker.png (Stored with Git LFS) vendored
View File

Binary file not shown.

70
website/docs/administration/distributed/index.mdx vendored
View File

@ -1,70 +0,0 @@
---
sidebar_position: 9
---
import CompletionWorkerUrl from "./completion-worker.png";
import ChatPlaygroundUrl from "./chat-playground.png";
# Multi-node and Load Balancing
:::subscription
This feature is available in the **Team** and **Enterprise** Plans.
:::
Tabby provides built-in distributed support for multi-node setups. This allows you to scale your Tabby deployment horizontally and distribute the workload across multiple GPU workers.
## Start Tabby
Start the web UI using the following command:
```bash
tabby serve
```
By doing so, the web server will operate without a model attached to it. If you send a POST request to `/v1/completions`, you will receive a `501 Not Implemented` error.
## Check the Cluster Information
In the `Cluster Information` tab of the admin panel, you can see that there are no workers connected to the Tabby instance, except for the local code index.
![Cluster Information](./cluster-information.png)
You'll also notice the `Registration Token` displayed on this page. This token is used to authenticate the worker nodes with the Tabby instance and will be referred to as `TABBY_REGISTRATION_TOKEN` in the following sections.
## Register a Completion Worker
To register a worker, you need to run the following command:
```bash
# In this tutorial, we'll start the worker on the same machine as the web server.
export TABBY_WEBSERVER_URL=127.0.0.1:8080
export TABBY_REGISTRATION_TOKEN=<token from the admin panel>
tabby worker::completion \
--model StarCoder-1B \
--url $TABBY_WEBSERVER_URL \
--token $TABBY_REGISTRATION_TOKEN \
--port 8081
```
After this command executes successfully, you should see the new worker in the `Cluster Information` tab.
More workers can be added by running the same command on different machines to improve the concurrency of the system.
Tabby will distribute the workload across all the workers.
<img src={CompletionWorkerUrl} width={720} />
## (Optional) Register a Chat Worker
Similarly, you can register a chat worker by running the following command to enable the chat playground.
```bash
tabby worker::chat \
--model Mistral-7B \
--url $TABBY_WEBSERVER_URL \
--token $TABBY_REGISTRATION_TOKEN \
--port 8082
```
Once it's registered, you should see the `Chat Playground` entry under the avatar menu.
<img src={ChatPlaygroundUrl} width={360} />