From c4abeff74e7346b9226241613a388b1a417b4f03 Mon Sep 17 00:00:00 2001 From: Meng Zhang Date: Thu, 30 May 2024 21:26:44 +0800 Subject: [PATCH] chore: remove multi-node integration docs as we're moving to a http based implementation --- .../index.mdx | 4 +- .../distributed/chat-playground.png | 3 - .../distributed/cluster-information.png | 3 - .../distributed/completion-worker.png | 3 - .../docs/administration/distributed/index.mdx | 70 ------------------- 5 files changed, 2 insertions(+), 81 deletions(-) delete mode 100644 website/docs/administration/distributed/chat-playground.png delete mode 100644 website/docs/administration/distributed/cluster-information.png delete mode 100644 website/docs/administration/distributed/completion-worker.png delete mode 100644 website/docs/administration/distributed/index.mdx diff --git a/website/blog/2024-03-26-tabby-with-replicas-behind-reverse-proxy/index.mdx b/website/blog/2024-03-26-tabby-with-replicas-behind-reverse-proxy/index.mdx index 47d0bffe0..8cef754c3 100644 --- a/website/blog/2024-03-26-tabby-with-replicas-behind-reverse-proxy/index.mdx +++ b/website/blog/2024-03-26-tabby-with-replicas-behind-reverse-proxy/index.mdx @@ -84,5 +84,5 @@ For those interested in securing their setup, consider using Caddy directives li And there you have it! You've successfully set up Tabby with Caddy as a reverse proxy. Happy coding with your new AI assistant! -As an additional note, since the release of v0.9.0, Tabby enterprise edition now includes the built-in ability to handle replicas and load balancing, with a integrate account management system. -For more information, refer to the [official documentation](/docs/administration/distributed/) for details. \ No newline at end of file +As an additional note, since the release of v0.9.0, Tabby enterprise edition now includes the built-in account management system. +For more information, refer to the [official documentation](/) for details. \ No newline at end of file diff --git a/website/docs/administration/distributed/chat-playground.png b/website/docs/administration/distributed/chat-playground.png deleted file mode 100644 index bd3f61dee..000000000 --- a/website/docs/administration/distributed/chat-playground.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:ff943ee0cd4571209060ed936ca34ad3d62f17910863395db50ce90907bfffe9 -size 41533 diff --git a/website/docs/administration/distributed/cluster-information.png b/website/docs/administration/distributed/cluster-information.png deleted file mode 100644 index c04aaf42f..000000000 --- a/website/docs/administration/distributed/cluster-information.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:00ea25881856af9bf88a361fff8464ee8614995ee2386d8866e44ecc8d19f67d -size 274354 diff --git a/website/docs/administration/distributed/completion-worker.png b/website/docs/administration/distributed/completion-worker.png deleted file mode 100644 index cca39f0d3..000000000 --- a/website/docs/administration/distributed/completion-worker.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:674bed96e648ed2608165c6ec751e3ce1af22229ab5560b73d2fd306121b2c4e -size 103115 diff --git a/website/docs/administration/distributed/index.mdx b/website/docs/administration/distributed/index.mdx deleted file mode 100644 index dbc15db3f..000000000 --- a/website/docs/administration/distributed/index.mdx +++ /dev/null @@ -1,70 +0,0 @@ ---- -sidebar_position: 9 ---- - -import CompletionWorkerUrl from "./completion-worker.png"; -import ChatPlaygroundUrl from "./chat-playground.png"; - -# Multi-node and Load Balancing - -:::subscription -This feature is available in the **Team** and **Enterprise** Plans. -::: - -Tabby provides built-in distributed support for multi-node setups. This allows you to scale your Tabby deployment horizontally and distribute the workload across multiple GPU workers. - -## Start Tabby - -Start the web UI using the following command: - -```bash -tabby serve -``` - -By doing so, the web server will operate without a model attached to it. If you send a POST request to `/v1/completions`, you will receive a `501 Not Implemented` error. - -## Check the Cluster Information - -In the `Cluster Information` tab of the admin panel, you can see that there are no workers connected to the Tabby instance, except for the local code index. - -![Cluster Information](./cluster-information.png) - -You'll also notice the `Registration Token` displayed on this page. This token is used to authenticate the worker nodes with the Tabby instance and will be referred to as `TABBY_REGISTRATION_TOKEN` in the following sections. - -## Register a Completion Worker - -To register a worker, you need to run the following command: - -```bash -# In this tutorial, we'll start the worker on the same machine as the web server. -export TABBY_WEBSERVER_URL=127.0.0.1:8080 -export TABBY_REGISTRATION_TOKEN= - -tabby worker::completion \ - --model StarCoder-1B \ - --url $TABBY_WEBSERVER_URL \ - --token $TABBY_REGISTRATION_TOKEN \ - --port 8081 -``` - -After this command executes successfully, you should see the new worker in the `Cluster Information` tab. -More workers can be added by running the same command on different machines to improve the concurrency of the system. -Tabby will distribute the workload across all the workers. - - - -## (Optional) Register a Chat Worker - -Similarly, you can register a chat worker by running the following command to enable the chat playground. - -```bash -tabby worker::chat \ - --model Mistral-7B \ - --url $TABBY_WEBSERVER_URL \ - --token $TABBY_REGISTRATION_TOKEN \ - --port 8082 -``` - -Once it's registered, you should see the `Chat Playground` entry under the avatar menu. - - \ No newline at end of file