Go to file
Wei Zhang 1632dd054e
feat(llama-cpp-server): reuse local server for completion and chat (#2812)
*  support reuse the same server if model id is equaled

Signed-off-by: Wei Zhang <kweizh@gmail.com>

* 🎨 fix review

Signed-off-by: Wei Zhang <kweizh@gmail.com>

* 🔨 should compare model in llama cpp server

Signed-off-by: Wei Zhang <kweizh@gmail.com>

* match case for creating completion and chat

Signed-off-by: Wei Zhang <kweizh@gmail.com>

* if let case for creating completion and chat

Signed-off-by: Wei Zhang <kweizh@gmail.com>

* add changelog

* rebase && make fix

---------

Signed-off-by: Wei Zhang <kweizh@gmail.com>
Co-authored-by: Meng Zhang <meng@tabbyml.com>
2024-08-10 22:51:13 -07:00
.changes feat(llama-cpp-server): reuse local server for completion and chat (#2812) 2024-08-10 22:51:13 -07:00
.github ci: remove update-db-schema from ci auto fix (#2824) 2024-08-09 18:42:14 -07:00
ci ci: fix centos mirror address (#2582) 2024-07-05 14:20:25 +09:00
clients feat(ui): support multi selection in chat panel (#2808) 2024-08-09 14:18:59 -07:00
crates feat(llama-cpp-server): reuse local server for completion and chat (#2812) 2024-08-10 22:51:13 -07:00
docker ci: fix Dockerfile.cuda: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (#2629) 2024-07-12 17:01:45 +08:00
ee feat(db): add thread / thread_messages table (#2826) 2024-08-10 21:07:16 -07:00
experimental chore: remove demo deployment with modal 2024-05-03 00:03:09 -07:00
python feat(loadtest): add loadtest tools (#906) 2023-11-28 14:34:00 +08:00
rules feat(db): add thread / thread_messages table (#2826) 2024-08-10 21:07:16 -07:00
website docs: add code completion configuration docs (#2822) 2024-08-09 14:52:23 -07:00
.changie.yaml chore: adopt changie for automatic changelog management (#2092) 2024-05-10 15:11:25 -07:00
.dockerignore feat: Add rocm builds and documentation (#1012) 2023-12-13 15:59:04 +08:00
.env refactor(db): Rewrite database code to use sqlx instead of rusqlite (#1340) 2024-02-01 09:38:37 -08:00
.gitattributes chore: exclude ee/tabby-webserver/ui to language stats 2023-11-17 15:50:27 -08:00
.gitignore fix(scheduler): properly set parallelism for code indexer (#2504) 2024-06-26 12:44:22 +08:00
.gitmodules feat(webserver, db): refactor gitlab and github integrations into unified integration / third party repository services (#2088) 2024-05-17 18:50:26 +00:00
.rustfmt.toml refactor: rust nightly format (#197) 2023-06-05 14:17:07 -07:00
Cargo.lock feat(db): add thread / thread_messages table (#2826) 2024-08-10 21:07:16 -07:00
Cargo.toml Release 0.16.0-dev.0 2024-08-07 20:12:48 -07:00
CHANGELOG.md docs: update changelog for v0.15 (#2815) 2024-08-09 14:07:35 -07:00
CODE_OF_CONDUCT.md docs: Create CODE_OF_CONDUCT.md (#1228) 2024-01-24 00:18:32 -08:00
codecov.yml chore: update codecov.yml 2024-02-08 13:15:46 -08:00
CONTRIBUTING.md docs: Update CONTRIBUTING.md 2024-06-26 20:53:03 +08:00
LICENSE Update LICENSE 2024-01-10 23:16:11 -08:00
Makefile ci: remove update-db-schema from ci auto fix (#2824) 2024-08-09 18:42:14 -07:00
MODEL_SPEC.md feat(common): Migrate model path to "model.gguf" instead of "q8_0.v2.gguf" (#1847) 2024-04-17 19:38:38 +00:00
package.json feat: migrate to pnpm (#2214) 2024-05-21 21:43:15 +00:00
pnpm-lock.yaml feat(ui): add GraphQL syntax highlighting in code browser (#2751) 2024-07-30 06:57:02 -07:00
pnpm-workspace.yaml feat: migrate to pnpm (#2214) 2024-05-21 21:43:15 +00:00
README.md docs: update apt install command to make them consistent 2024-08-08 12:06:24 -07:00
sgconfig.yml refactor(webserver): ensure only services could rely on tabby_db dire… (#1407) 2024-02-07 19:15:53 +00:00
turbo.json chore(email): fix build (#2215) 2024-05-21 15:07:43 -07:00

🐾 Tabby

📚 Docs💬 Slack Live Demo

latest release PRs Welcome Docker pulls codecov

Tabby is a self-hosted AI coding assistant, offering an open-source and on-premises alternative to GitHub Copilot. It boasts several key features:

  • Self-contained, with no need for a DBMS or cloud service.
  • OpenAPI interface, easy to integrate with existing infrastructure (e.g Cloud IDE).
  • Supports consumer-grade GPUs.

Open in Playground

Demo

🔥 What's New

  • 07/09/2024 🎉Announce Codestral integration in Tabby!
  • 07/05/2024 Tabby v0.13.0 introduces Answer Engine, a central knowledge engine for internal engineering teams. It seemlessly integrates with dev team's internal data, delivering reliable and precise answers to empower developers.
  • 06/13/2024 VSCode 1.7 marks a significant milestone with a versatile Chat experience throughout your coding exprience. Come and they the latest chat in side-panel and editing via chat command!
  • 06/10/2024 Latest 📃blogpost drop on an enhanced code context understanding in Tabby!
  • 06/06/2024 Tabby v0.12.0 release brings 🔗seamless integrations (Gitlab SSO, Self-hosted GitHub/GitLab, etc.), to ⚙️flexible configurations (HTTP API integration) and 🌐expanded capabilities (repo-context in Code Browser)!
  • 05/22/2024 Tabby VSCode 1.6 comes with multiple choices in inline completion, and the auto-generated commit messages🐱💻!
Archived
  • 05/11/2024 v0.11.0 brings significant enterprise upgrades, including 📊storage usage stats, 🔗GitHub & GitLab integration, 📋Activities page, and the long-awaited 🤖Ask Tabby feature!
  • 04/22/2024 v0.10.0 released, featuring the latest Reports tab with team-wise analytics for Tabby usage.
  • 04/19/2024 📣 Tabby now incorporates locally relevant snippets(declarations from local LSP, and recently modified code) for code completion!
  • 04/17/2024 CodeGemma and CodeQwen model series have now been added to the official registry!
  • 03/20/2024 v0.9 released, highlighting a full feature admin UI.
  • 12/23/2023 Seamlessly deploy Tabby on any cloud with SkyServe 🛫 from SkyPilot.
  • 12/15/2023 v0.7.0 released with team management and secured access!
  • 10/15/2023 RAG-based code completion is enabled by detail in v0.3.0🎉! Check out the blogpost explaining how Tabby utilizes repo-level context to get even smarter!
  • 11/27/2023 v0.6.0 released!
  • 11/09/2023 v0.5.5 released! With a redesign of UI + performance improvement.
  • 10/24/2023 Major updates for Tabby IDE plugins across VSCode/Vim/IntelliJ!
  • 10/04/2023 Check out the model directory for the latest models supported by Tabby.
  • 09/18/2023 Apple's M1/M2 Metal inference support has landed in v0.1.1!
  • 08/31/2023 Tabby's first stable release v0.0.1 🥳.
  • 08/28/2023 Experimental support for the CodeLlama 7B.
  • 08/24/2023 Tabby is now on JetBrains Marketplace!

👋 Getting Started

You can find our documentation here.

Run Tabby in 1 Minute

The easiest way to start a Tabby server is by using the following Docker command:

docker run -it \
  --gpus all -p 8080:8080 -v $HOME/.tabby:/data \
  tabbyml/tabby \
  serve --model StarCoder-1B --device cuda --chat-model Qwen2-1.5B-Instruct

For additional options (e.g inference type, parallelism), please refer to the documentation page.

🤝 Contributing

Full guide at CONTRIBUTING.md;

Get the Code

git clone --recurse-submodules https://github.com/TabbyML/tabby
cd tabby

If you have already cloned the repository, you could run the git submodule update --recursive --init command to fetch all submodules.

Build

  1. Set up the Rust environment by following this tutorial.

  2. Install the required dependencies:

# For MacOS
brew install protobuf

# For Ubuntu / Debian
apt install protobuf-compiler libopenblas-dev
  1. Install useful tools:
# For Ubuntu
apt install make sqlite3 graphviz
  1. Now, you can build Tabby by running the command cargo build.

Start Hacking!

... and don't forget to submit a Pull Request

🌍 Community

  • 🎤 Twitter / X - engage with TabbyML for all things possible
  • 📚 LinkedIn - follow for the latest from the community
  • 💌 Newsletter - subscribe to unlock Tabby insights and secrets

🔆 Activity

Git Repository Activity

🌟 Star History

Star History Chart