mirrors/tabby
1
0
Fork 0
mirror of https://github.com/TabbyML/tabby synced 2024-11-23 10:05:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
12686f9d6c
tabby/experimental/prompt-rewrite-eval/README.md
vodkaslime 96cea7d112
feat: prompt rewrite eval tool (#406) 
* feat: prompt rewrite eval tool

* chore: readme

* chore: language

* chore: resolve comments

* chore: prefix generation

* chore: clean

* chore: dependency
2023-09-08 09:11:14 +08:00

610 B
Raw Blame History

Prompt rewriting evaluation tool

Install dependencies

pip install -r requirements.txt

Run backend rewriting script

  1. tweak eval.toml

    • tabby binary path
    • index repo url (the repo you want tabby to index from)
    • sample repo url (the repo you want to generate completion requests from)
    • language
    • prompt count

  2. run python evaluation.py

Run dashboard to view prompts

streamlit run dashboard.py
  • Tweak the slider bar to change how many recent prompts you want to review.
  • Change the language to filter only the specific language you are interested in.