From bdfcf6591e58b8a5984108c4877e58a4dcf1f609 Mon Sep 17 00:00:00 2001 From: Nick Rogers Date: Fri, 6 Oct 2023 21:20:40 -0700 Subject: [PATCH] Fix missing documentation for using Llama_cpp --- docs/pages/Guides/How-to-use-different-LLM.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/pages/Guides/How-to-use-different-LLM.md b/docs/pages/Guides/How-to-use-different-LLM.md index aa5815f..f9a2fec 100644 --- a/docs/pages/Guides/How-to-use-different-LLM.md +++ b/docs/pages/Guides/How-to-use-different-LLM.md @@ -21,9 +21,13 @@ By default, we use OpenAI's models but if you want to change it or even run it l You don't need to provide keys if you are happy with users providing theirs, so make sure you set `LLM_NAME` and `EMBEDDINGS_NAME`. Options: -LLM_NAME (openai, manifest, cohere, Arc53/docsgpt-14b, Arc53/docsgpt-7b-falcon) +LLM_NAME (openai, manifest, cohere, Arc53/docsgpt-14b, Arc53/docsgpt-7b-falcon, llama.cpp) EMBEDDINGS_NAME (openai_text-embedding-ada-002, huggingface_sentence-transformers/all-mpnet-base-v2, huggingface_hkunlp/instructor-large, cohere_medium) +If using Llama, set the `EMBEDDINGS_NAME` to `huggingface_sentence-transformers/all-mpnet-base-v2`. + +Alternatively, if you wish to run Llama locally, you can run `setup.sh` and choose option 1 when prompted. + That's it! ### Hosting everything locally and privately (for using our optimised open-source models)