From bdfcf6591e58b8a5984108c4877e58a4dcf1f609 Mon Sep 17 00:00:00 2001 From: Nick Rogers Date: Fri, 6 Oct 2023 21:20:40 -0700 Subject: [PATCH 1/2] Fix missing documentation for using Llama_cpp --- docs/pages/Guides/How-to-use-different-LLM.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/pages/Guides/How-to-use-different-LLM.md b/docs/pages/Guides/How-to-use-different-LLM.md index aa5815f0..f9a2fecf 100644 --- a/docs/pages/Guides/How-to-use-different-LLM.md +++ b/docs/pages/Guides/How-to-use-different-LLM.md @@ -21,9 +21,13 @@ By default, we use OpenAI's models but if you want to change it or even run it l You don't need to provide keys if you are happy with users providing theirs, so make sure you set `LLM_NAME` and `EMBEDDINGS_NAME`. Options: -LLM_NAME (openai, manifest, cohere, Arc53/docsgpt-14b, Arc53/docsgpt-7b-falcon) +LLM_NAME (openai, manifest, cohere, Arc53/docsgpt-14b, Arc53/docsgpt-7b-falcon, llama.cpp) EMBEDDINGS_NAME (openai_text-embedding-ada-002, huggingface_sentence-transformers/all-mpnet-base-v2, huggingface_hkunlp/instructor-large, cohere_medium) +If using Llama, set the `EMBEDDINGS_NAME` to `huggingface_sentence-transformers/all-mpnet-base-v2`. + +Alternatively, if you wish to run Llama locally, you can run `setup.sh` and choose option 1 when prompted. + That's it! ### Hosting everything locally and privately (for using our optimised open-source models) From 7bf67869b015d65b894f7ea3f6bee269f70420c6 Mon Sep 17 00:00:00 2001 From: Nick Rogers Date: Sun, 8 Oct 2023 22:03:32 -0700 Subject: [PATCH 2/2] Reference DocsGPT model in custom model steps. --- docs/pages/Guides/How-to-use-different-LLM.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/pages/Guides/How-to-use-different-LLM.md b/docs/pages/Guides/How-to-use-different-LLM.md index 413bef12..0eaf483e 100644 --- a/docs/pages/Guides/How-to-use-different-LLM.md +++ b/docs/pages/Guides/How-to-use-different-LLM.md @@ -24,9 +24,9 @@ Options: LLM_NAME (openai, manifest, cohere, Arc53/docsgpt-14b, Arc53/docsgpt-7b-falcon, llama.cpp) EMBEDDINGS_NAME (openai_text-embedding-ada-002, huggingface_sentence-transformers/all-mpnet-base-v2, huggingface_hkunlp/instructor-large, cohere_medium) -If using Llama, set the `EMBEDDINGS_NAME` to `huggingface_sentence-transformers/all-mpnet-base-v2`. +If using Llama, set the `EMBEDDINGS_NAME` to `huggingface_sentence-transformers/all-mpnet-base-v2` and be sure to download [this model](https://d3dg1063dc54p9.cloudfront.net/models/docsgpt-7b-f16.gguf) into the `models/` folder: `https://d3dg1063dc54p9.cloudfront.net/models/docsgpt-7b-f16.gguf`. -Alternatively, if you wish to run Llama locally, you can run `setup.sh` and choose option 1 when prompted. +Alternatively, if you wish to run Llama locally, you can run `setup.sh` and choose option 1 when prompted. You do not need to manually add the DocsGPT model mentioned above to your `models/` folder if you use `setup.sh`, as the script will manage that step for you. That's it!