Archives
/
gpt4all
mirror of https://github.com/nomic-ai/gpt4all
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Move FAQ entries to general FAQ and adjust, plus minor improvements

Browse Source
light-mode-fix
Cosmic Snow 11 months ago committed by cosmic-snow
parent e56f977b67
commit 55f96aacc6
2 changed files with 51 additions and 31 deletions
Show Stats Download Patch File Download Diff File

54
gpt4all-bindings/python/docs/gpt4all_faq.md
Unescape Escape View File

@ -9,6 +9,7 @@ Currently, there are five different model architectures that are supported:
3. MPT - Based off of Mosaic ML's MPT architecture with examples found [here](https://huggingface.co/mosaicml/mpt-7b)
4. Replit - Based off of Replit Inc.'s Replit architecture with examples found [here](https://huggingface.co/replit/replit-code-v1-3b)
5. Falcon - Based off of TII's Falcon architecture with examples found [here](https://huggingface.co/tiiuae/falcon-40b)
6. StarCoder - Based off of BigCode's StarCoder architecture with examples found [here](https://huggingface.co/bigcode/starcoder)
## Why so many different architectures? What differentiates them?
@ -16,7 +17,7 @@ One of the major differences is license. Currently, the LLAMA based models are s
## How does GPT4All make these models available for CPU inference?
By leveraging the ggml library written by Georgi Gerganov and a growing community of developers. There are currently multiple different versions of this library. The original github repo can be found [here](https://github.com/ggerganov/ggml), but the developer of the library has also created a LLAMA based version [here](https://github.com/ggerganov/llama.cpp). Currently, this backend is using the latter as a submodule.
By leveraging the ggml library written by Georgi Gerganov and a growing community of developers. There are currently multiple different versions of this library. The original GitHub repo can be found [here](https://github.com/ggerganov/ggml), but the developer of the library has also created a LLAMA based version [here](https://github.com/ggerganov/llama.cpp). Currently, this backend is using the latter as a submodule.
## Does that mean GPT4All is compatible with all llama.cpp models and vice versa?
@ -35,8 +36,55 @@ Your CPU needs to support [AVX or AVX2 instructions](https://en.wikipedia.org/wi
In newer versions of llama.cpp, there has been some added support for NVIDIA GPU's for inference. We're investigating how to incorporate this into our downloadable installers.
## Ok, so bottom line... how do I make my model on huggingface compatible with GPT4All ecosystem right now?
## Ok, so bottom line... how do I make my model on Hugging Face compatible with GPT4All ecosystem right now?
1. Check to make sure the huggingface model is available in one of our three supported architectures
1. Check to make sure the Hugging Face model is available in one of our three supported architectures
2. If it is, then you can use the conversion script inside of our pinned llama.cpp submodule for GPTJ and LLAMA based models
3. Or if your model is an MPT model you can use the conversion script located directly in this backend directory under the scripts subdirectory
## Language Bindings
#### There's a problem with the download
Some bindings can download a model, if allowed to do so. For example, in Python or TypeScript if `allow_download=True`
or `allowDownload=true` (default), a model is automatically downloaded into `.cache/gpt4all/` in the user's home folder,
unless it already exists.
In case of connection issues or errors during the download, you might want to manually verify the model file's MD5
checksum by comparing it with the one listed in [models.json].
As an alternative to the basic downloader built into the bindings, you can choose to download from the
<https://gpt4all.io/> website instead. Scroll down to 'Model Explorer' and pick your preferred model.
[models.json]: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/metadata/models.json
#### I need the chat GUI and bindings to behave the same
The chat GUI and bindings are based on the same backend. You can make them behave the same way by following these steps:
- First of all, ensure that all parameters in the chat GUI settings match those passed to the generating API, e.g.:
=== "Python"
``` py
from gpt4all import GPT4All
model = GPT4All(...)
model.generate("prompt text", temp=0, ...) # adjust parameters
```
=== "TypeScript"
``` ts
import { createCompletion, loadModel } from '../src/gpt4all.js'
const ll = await loadModel(...);
const messages = ...
const re = await createCompletion(ll, messages, { temp: 0, ... }); // adjust parameters
```
- To make comparing the output easier, set _Temperature_ in both to 0 for now. This will make the output deterministic.
- Next you'll have to compare the templates, adjusting them as necessary, based on how you're using the bindings.
- Specifically, in Python:
- With simple `generate()` calls, the input has to be surrounded with system and prompt templates.
- When using a chat session, it depends on whether the bindings are allowed to download [models.json]. If yes,
and in the chat GUI the default templates are used, it'll be handled automatically. If no, use
`chat_session()` template parameters to customize them.
- Once you're done, remember to reset _Temperature_ to its previous value in both chat GUI and your custom code.

28
gpt4all-bindings/python/docs/gpt4all_python.md
Unescape Escape View File

@ -433,33 +433,5 @@ If you know exactly when a model should stop responding, you can add a custom ca
```
### FAQ
#### There's a problem with the download
If `allow_download=True` (default), a model is automatically downloaded into `.cache/gpt4all/` in the user's home
folder, unless it already exists.
In case of connection issues or errors during the download, you might want to manually verify the model file's MD5
checksum by comparing it with the one listed in [models.json].
As an alternative to the basic downloader built into the bindings, you can choose to download from the
<https://gpt4all.io/> website instead. Scroll down to 'Model Explorer' and pick your preferred model.
#### I need the chat GUI and bindings to behave the same
The chat GUI and bindings are based on the same backend. You can make them behave the same way by following these steps:
- First of all, ensure that all parameters in the chat GUI settings match those passed to `generate()`.
- To make comparing the output easier, set _Temperature_ in both to 0 for now. This will make the output deterministic.
- Next you'll have to compare the templates, adjusting them as necessary, based on how you're using the bindings:
- With simple `generate()` calls, the input has to be surrounded with system and prompt templates.
- When using a chat session, it depends on whether the bindings are allowed to download [models.json]. If yes, and
in the chat GUI the default templates are used, it'll be handled automatically. If no, use `chat_session()`
template parameters to customize them.
- Once you're done, remember to reset _Temperature_ to its previous value in both chat GUI and your Python code.
## API Documentation
::: gpt4all.gpt4all.GPT4All

Write Preview
Loading…
Cancel
Save