This package contains a set of Python bindings that runs the `llmodel` C-API.
This package contains a set of Python bindings around the `llmodel` C-API.
# Local Installation Instructions
TODO: Right now instructions in main README still depend on Qt6 setup. To setup Python bindings, we just need `llmodel` to be built which is much simpler. However, in the future, the below installation instructions should be sequentially organized such that we expect the main README's instructions were followed first.
## Local Build Instructions
1. Setup `llmodel`
@ -23,7 +21,6 @@ Confirm that `libllmodel.*` exists in `gpt4all-backend/llmodel/build`.
In this package, we introduce Python bindings built around GPT4All's C/C++ ecosystem.
In this package, we introduce Python bindings built around GPT4All's C/C++ model backends.
## Quickstart
@ -11,7 +11,7 @@ pip install gpt4all
In Python, run the following commands to retrieve a GPT4All model and generate a response
to a prompt.
**Download Note*:*
**Download Note*:**
By default, models are stored in `~/.cache/gpt4all/` (you can change this with `model_path`). If the file already exists, model download will be skipped.
GPT4All models are designed to run locally on your own CPU. Large prompts may require longer computation time and
result in worse performance. Giving an instruction to the model will typically produce the best results.
There are two methods to interface with the underlying language model, `chat_completion()` and `generate()`. Chat completion formats a user-provided message dictionary into a prompt template (see API documentation for more details and options). This will usually produce much better results and is the approach we recommend. You may also prompt the model with `generate()` which will just pass the raw input string to the model.