mirror of
https://github.com/hwchase17/langchain
synced 2024-11-08 07:10:35 +00:00
e2d7677526
# Docs: compound ecosystem and integrations **Problem statement:** We have a big overlap between the References/Integrations and Ecosystem/LongChain Ecosystem pages. It confuses users. It creates a situation when new integration is added only on one of these pages, which creates even more confusion. - removed References/Integrations page (but move all its information into the individual integration pages - in the next PR). - renamed Ecosystem/LongChain Ecosystem into Integrations/Integrations. I like the Ecosystem term. It is more generic and semantically richer than the Integration term. But it mentally overloads users. The `integration` term is more concrete. UPDATE: after discussion, the Ecosystem is the term. Ecosystem/Integrations is the page (in place of Ecosystem/LongChain Ecosystem). As a result, a user gets a single place to start with the individual integration.
49 lines
1.6 KiB
Markdown
49 lines
1.6 KiB
Markdown
# GPT4All
|
|
|
|
This page covers how to use the `GPT4All` wrapper within LangChain. The tutorial is divided into two parts: installation and setup, followed by usage with an example.
|
|
|
|
## Installation and Setup
|
|
|
|
- Install the Python package with `pip install pyllamacpp`
|
|
- Download a [GPT4All model](https://github.com/nomic-ai/pyllamacpp#supported-model) and place it in your desired directory
|
|
|
|
## Usage
|
|
|
|
### GPT4All
|
|
|
|
To use the GPT4All wrapper, you need to provide the path to the pre-trained model file and the model's configuration.
|
|
|
|
```python
|
|
from langchain.llms import GPT4All
|
|
|
|
# Instantiate the model. Callbacks support token-wise streaming
|
|
model = GPT4All(model="./models/gpt4all-model.bin", n_ctx=512, n_threads=8)
|
|
|
|
# Generate text
|
|
response = model("Once upon a time, ")
|
|
```
|
|
|
|
You can also customize the generation parameters, such as n_predict, temp, top_p, top_k, and others.
|
|
|
|
To stream the model's predictions, add in a CallbackManager.
|
|
|
|
```python
|
|
from langchain.llms import GPT4All
|
|
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
|
|
|
|
# There are many CallbackHandlers supported, such as
|
|
# from langchain.callbacks.streamlit import StreamlitCallbackHandler
|
|
|
|
callbacks = [StreamingStdOutCallbackHandler()]
|
|
model = GPT4All(model="./models/gpt4all-model.bin", n_ctx=512, n_threads=8)
|
|
|
|
# Generate text. Tokens are streamed through the callback manager.
|
|
model("Once upon a time, ", callbacks=callbacks)
|
|
```
|
|
|
|
## Model File
|
|
|
|
You can find links to model file downloads in the [pyllamacpp](https://github.com/nomic-ai/pyllamacpp) repository.
|
|
|
|
For a more detailed walkthrough of this, see [this notebook](../modules/models/llms/integrations/gpt4all.ipynb)
|