mirror of
https://github.com/hwchase17/langchain
synced 2024-11-06 03:20:49 +00:00
246710def9
One of our users noticed a bug when calling streaming models. This is
because those models return an iterator. So, I've updated the Replicate
`_call` code to join together the output. The other advantage of this
fix is that if you requested multiple outputs you would get them all –
previously I was just returning output[0].
I also adjusted the demo docs to use dolly, because we're featuring that
model right now and it's always hot, so people won't have to wait for
the model to boot up.
The error that this fixes:
```
> llm = Replicate(model=“replicate/flan-t5-xl:eec2f71c986dfa3b7a5d842d22e1130550f015720966bec48beaae059b19ef4c”)
> llm(“hello”)
> Traceback (most recent call last):
File "/Users/charlieholtz/workspace/dev/python/main.py", line 15, in <module>
print(llm(prompt))
File "/opt/homebrew/lib/python3.10/site-packages/langchain/llms/base.py", line 246, in __call__
return self.generate([prompt], stop=stop).generations[0][0].text
File "/opt/homebrew/lib/python3.10/site-packages/langchain/llms/base.py", line 140, in generate
raise e
File "/opt/homebrew/lib/python3.10/site-packages/langchain/llms/base.py", line 137, in generate
output = self._generate(prompts, stop=stop)
File "/opt/homebrew/lib/python3.10/site-packages/langchain/llms/base.py", line 324, in _generate
text = self._call(prompt, stop=stop)
File "/opt/homebrew/lib/python3.10/site-packages/langchain/llms/replicate.py", line 108, in _call
return outputs[0]
TypeError: 'generator' object is not subscriptable
```
47 lines
1.9 KiB
Markdown
47 lines
1.9 KiB
Markdown
# Replicate
|
|
This page covers how to run models on Replicate within LangChain.
|
|
|
|
## Installation and Setup
|
|
- Create a [Replicate](https://replicate.com) account. Get your API key and set it as an environment variable (`REPLICATE_API_TOKEN`)
|
|
- Install the [Replicate python client](https://github.com/replicate/replicate-python) with `pip install replicate`
|
|
|
|
## Calling a model
|
|
|
|
Find a model on the [Replicate explore page](https://replicate.com/explore), and then paste in the model name and version in this format: `owner-name/model-name:version`
|
|
|
|
For example, for this [dolly model](https://replicate.com/replicate/dolly-v2-12b), click on the API tab. The model name/version would be: `"replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5"`
|
|
|
|
Only the `model` param is required, but any other model parameters can also be passed in with the format `input={model_param: value, ...}`
|
|
|
|
|
|
For example, if we were running stable diffusion and wanted to change the image dimensions:
|
|
|
|
```
|
|
Replicate(model="stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf", input={'image_dimensions': '512x512'})
|
|
```
|
|
|
|
*Note that only the first output of a model will be returned.*
|
|
From here, we can initialize our model:
|
|
|
|
```python
|
|
llm = Replicate(model="replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5")
|
|
```
|
|
|
|
And run it:
|
|
|
|
```python
|
|
prompt = """
|
|
Answer the following yes/no question by reasoning step by step.
|
|
Can a dog drive a car?
|
|
"""
|
|
llm(prompt)
|
|
```
|
|
|
|
We can call any Replicate model (not just LLMs) using this syntax. For example, we can call [Stable Diffusion](https://replicate.com/stability-ai/stable-diffusion):
|
|
|
|
```python
|
|
text2image = Replicate(model="stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf", input={'image_dimensions':'512x512'})
|
|
|
|
image_output = text2image("A cat riding a motorcycle by Picasso")
|
|
```
|