Various improvements to the Model I/O section of the documentation
- Changed "Chat Model" to "chat model" in a few spots for internal
consistency
- Minor spelling & grammar fixes to improve readability & comprehension
Some Chat models provide a streaming response. This means that instead of waiting for the entire response to be returned, you can start processing it as soon as it's available. This is useful if you want to display the response to the user as it's being generated, or if you want to process the response as it's being generated.
Some chat models provide a streaming response. This means that instead of waiting for the entire response to be returned, you can start processing it as soon as it's available. This is useful if you want to display the response to the user as it's being generated, or if you want to process the response as it's being generated.
import StreamingChatModel from "@snippets/modules/model_io/models/chat/how_to/streaming.mdx"
@ -8,16 +8,16 @@ LangChain provides interfaces and integrations for two types of models:
- [LLMs](/docs/modules/model_io/models/llms/): Models that take a text string as input and return a text string
- [Chat models](/docs/modules/model_io/models/chat/): Models that are backed by a language model but take a list of Chat Messages as input and return a Chat Message
## LLMs vs Chat Models
## LLMs vs chat models
LLMs and Chat Models are subtly but importantly different. LLMs in LangChain refer to pure text completion models.
LLMs and chat models are subtly but importantly different. LLMs in LangChain refer to pure text completion models.
The APIs they wrap take a string prompt as input and output a string completion. OpenAI's GPT-3 is implemented as an LLM.
Chat models are often backed by LLMs but tuned specifically for having conversations.
And, crucially, their provider APIs expose a different interface than pure text completion models. Instead of a single string,
And, crucially, their provider APIs use a different interface than pure text completion models. Instead of a single string,
they take a list of chat messages as input. Usually these messages are labeled with the speaker (usually one of "System",
"AI", and "Human"). And they return a ("AI") chat message as output. GPT-4 and Anthropic's Claude are both implemented as Chat Models.
"AI", and "Human"). And they return an AI chat message as output. GPT-4 and Anthropic's Claude are both implemented as chat models.
To make it possible to swap LLMs and Chat Models, both implement the Base Language Model interface. This exposes common
To make it possible to swap LLMs and chat models, both implement the Base Language Model interface. This includes common
methods "predict", which takes a string and returns a string, and "predict messages", which takes messages and returns a message.
If you are using a specific model it's recommended you use the methods specific to that model class (i.e., "predict" for LLMs and "predict messages" for Chat Models),
If you are using a specific model it's recommended you use the methods specific to that model class (i.e., "predict" for LLMs and "predict messages" for chat models),
but if you're creating an application that should work with different types of models the shared interface can be helpful.
"Along with HumanInputLLM, LangChain also provides a pseudo Chat Model class that can be used for testing, debugging, or educational purposes. This allows you to mock out calls to the Chat Model and simulate how a human would respond if they received the messages.\n",
"Along with HumanInputLLM, LangChain also provides a pseudo chat model class that can be used for testing, debugging, or educational purposes. This allows you to mock out calls to the chat model and simulate how a human would respond if they received the messages.\n",
"\n",
"In this notebook, we go over how to use this.\n",
"We expose a fake LLM class that can be used for testing. This allows you to mock out calls to the LLM and simulate what would happen if the LLM responded in a certain way.\n",
"LangChain provides a fake LLM class that can be used for testing. This allows you to mock out calls to the LLM and simulate what would happen if the LLM responded in a certain way.\n",
"Keep in mind that large language models are leaky abstractions! You'll have to use an LLM with sufficient capacity to generate well-formed JSON. In the OpenAI family, DaVinci can do reliably but Curie's ability already drops off dramatically. \n",
"\n",
"Use Pydantic to declare your data model. Pydantic's BaseModel like a Python dataclass, but with actual type checking + coercion."
"Use Pydantic to declare your data model. Pydantic's BaseModel is like a Python dataclass, but with actual type checking + coercion."
"While in some cases it is possible to fix any parsing mistakes by only looking at the output, in other cases it can't. An example of this is when the output is not just in the incorrect format, but is partially complete. Consider the below example."
"While in some cases it is possible to fix any parsing mistakes by only looking at the output, in other cases it isn't. An example of this is when the output is not just in the incorrect format, but is partially complete. Consider the below example."
]
},
{
@ -93,7 +93,7 @@
"id": "25631465",
"metadata": {},
"source": [
"If we try to parse this response as is, we will get an error"
"If we try to parse this response as is, we will get an error:"
By default, `PromptTemplate` will validate the `template` string by checking whether the `input_variables` match the variables defined in `template`. You can disable this behavior by setting `validate_template` to `False`
By default, `PromptTemplate` will validate the `template` string by checking whether the `input_variables` match the variables defined in `template`. You can disable this behavior by setting `validate_template` to `False`.
```python
template = "I am learning langchain because {reason}."
You can make use of templating by using a `MessagePromptTemplate`. You can build a `ChatPromptTemplate` from one or more `MessagePromptTemplates`. You can use `ChatPromptTemplate`'s `format_prompt` -- this returns a `PromptValue`, which you can convert to a string or Message object, depending on whether you want to use the formatted value as input to an llm or chat model.
For convenience, there is a `from_template` method exposed on the template. If you were to use this template, this is what it would look like:
For convenience, there is a `from_template` method defined on the template. If you were to use this template, this is what it would look like:
You can also turn off caching for particular nodes in chains. Note that because of certain interfaces, its often easier to construct the chain first, and then edit the LLM afterwards.
## Optional caching in chains
You can also turn off caching for particular nodes in chains. Note that because of certain interfaces, it's often easier to construct the chain first, and then edit the LLM afterwards.
As an example, we will load a summarizer map-reduce chain. We will cache results for the map-step, but then not freeze it for the combine step.
"""Select which examples to use based on the inputs."""
```
The only method it needs to expose is a ``select_examples`` method. This takes in the input variables and then returns a list of examples. It is up to each specific implementation as to how those examples are selected. Let's take a look at some below.
The only method it needs to define is a ``select_examples`` method. This takes in the input variables and then returns a list of examples. It is up to each specific implementation as to how those examples are selected.
@ -55,7 +55,7 @@ For more information, see [Custom Prompt Templates](./custom_prompt_template.htm
## Chat prompt template
The prompt to [Chat Models](../models/chat) is a list of chat messages.
The prompt to [chat models](../models/chat) is a list of chat messages.
Each chat message is associated with content, and an additional parameter called `role`.
For example, in the OpenAI [Chat Completions API](https://platform.openai.com/docs/guides/chat/introduction), a chat message can be associated with an AI assistant, a human or a system role.
seamusp
10 months ago
committed by
GitHub