EXPERIMENTAL Generic LLM wrapper to support chat model interface with configurable chat prompt format (#8295)
## Update 2023-09-08
This PR now supports further models in addition to Lllama-2 chat models.
See [this comment](#issuecomment-1668988543) for further details. The
title of this PR has been updated accordingly.
## Original PR description
This PR adds a generic `Llama2Chat` model, a wrapper for LLMs able to
serve Llama-2 chat models (like `LlamaCPP`,
`HuggingFaceTextGenInference`, ...). It implements `BaseChatModel`,
converts a list of chat messages into the [required Llama-2 chat prompt
format](https://huggingface.co/blog/llama2#how-to-prompt-llama-2) and
forwards the formatted prompt as `str` to the wrapped `LLM`. Usage
example:
```python
# uses a locally hosted Llama2 chat model
llm = HuggingFaceTextGenInference(
inference_server_url="http://127.0.0.1:8080/",
max_new_tokens=512,
top_k=50,
temperature=0.1,
repetition_penalty=1.03,
)
# Wrap llm to support Llama2 chat prompt format.
# Resulting model is a chat model
model = Llama2Chat(llm=llm)
messages = [
SystemMessage(content="You are a helpful assistant."),
MessagesPlaceholder(variable_name="chat_history"),
HumanMessagePromptTemplate.from_template("{text}"),
]
prompt = ChatPromptTemplate.from_messages(messages)
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
chain = LLMChain(llm=model, prompt=prompt, memory=memory)
# use chat model in a conversation
# ...
```
Also part of this PR are tests and a demo notebook.
- Tag maintainer: @hwchase17
- Twitter handle: `@mrt1nz`
---------
Co-authored-by: Erick Friis <erick@langchain.dev>
2023-11-18 00:32:13 +00:00
|
|
|
"""**Chat Models** are a variation on language models.
|
|
|
|
|
|
|
|
|
|
While Chat Models use language models under the hood, the interface they expose
|
|
|
|
|
is a bit different. Rather than expose a "text in, text out" API, they expose
|
|
|
|
|
an interface where "chat messages" are the inputs and outputs.
|
|
|
|
|
|
|
|
|
|
**Class hierarchy:**
|
|
|
|
|
|
|
|
|
|
.. code-block::
|
|
|
|
|
|
|
|
|
|
BaseLanguageModel --> BaseChatModel --> <name> # Examples: ChatOpenAI, ChatGooglePalm
|
|
|
|
|
|
|
|
|
|
**Main helpers:**
|
|
|
|
|
|
|
|
|
|
.. code-block::
|
|
|
|
|
|
|
|
|
|
AIMessage, BaseMessage, HumanMessage
|
|
|
|
|
""" # noqa: E501
|
|
|
|
|
|
2024-03-09 01:14:23 +00:00
|
|
|
from langchain_experimental.chat_models.llm_wrapper import (
|
|
|
|
|
Llama2Chat,
|
|
|
|
|
Mixtral,
|
|
|
|
|
Orca,
|
|
|
|
|
Vicuna,
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
__all__ = ["Llama2Chat", "Orca", "Vicuna", "Mixtral"]
|
