mirror of
https://github.com/hwchase17/langchain
synced 2024-11-08 07:10:35 +00:00
79ed66f870
## Update 2023-09-08 This PR now supports further models in addition to Lllama-2 chat models. See [this comment](#issuecomment-1668988543) for further details. The title of this PR has been updated accordingly. ## Original PR description This PR adds a generic `Llama2Chat` model, a wrapper for LLMs able to serve Llama-2 chat models (like `LlamaCPP`, `HuggingFaceTextGenInference`, ...). It implements `BaseChatModel`, converts a list of chat messages into the [required Llama-2 chat prompt format](https://huggingface.co/blog/llama2#how-to-prompt-llama-2) and forwards the formatted prompt as `str` to the wrapped `LLM`. Usage example: ```python # uses a locally hosted Llama2 chat model llm = HuggingFaceTextGenInference( inference_server_url="http://127.0.0.1:8080/", max_new_tokens=512, top_k=50, temperature=0.1, repetition_penalty=1.03, ) # Wrap llm to support Llama2 chat prompt format. # Resulting model is a chat model model = Llama2Chat(llm=llm) messages = [ SystemMessage(content="You are a helpful assistant."), MessagesPlaceholder(variable_name="chat_history"), HumanMessagePromptTemplate.from_template("{text}"), ] prompt = ChatPromptTemplate.from_messages(messages) memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True) chain = LLMChain(llm=model, prompt=prompt, memory=memory) # use chat model in a conversation # ... ``` Also part of this PR are tests and a demo notebook. - Tag maintainer: @hwchase17 - Twitter handle: `@mrt1nz` --------- Co-authored-by: Erick Friis <erick@langchain.dev>
27 lines
648 B
Python
27 lines
648 B
Python
"""**Chat Models** are a variation on language models.
|
|
|
|
While Chat Models use language models under the hood, the interface they expose
|
|
is a bit different. Rather than expose a "text in, text out" API, they expose
|
|
an interface where "chat messages" are the inputs and outputs.
|
|
|
|
**Class hierarchy:**
|
|
|
|
.. code-block::
|
|
|
|
BaseLanguageModel --> BaseChatModel --> <name> # Examples: ChatOpenAI, ChatGooglePalm
|
|
|
|
**Main helpers:**
|
|
|
|
.. code-block::
|
|
|
|
AIMessage, BaseMessage, HumanMessage
|
|
""" # noqa: E501
|
|
|
|
from langchain_experimental.chat_models.llm_wrapper import Llama2Chat, Orca, Vicuna
|
|
|
|
__all__ = [
|
|
"Llama2Chat",
|
|
"Orca",
|
|
"Vicuna",
|
|
]
|