You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
langchain/docs
Martin Krasser 79ed66f870
EXPERIMENTAL Generic LLM wrapper to support chat model interface with configurable chat prompt format (#8295)
## Update 2023-09-08

This PR now supports further models in addition to Lllama-2 chat models.
See [this comment](#issuecomment-1668988543) for further details. The
title of this PR has been updated accordingly.

## Original PR description

This PR adds a generic `Llama2Chat` model, a wrapper for LLMs able to
serve Llama-2 chat models (like `LlamaCPP`,
`HuggingFaceTextGenInference`, ...). It implements `BaseChatModel`,
converts a list of chat messages into the [required Llama-2 chat prompt
format](https://huggingface.co/blog/llama2#how-to-prompt-llama-2) and
forwards the formatted prompt as `str` to the wrapped `LLM`. Usage
example:

```python
# uses a locally hosted Llama2 chat model
llm = HuggingFaceTextGenInference(
    inference_server_url="http://127.0.0.1:8080/",
    max_new_tokens=512,
    top_k=50,
    temperature=0.1,
    repetition_penalty=1.03,
)

# Wrap llm to support Llama2 chat prompt format.
# Resulting model is a chat model
model = Llama2Chat(llm=llm)

messages = [
    SystemMessage(content="You are a helpful assistant."),
    MessagesPlaceholder(variable_name="chat_history"),
    HumanMessagePromptTemplate.from_template("{text}"),
]

prompt = ChatPromptTemplate.from_messages(messages)
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
chain = LLMChain(llm=model, prompt=prompt, memory=memory)

# use chat model in a conversation
# ...
```

Also part of this PR are tests and a demo notebook.

- Tag maintainer: @hwchase17
- Twitter handle: `@mrt1nz`

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
11 months ago
..
api_reference api doc newlines (#13378) 11 months ago
docs EXPERIMENTAL Generic LLM wrapper to support chat model interface with configurable chat prompt format (#8295) 11 months ago
scripts DOCS: format notebooks (#13371) 11 months ago
src add cookbook table (#12043) 12 months ago
static DOCS: langchain stack img update (#13421) 11 months ago
.local_build.sh Harrison/docs smith serve (#12898) 11 months ago
README.md Fix typos (#11663) 12 months ago
babel.config.js Restructure docs (#11620) 12 months ago
code-block-loader.js Restructure docs (#11620) 12 months ago
docusaurus.config.js FEAT docs integration cards site (#13379) 11 months ago
package-lock.json Upgrade docs postcss (#13031) 11 months ago
package.json Restructure docs (#11620) 12 months ago
settings.ini Restructure docs (#11620) 12 months ago
sidebars.js template readme's in docs (#13152) 11 months ago
vercel.json DOCS updated `async-faiss` example (#13434) 11 months ago
vercel_build.sh template readme's in docs (#13152) 11 months ago
vercel_requirements.txt Add api cross ref linking (#8275) 1 year ago

README.md

Website

This website is built using Docusaurus 2, a modern static website generator.

Installation

$ yarn

Local Development

$ yarn start

This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.

Build

$ yarn build

This command generates static content into the build directory and can be served using any static contents hosting service.

Deployment

Using SSH:

$ USE_SSH=true yarn deploy

Not using SSH:

$ GIT_USER=<Your GitHub username> yarn deploy

If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the gh-pages branch.

Continuous Integration

Some common defaults for linting/formatting have been set for you. If you integrate your project with an open-source Continuous Integration system (e.g. Travis CI, CircleCI), you may check for issues using the following command.

$ yarn ci