mirror of
https://github.com/hwchase17/langchain
synced 2024-10-29 17:07:25 +00:00
ebe08412ad
- **Description:** QianfanEndpoint bugs for SystemMessages. When the `SystemMessage` is input as the messages to `chat_models.QianfanEndpoint`. A `TypeError` will be raised. - **Issue:** #10643 - **Dependencies:** - **Tag maintainer:** @baskaryan - **Twitter handle:** no
254 lines
8.8 KiB
Plaintext
254 lines
8.8 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Baidu Qianfan\n",
|
||
"\n",
|
||
"Baidu AI Cloud Qianfan Platform is a one-stop large model development and service operation platform for enterprise developers. Qianfan not only provides including the model of Wenxin Yiyan (ERNIE-Bot) and the third-party open source models, but also provides various AI development tools and the whole set of development environment, which facilitates customers to use and develop large model applications easily.\n",
|
||
"\n",
|
||
"Basically, those model are split into the following type:\n",
|
||
"\n",
|
||
"- Embedding\n",
|
||
"- Chat\n",
|
||
"- Completion\n",
|
||
"\n",
|
||
"In this notebook, we will introduce how to use langchain with [Qianfan](https://cloud.baidu.com/doc/WENXINWORKSHOP/index.html) mainly in `Chat` corresponding\n",
|
||
" to the package `langchain/chat_models` in langchain:\n",
|
||
"\n",
|
||
"\n",
|
||
"## API Initialization\n",
|
||
"\n",
|
||
"To use the LLM services based on Baidu Qianfan, you have to initialize these parameters:\n",
|
||
"\n",
|
||
"You could either choose to init the AK,SK in enviroment variables or init params:\n",
|
||
"\n",
|
||
"```base\n",
|
||
"export QIANFAN_AK=XXX\n",
|
||
"export QIANFAN_SK=XXX\n",
|
||
"```\n",
|
||
"\n",
|
||
"## Current supported models:\n",
|
||
"\n",
|
||
"- ERNIE-Bot-turbo (default models)\n",
|
||
"- ERNIE-Bot\n",
|
||
"- BLOOMZ-7B\n",
|
||
"- Llama-2-7b-chat\n",
|
||
"- Llama-2-13b-chat\n",
|
||
"- Llama-2-70b-chat\n",
|
||
"- Qianfan-BLOOMZ-7B-compressed\n",
|
||
"- Qianfan-Chinese-Llama-2-7B\n",
|
||
"- ChatGLM2-6B-32K\n",
|
||
"- AquilaChat-7B"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[INFO] [09-15 20:00:29] logging.py:55 [t:139698882193216]: requesting llm api endpoint: /chat/eb-instant\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"\"\"\"For basic init and call\"\"\"\n",
|
||
"from langchain.chat_models import QianfanChatEndpoint \n",
|
||
"from langchain.chat_models.base import HumanMessage\n",
|
||
"import os\n",
|
||
"os.environ[\"QIANFAN_AK\"] = \"your_ak\"\n",
|
||
"os.environ[\"QIANFAN_SK\"] = \"your_sk\"\n",
|
||
"\n",
|
||
"chat = QianfanChatEndpoint(\n",
|
||
" streaming=True, \n",
|
||
" )\n",
|
||
"res = chat([HumanMessage(content=\"write a funny joke\")])\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[INFO] [09-15 20:00:36] logging.py:55 [t:139698882193216]: requesting llm api endpoint: /chat/eb-instant\n",
|
||
"[INFO] [09-15 20:00:37] logging.py:55 [t:139698882193216]: async requesting llm api endpoint: /chat/eb-instant\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"chat resp: content='您好，您似乎输入' additional_kwargs={} example=False\n",
|
||
"chat resp: content='了一个话题标签，请问需要我帮您找到什么资料或者帮助您解答什么问题吗？' additional_kwargs={} example=False\n",
|
||
"chat resp: content='' additional_kwargs={} example=False\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[INFO] [09-15 20:00:39] logging.py:55 [t:139698882193216]: async requesting llm api endpoint: /chat/eb-instant\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"generations=[[ChatGeneration(text=\"The sea is a vast expanse of water that covers much of the Earth's surface. It is a source of travel, trade, and entertainment, and is also a place of scientific exploration and marine conservation. The sea is an important part of our world, and we should cherish and protect it.\", generation_info={'finish_reason': 'finished'}, message=AIMessage(content=\"The sea is a vast expanse of water that covers much of the Earth's surface. It is a source of travel, trade, and entertainment, and is also a place of scientific exploration and marine conservation. The sea is an important part of our world, and we should cherish and protect it.\", additional_kwargs={}, example=False))]] llm_output={} run=[RunInfo(run_id=UUID('d48160a6-5960-4c1d-8a0e-90e6b51a209b'))]\n",
|
||
"astream content='The sea is a vast' additional_kwargs={} example=False\n",
|
||
"astream content=' expanse of water, a place of mystery and adventure. It is the source of many cultures and civilizations, and a center of trade and exploration. The sea is also a source of life and beauty, with its unique marine life and diverse' additional_kwargs={} example=False\n",
|
||
"astream content=' coral reefs. Whether you are swimming, diving, or just watching the sea, it is a place that captivates the imagination and transforms the spirit.' additional_kwargs={} example=False\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
" \n",
|
||
"from langchain.chat_models import QianfanChatEndpoint\n",
|
||
"from langchain.schema import HumanMessage\n",
|
||
"\n",
|
||
"chatLLM = QianfanChatEndpoint(\n",
|
||
" streaming=True,\n",
|
||
")\n",
|
||
"res = chatLLM.stream([HumanMessage(content=\"hi\")], streaming=True)\n",
|
||
"for r in res:\n",
|
||
" print(\"chat resp:\", r)\n",
|
||
"\n",
|
||
"\n",
|
||
"async def run_aio_generate():\n",
|
||
" resp = await chatLLM.agenerate(messages=[[HumanMessage(content=\"write a 20 words sentence about sea.\")]])\n",
|
||
" print(resp)\n",
|
||
" \n",
|
||
"await run_aio_generate()\n",
|
||
"\n",
|
||
"async def run_aio_stream():\n",
|
||
" async for res in chatLLM.astream([HumanMessage(content=\"write a 20 words sentence about sea.\")]):\n",
|
||
" print(\"astream\", res)\n",
|
||
" \n",
|
||
"await run_aio_stream()"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Use different models in Qianfan\n",
|
||
"\n",
|
||
"In the case you want to deploy your own model based on Ernie Bot or third-party open sources model, you could follow these steps:\n",
|
||
"\n",
|
||
"- 1. （Optional, if the model are included in the default models, skip it）Deploy your model in Qianfan Console, get your own customized deploy endpoint.\n",
|
||
"- 2. Set up the field called `endpoint` in the initlization:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[INFO] [09-15 20:00:50] logging.py:55 [t:139698882193216]: requesting llm api endpoint: /chat/bloomz_7b1\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"content='你好！很高兴见到你。' additional_kwargs={} example=False\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"chatBloom = QianfanChatEndpoint(\n",
|
||
" streaming=True, \n",
|
||
" model=\"BLOOMZ-7B\",\n",
|
||
" )\n",
|
||
"res = chatBloom([HumanMessage(content=\"hi\")])\n",
|
||
"print(res)"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Model Params:\n",
|
||
"\n",
|
||
"For now, only `ERNIE-Bot` and `ERNIE-Bot-turbo` support model params below, we might support more models in the future.\n",
|
||
"\n",
|
||
"- temperature\n",
|
||
"- top_p\n",
|
||
"- penalty_score\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[INFO] [09-15 20:00:57] logging.py:55 [t:139698882193216]: requesting llm api endpoint: /chat/eb-instant\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"content='您好，您似乎输入' additional_kwargs={} example=False\n",
|
||
"content='了一个文本字符串，但并没有给出具体的问题或场景。' additional_kwargs={} example=False\n",
|
||
"content='如果您能提供更多信息，我可以更好地回答您的问题。' additional_kwargs={} example=False\n",
|
||
"content='' additional_kwargs={} example=False\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"res = chat.stream([HumanMessage(content=\"hi\")], **{'top_p': 0.4, 'temperature': 0.1, 'penalty_score': 1})\n",
|
||
"\n",
|
||
"for r in res:\n",
|
||
" print(r)"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "base",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.11.4"
|
||
},
|
||
"vscode": {
|
||
"interpreter": {
|
||
"hash": "6fa70026b407ae751a5c9e6bd7f7d482379da8ad616f98512780b705c84ee157"
|
||
}
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 2
|
||
}
|