mirror of
https://github.com/hwchase17/langchain
synced 2024-10-29 17:07:25 +00:00
24f889f2bc
ChatGLM LLM integration will by default accumulate conversation history(with_history=True) to ChatGLM backend api, which is not expected in most cases. This PR set with_history=False by default, user should explicitly set llm.with_history=True to turn this feature on. Related PR: #8048 #7774 --------- Co-authored-by: mlot <limpo2000@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>
126 lines
4.8 KiB
Plaintext
126 lines
4.8 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# ChatGLM\n",
|
||
"\n",
|
||
"[ChatGLM-6B](https://github.com/THUDM/ChatGLM-6B) is an open bilingual language model based on General Language Model (GLM) framework, with 6.2 billion parameters. With the quantization technique, users can deploy locally on consumer-grade graphics cards (only 6GB of GPU memory is required at the INT4 quantization level). \n",
|
||
"\n",
|
||
"[ChatGLM2-6B](https://github.com/THUDM/ChatGLM2-6B) is the second-generation version of the open-source bilingual (Chinese-English) chat model ChatGLM-6B. It retains the smooth conversation flow and low deployment threshold of the first-generation model, while introducing the new features like better performance, longer context and more efficient inference.\n",
|
||
"\n",
|
||
"This example goes over how to use LangChain to interact with ChatGLM2-6B Inference for text completion.\n",
|
||
"ChatGLM-6B and ChatGLM2-6B has the same api specs, so this example should work with both."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 21,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from langchain.llms import ChatGLM\n",
|
||
"from langchain import PromptTemplate, LLMChain\n",
|
||
"\n",
|
||
"# import os"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 22,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"template = \"\"\"{question}\"\"\"\n",
|
||
"prompt = PromptTemplate(template=template, input_variables=[\"question\"])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 23,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# default endpoint_url for a local deployed ChatGLM api server\n",
|
||
"endpoint_url = \"http://127.0.0.1:8000\"\n",
|
||
"\n",
|
||
"# direct access endpoint in a proxied environment\n",
|
||
"# os.environ['NO_PROXY'] = '127.0.0.1'\n",
|
||
"\n",
|
||
"llm = ChatGLM(\n",
|
||
" endpoint_url=endpoint_url,\n",
|
||
" max_token=80000,\n",
|
||
" history=[[\"我将从美国到中国来旅游，出行前希望了解中国的城市\", \"欢迎问我任何问题。\"]],\n",
|
||
" top_p=0.9,\n",
|
||
" model_kwargs={\"sample_model_args\": False},\n",
|
||
")\n",
|
||
"\n",
|
||
"# turn on with_history only when you want the LLM object to keep track of the conversation history\n",
|
||
"# and send the accumulated context to the backend model api, which make it stateful. By default it is stateless.\n",
|
||
"# llm.with_history = True"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 24,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"llm_chain = LLMChain(prompt=prompt, llm=llm)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 25,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"ChatGLM payload: {'prompt': '北京和上海两座城市有什么不同？', 'temperature': 0.1, 'history': [['我将从美国到中国来旅游，出行前希望了解中国的城市', '欢迎问我任何问题。']], 'max_length': 80000, 'top_p': 0.9, 'sample_model_args': False}\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"'北京和上海是中国的两个首都，它们在许多方面都有所不同。\\n\\n北京是中国的政治和文化中心，拥有悠久的历史和灿烂的文化。它是中国最重要的古都之一，也是中国历史上最后一个封建王朝的都城。北京有许多著名的古迹和景点，例如紫禁城、天安门广场和长城等。\\n\\n上海是中国最现代化的城市之一，也是中国商业和金融中心。上海拥有许多国际知名的企业和金融机构，同时也有许多著名的景点和美食。上海的外滩是一个历史悠久的商业区，拥有许多欧式建筑和餐馆。\\n\\n除此之外，北京和上海在交通和人口方面也有很大差异。北京是中国的首都，人口众多，交通拥堵问题较为严重。而上海是中国的商业和金融中心，人口密度较低，交通相对较为便利。\\n\\n总的来说，北京和上海是两个拥有独特魅力和特点的城市，可以根据自己的兴趣和时间来选择前往其中一座城市旅游。'"
|
||
]
|
||
},
|
||
"execution_count": 25,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"question = \"北京和上海两座城市有什么不同？\"\n",
|
||
"\n",
|
||
"llm_chain.run(question)"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "langchain-dev",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.10.12"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 2
|
||
}
|