"generations=[[ChatGeneration(text=\"The sea is a vast expanse of water that covers much of the Earth's surface. It is a source of travel, trade, and entertainment, and is also a place of scientific exploration and marine conservation. The sea is an important part of our world, and we should cherish and protect it.\", generation_info={'finish_reason': 'finished'}, message=AIMessage(content=\"The sea is a vast expanse of water that covers much of the Earth's surface. It is a source of travel, trade, and entertainment, and is also a place of scientific exploration and marine conservation. The sea is an important part of our world, and we should cherish and protect it.\", additional_kwargs={}, example=False))]] llm_output={} run=[RunInfo(run_id=UUID('d48160a6-5960-4c1d-8a0e-90e6b51a209b'))]\n",
"astream content='The sea is a vast' additional_kwargs={} example=False\n",
"astream content=' expanse of water, a place of mystery and adventure. It is the source of many cultures and civilizations, and a center of trade and exploration. The sea is also a source of life and beauty, with its unique marine life and diverse' additional_kwargs={} example=False\n",
"astream content=' coral reefs. Whether you are swimming, diving, or just watching the sea, it is a place that captivates the imagination and transforms the spirit.' additional_kwargs={} example=False\n"
" messages=[[HumanMessage(content=\"write a 20 words sentence about sea.\")]]\n",
" )\n",
" print(resp)\n",
"\n",
"\n",
"await run_aio_generate()\n",
"\n",
"\n",
"async def run_aio_stream():\n",
" async for res in chatLLM.astream(\n",
" [HumanMessage(content=\"write a 20 words sentence about sea.\")]\n",
" ):\n",
" print(\"astream\", res)\n",
"\n",
"\n",
"await run_aio_stream()"
"try:\n",
" for chunk in chat.stream(messages):\n",
" print(chunk.content, end=\"\", flush=True)\n",
"except TypeError as e:\n",
" print(\"\")"
]
},
{
@ -151,39 +180,36 @@
"source": [
"## Use different models in Qianfan\n",
"\n",
"In the case you want to deploy your own model based on Ernie Bot or third-party open-source model, you could follow these steps:\n",
"The default model is ERNIE-Bot-turbo, in the case you want to deploy your own model based on Ernie Bot or third-party open-source model, you could follow these steps:\n",
"\n",
"- 1. (Optional, if the model are included in the default models, skip it)Deploy your model in Qianfan Console, get your own customized deploy endpoint.\n",
"- 2. Set up the field called `endpoint` in the initialization:"
"1. (Optional, if the model are included in the default models, skip it) Deploy your model in Qianfan Console, get your own customized deploy endpoint.\n",
"2. Set up the field called `endpoint` in the initialization:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO] [09-15 20:00:50] logging.py:55 [t:139698882193216]: requesting llm api endpoint: /chat/bloomz_7b1\n"