{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Baidu Qianfan\n",
    "\n",
    "Baidu AI Cloud Qianfan Platform is a one-stop large model development and service operation platform for enterprise developers. Qianfan not only provides including the model of Wenxin Yiyan (ERNIE-Bot) and the third-party open source models, but also provides various AI development tools and the whole set of development environment, which facilitates customers to use and develop large model applications easily.\n",
    "\n",
    "Basically, those model are split into the following type:\n",
    "\n",
    "- Embedding\n",
    "- Chat\n",
    "- Completion\n",
    "\n",
    "In this notebook, we will introduce how to use langchain with [Qianfan](https://cloud.baidu.com/doc/WENXINWORKSHOP/index.html) mainly in `Chat` corresponding\n",
    " to the package `langchain/chat_models` in langchain:\n",
    "\n",
    "\n",
    "## API Initialization\n",
    "\n",
    "To use the LLM services based on Baidu Qianfan, you have to initialize these parameters:\n",
    "\n",
    "You could either choose to init the AK,SK in enviroment variables or init params:\n",
    "\n",
    "```base\n",
    "export QIANFAN_AK=XXX\n",
    "export QIANFAN_SK=XXX\n",
    "```\n",
    "\n",
    "## Current supported models:\n",
    "\n",
    "- ERNIE-Bot-turbo (default models)\n",
    "- ERNIE-Bot\n",
    "- BLOOMZ-7B\n",
    "- Llama-2-7b-chat\n",
    "- Llama-2-13b-chat\n",
    "- Llama-2-70b-chat\n",
    "- Qianfan-BLOOMZ-7B-compressed\n",
    "- Qianfan-Chinese-Llama-2-7B\n",
    "- ChatGLM2-6B-32K\n",
    "- AquilaChat-7B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [09-15 20:00:29] logging.py:55 [t:139698882193216]: requesting llm api endpoint: /chat/eb-instant\n"
     ]
    }
   ],
   "source": [
    "\"\"\"For basic init and call\"\"\"\n",
    "from langchain.chat_models import QianfanChatEndpoint \n",
    "from langchain.chat_models.base import HumanMessage\n",
    "import os\n",
    "os.environ[\"QIANFAN_AK\"] = \"your_ak\"\n",
    "os.environ[\"QIANFAN_SK\"] = \"your_sk\"\n",
    "\n",
    "chat = QianfanChatEndpoint(\n",
    "                            streaming=True, \n",
    "                            )\n",
    "res = chat([HumanMessage(content=\"write a funny joke\")])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [09-15 20:00:36] logging.py:55 [t:139698882193216]: requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [09-15 20:00:37] logging.py:55 [t:139698882193216]: async requesting llm api endpoint: /chat/eb-instant\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chat resp: content='您好，您似乎输入' additional_kwargs={} example=False\n",
      "chat resp: content='了一个话题标签，请问需要我帮您找到什么资料或者帮助您解答什么问题吗？' additional_kwargs={} example=False\n",
      "chat resp: content='' additional_kwargs={} example=False\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [09-15 20:00:39] logging.py:55 [t:139698882193216]: async requesting llm api endpoint: /chat/eb-instant\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "generations=[[ChatGeneration(text=\"The sea is a vast expanse of water that covers much of the Earth's surface. It is a source of travel, trade, and entertainment, and is also a place of scientific exploration and marine conservation. The sea is an important part of our world, and we should cherish and protect it.\", generation_info={'finish_reason': 'finished'}, message=AIMessage(content=\"The sea is a vast expanse of water that covers much of the Earth's surface. It is a source of travel, trade, and entertainment, and is also a place of scientific exploration and marine conservation. The sea is an important part of our world, and we should cherish and protect it.\", additional_kwargs={}, example=False))]] llm_output={} run=[RunInfo(run_id=UUID('d48160a6-5960-4c1d-8a0e-90e6b51a209b'))]\n",
      "astream content='The sea is a vast' additional_kwargs={} example=False\n",
      "astream content=' expanse of water, a place of mystery and adventure. It is the source of many cultures and civilizations, and a center of trade and exploration. The sea is also a source of life and beauty, with its unique marine life and diverse' additional_kwargs={} example=False\n",
      "astream content=' coral reefs. Whether you are swimming, diving, or just watching the sea, it is a place that captivates the imagination and transforms the spirit.' additional_kwargs={} example=False\n"
     ]
    }
   ],
   "source": [
    "    \n",
    "from langchain.chat_models import QianfanChatEndpoint\n",
    "from langchain.schema import HumanMessage\n",
    "\n",
    "chatLLM = QianfanChatEndpoint(\n",
    "    streaming=True,\n",
    ")\n",
    "res = chatLLM.stream([HumanMessage(content=\"hi\")], streaming=True)\n",
    "for r in res:\n",
    "    print(\"chat resp:\", r)\n",
    "\n",
    "\n",
    "async def run_aio_generate():\n",
    "    resp = await chatLLM.agenerate(messages=[[HumanMessage(content=\"write a 20 words sentence about sea.\")]])\n",
    "    print(resp)\n",
    "        \n",
    "await run_aio_generate()\n",
    "\n",
    "async def run_aio_stream():\n",
    "    async for res in chatLLM.astream([HumanMessage(content=\"write a 20 words sentence about sea.\")]):\n",
    "        print(\"astream\", res)\n",
    "        \n",
    "await run_aio_stream()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use different models in Qianfan\n",
    "\n",
    "In the case you want to deploy your own model based on Ernie Bot or third-party open sources model, you could follow these steps:\n",
    "\n",
    "- 1. （Optional, if the model are included in the default models, skip it）Deploy your model in Qianfan Console, get your own customized deploy endpoint.\n",
    "- 2. Set up the field called `endpoint` in the initlization:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [09-15 20:00:50] logging.py:55 [t:139698882193216]: requesting llm api endpoint: /chat/bloomz_7b1\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "content='你好！很高兴见到你。' additional_kwargs={} example=False\n"
     ]
    }
   ],
   "source": [
    "chatBloom = QianfanChatEndpoint(\n",
    "                            streaming=True, \n",
    "                            model=\"BLOOMZ-7B\",\n",
    "                            )\n",
    "res = chatBloom([HumanMessage(content=\"hi\")])\n",
    "print(res)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Params:\n",
    "\n",
    "For now, only `ERNIE-Bot` and `ERNIE-Bot-turbo` support model params below, we might support more models in the future.\n",
    "\n",
    "- temperature\n",
    "- top_p\n",
    "- penalty_score\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [09-15 20:00:57] logging.py:55 [t:139698882193216]: requesting llm api endpoint: /chat/eb-instant\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "content='您好，您似乎输入' additional_kwargs={} example=False\n",
      "content='了一个文本字符串，但并没有给出具体的问题或场景。' additional_kwargs={} example=False\n",
      "content='如果您能提供更多信息，我可以更好地回答您的问题。' additional_kwargs={} example=False\n",
      "content='' additional_kwargs={} example=False\n"
     ]
    }
   ],
   "source": [
    "res = chat.stream([HumanMessage(content=\"hi\")], **{'top_p': 0.4, 'temperature': 0.1, 'penalty_score': 1})\n",
    "\n",
    "for r in res:\n",
    "    print(r)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  },
  "vscode": {
   "interpreter": {
    "hash": "6fa70026b407ae751a5c9e6bd7f7d482379da8ad616f98512780b705c84ee157"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}