langchain/docs/modules/agents/streaming_stdout_final_only.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "23234b50-e6c6-4c87-9f97-259c15f36894",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Only streaming final agent output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29dd6333-307c-43df-b848-65001c01733b",
   "metadata": {},
   "source": [
    "If you only want the final output of an agent to be streamed, you can use the callback ``FinalStreamingStdOutCallbackHandler``.\n",
    "For this, the underlying LLM has to support streaming as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e4592215-6604-47e2-89ff-5db3af6d1e40",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.agents import load_tools\n",
    "from langchain.agents import initialize_agent\n",
    "from langchain.agents import AgentType\n",
    "from langchain.callbacks.streaming_stdout_final_only import FinalStreamingStdOutCallbackHandler\n",
    "from langchain.llms import OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19a813f7",
   "metadata": {},
   "source": [
    "Let's create the underlying LLM with ``streaming = True`` and pass a new instance of ``FinalStreamingStdOutCallbackHandler``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7fe81ef4",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = OpenAI(streaming=True, callbacks=[FinalStreamingStdOutCallbackHandler()], temperature=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ff45b85d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Konrad Adenauer became Chancellor of Germany in 1949, 74 years ago in 2023."
     ]
    },
    {
     "data": {
      "text/plain": [
       "'Konrad Adenauer became Chancellor of Germany in 1949, 74 years ago in 2023.'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tools = load_tools([\"wikipedia\", \"llm-math\"], llm=llm)\n",
    "agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=False)\n",
    "agent.run(\"It's 2023 now. How many years ago did Konrad Adenauer become Chancellor of Germany.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53a743b8",
   "metadata": {},
   "source": [
    "### Handling custom answer prefixes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23602c62",
   "metadata": {},
   "source": [
    "By default, we assume that the token sequence ``\"\\nFinal\", \" Answer\", \":\"`` indicates that the agent has reached an answers. We can, however, also pass a custom sequence to use as answer prefix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "5662a638",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = OpenAI(\n",
    "    streaming=True,\n",
    "    callbacks=[FinalStreamingStdOutCallbackHandler(answer_prefix_tokens=[\"\\nThe\", \" answer\", \":\"])],\n",
    "    temperature=0\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1a96cc0",
   "metadata": {},
   "source": [
    "Be aware you likely need to include whitespaces and new line characters in your token. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9278b522",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}