langchain/docs/modules/prompts/examples/output_parsers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "084ee2f0",
   "metadata": {},
   "source": [
    "# Output Parsers\n",
    "\n",
    "Language models output text. But many times you may want to get more structured information than just text back. This is where output parsers come in.\n",
    "\n",
    "Output parsers are classes that help structure language model responses. There are two main methods an output parser must implement:\n",
    "\n",
    "- `get_format_instructions() -> str`: A method which returns a string containing instructions for how the output of a language model should be formatted.\n",
    "- `parse(str) -> Any`: A method which takes in a string (assumed to be the response from a language model) and parses it into some structure.\n",
    "\n",
    "Below we go over some examples of output parsers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "5f0c8a33",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate\n",
    "from langchain.llms import OpenAI\n",
    "from langchain.chat_models import ChatOpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1ae632a",
   "metadata": {},
   "source": [
    "## PydanticOutputParser\n",
    "This output parser allows users to specify an arbitrary JSON schema and query LLMs for JSON outputs that conform to that schema.\n",
    "\n",
    "Keep in mind that large language models are leaky abstractions! You'll have to use an LLM with sufficient capacity to generate well-formed JSON. In the OpenAI family, DaVinci can do reliably but Curie's ability already drops off dramatically. \n",
    "\n",
    "Use Pydantic to declare your data model. Pydantic's BaseModel like a Python dataclass, but with actual type checking + coercion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "cba6d8e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.output_parsers import PydanticOutputParser\n",
    "from pydantic import BaseModel, Field, validator\n",
    "from typing import List"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0a203100",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = 'text-davinci-003'\n",
    "temperature = 0.0\n",
    "model = OpenAI(model_name=model_name, temperature=temperature)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b3f16168",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Joke(setup='Why did the chicken cross the playground?', punchline='To get to the other slide!')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define your desired data structure.\n",
    "class Joke(BaseModel):\n",
    "    setup: str = Field(description=\"question to set up a joke\")\n",
    "    punchline: str = Field(description=\"answer to resolve the joke\")\n",
    "    \n",
    "    # You can add custom validation logic easily with Pydantic.\n",
    "    @validator('setup')\n",
    "    def question_ends_with_question_mark(cls, field):\n",
    "        if field[-1] != '?':\n",
    "            raise ValueError(\"Badly formed question!\")\n",
    "        return field\n",
    "\n",
    "# And a query intented to prompt a language model to populate the data structure.\n",
    "joke_query = \"Tell me a joke.\"\n",
    "\n",
    "# Set up a parser + inject instructions into the prompt template.\n",
    "parser = PydanticOutputParser(pydantic_object=Joke)\n",
    "\n",
    "prompt = PromptTemplate(\n",
    "    template=\"Answer the user query.\\n{format_instructions}\\n{query}\\n\",\n",
    "    input_variables=[\"query\"],\n",
    "    partial_variables={\"format_instructions\": parser.get_format_instructions()}\n",
    ")\n",
    "\n",
    "_input = prompt.format_prompt(query=joke_query)\n",
    "\n",
    "output = model(_input.to_string())\n",
    "\n",
    "parser.parse(output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "03049f88",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Actor(name='Tom Hanks', film_names=['Forrest Gump', 'Saving Private Ryan', 'The Green Mile', 'Cast Away', 'Toy Story', 'A League of Their Own'])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Here's another example, but with a compound typed field.\n",
    "class Actor(BaseModel):\n",
    "    name: str = Field(description=\"name of an actor\")\n",
    "    film_names: List[str] = Field(description=\"list of names of films they starred in\")\n",
    "        \n",
    "actor_query = \"Generate the filmography for a random actor.\"\n",
    "\n",
    "parser = PydanticOutputParser(pydantic_object=Actor)\n",
    "\n",
    "prompt = PromptTemplate(\n",
    "    template=\"Answer the user query.\\n{format_instructions}\\n{query}\\n\",\n",
    "    input_variables=[\"query\"],\n",
    "    partial_variables={\"format_instructions\": parser.get_format_instructions()}\n",
    ")\n",
    "\n",
    "_input = prompt.format_prompt(query=actor_query)\n",
    "\n",
    "output = model(_input.to_string())\n",
    "\n",
    "parser.parse(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61f67890",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91871002",
   "metadata": {},
   "source": [
    "## Structured Output Parser\n",
    "\n",
    "While the Pydantic/JSON parser is more powerful, we initially experimented data structures having text fields only."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b492997a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.output_parsers import StructuredOutputParser, ResponseSchema"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09473dce",
   "metadata": {},
   "source": [
    "Here we define the response schema we want to receive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "432ac44a",
   "metadata": {},
   "outputs": [],
   "source": [
    "response_schemas = [\n",
    "    ResponseSchema(name=\"answer\", description=\"answer to the user's question\"),\n",
    "    ResponseSchema(name=\"source\", description=\"source used to answer the user's question, should be a website.\")\n",
    "]\n",
    "output_parser = StructuredOutputParser.from_response_schemas(response_schemas)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b92ce96",
   "metadata": {},
   "source": [
    "We now get a string that contains instructions for how the response should be formatted, and we then insert that into our prompt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "593cfc25",
   "metadata": {},
   "outputs": [],
   "source": [
    "format_instructions = output_parser.get_format_instructions()\n",
    "prompt = PromptTemplate(\n",
    "    template=\"answer the users question as best as possible.\\n{format_instructions}\\n{question}\",\n",
    "    input_variables=[\"question\"],\n",
    "    partial_variables={\"format_instructions\": format_instructions}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0943e783",
   "metadata": {},
   "source": [
    "We can now use this to format a prompt to send to the language model, and then parse the returned result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "106f1ba6",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = OpenAI(temperature=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "86d9d24f",
   "metadata": {},
   "outputs": [],
   "source": [
    "_input = prompt.format_prompt(question=\"what's the capital of france\")\n",
    "output = model(_input.to_string())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "956bdc99",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'answer': 'Paris', 'source': 'https://en.wikipedia.org/wiki/Paris'}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output_parser.parse(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da639285",
   "metadata": {},
   "source": [
    "And here's an example of using this in a chat model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "8f483d7d",
   "metadata": {},
   "outputs": [],
   "source": [
    "chat_model = ChatOpenAI(temperature=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f761cbf1",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = ChatPromptTemplate(\n",
    "    messages=[\n",
    "        HumanMessagePromptTemplate.from_template(\"answer the users question as best as possible.\\n{format_instructions}\\n{question}\")  \n",
    "    ],\n",
    "    input_variables=[\"question\"],\n",
    "    partial_variables={\"format_instructions\": format_instructions}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "edd73ae3",
   "metadata": {},
   "outputs": [],
   "source": [
    "_input = prompt.format_prompt(question=\"what's the capital of france\")\n",
    "output = chat_model(_input.to_messages())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "a3c8b91e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'answer': 'Paris', 'source': 'https://en.wikipedia.org/wiki/Paris'}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output_parser.parse(output.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9936fa27",
   "metadata": {},
   "source": [
    "## CommaSeparatedListOutputParser\n",
    "\n",
    "Here's another parser strictly less powerful than Pydantic/JSON parsing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "872246d7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.output_parsers import CommaSeparatedListOutputParser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "c3f9aee6",
   "metadata": {},
   "outputs": [],
   "source": [
    "output_parser = CommaSeparatedListOutputParser()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "e77871b7",
   "metadata": {},
   "outputs": [],
   "source": [
    "format_instructions = output_parser.get_format_instructions()\n",
    "prompt = PromptTemplate(\n",
    "    template=\"List five {subject}.\\n{format_instructions}\",\n",
    "    input_variables=[\"subject\"],\n",
    "    partial_variables={\"format_instructions\": format_instructions}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "a71cb5d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = OpenAI(temperature=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "783d7d98",
   "metadata": {},
   "outputs": [],
   "source": [
    "_input = prompt.format(subject=\"ice cream flavors\")\n",
    "output = model(_input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "fcb81344",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Vanilla',\n",
       " 'Chocolate',\n",
       " 'Strawberry',\n",
       " 'Mint Chocolate Chip',\n",
       " 'Cookies and Cream']"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output_parser.parse(output)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}