"Getting structured output from raw LLM generations is hard.\n",
"\n",
"For example, suppose you need the model output formatted with a specific schema for:\n",
"LLMs can be used to generate text that is structured according to a specific schema. This can be useful in a number of scenarios, including:\n",
"\n",
"- Extracting a structured row to insert into a database \n",
"- Extracting API parameters\n",
@ -43,17 +41,23 @@
"source": [
"## Overview \n",
"\n",
"There are two primary approaches for this:\n",
"\n",
"- `Functions`: Some LLMs can call [functions](https://openai.com/blog/function-calling-and-other-api-updates) to extract arbitrary entities from LLM responses.\n",
"There are two broad approaches for this:\n",
"\n",
"- `Parsing`: [Output parsers](/docs/modules/model_io/output_parsers/) are classes that structure LLM responses. \n",
"- `Tools and JSON mode`: Some LLMs specifically support structured output generation in certain contexts. Examples include OpenAI's [function and tool calling](https://platform.openai.com/docs/guides/function-calling) or [JSON mode](https://platform.openai.com/docs/guides/text-generation/json-mode).\n",
"\n",
"Only some LLMs support functions (e.g., OpenAI), and they are more general than parsers. \n",
"- `Parsing`: LLMs can often be instructed to output their response in a dseired format. [Output parsers](/docs/modules/model_io/output_parsers/) will parse text generations into a structured form.\n",
"\n",
"Parsers extract precisely what is enumerated in a provided schema (e.g., specific attributes of a person).\n",
"\n",
"Functions can infer things beyond of a provided schema (e.g., attributes about a person that you did not ask for)."
"Functions and tools can infer things beyond of a provided schema (e.g., attributes about a person that you did not ask for)."
]
},
{
"cell_type": "markdown",
"id": "fbea06b5-66b6-4958-936d-23212061e4c8",
"metadata": {},
"source": [
"## Option 1: Leveraging tools and JSON mode"
]
},
{
@ -61,13 +65,16 @@
"id": "25d89f21",
"metadata": {},
"source": [
"## Quickstart\n",
"### Quickstart\n",
"\n",
"OpenAI functions are one way to get started with extraction.\n",
"`create_structured_output_runnable` will create Runnables to support structured data extraction via OpenAI tool use and JSON mode.\n",
"\n",
"Define a schema that specifies the properties we want to extract from the LLM output.\n",
"The desired output schema can be expressed either via a Pydantic model or a Python dict representing valid [JsonSchema](https://json-schema.org/).\n",
"\n",
"Then, we can use `create_extraction_chain` to extract our desired schema using an OpenAI function call."
"This function supports three modes for structured data extraction:\n",
"- `\"openai-functions\"` will define OpenAI functions and bind them to the given LLM;\n",
"- `\"openai-tools\"` will define OpenAI tools and bind them to the given LLM;\n",
"- `\"openai-json\"` will bind `response_format={\"type\": \"json_object\"}` to the given LLM.\n"
"inp = \"Alex is 5 feet tall and has blond hair.\"\n",
"runnable.invoke(inp)"
]
},
{
"cell_type": "markdown",
"id": "6f7eb826",
"id": "02fd21ff-27a8-4890-bb18-fc852cafb18a",
"metadata": {},
"source": [
"## Option 1: OpenAI functions\n",
"\n",
"### Looking under the hood\n",
"\n",
"Let's dig into what is happening when we call `create_extraction_chain`.\n",
"\n",
"The [LangSmith trace](https://smith.langchain.com/public/72bc3205-7743-4ca6-929a-966a9d4c2a77/r) shows that we call the function `information_extraction` on the input string, `inp`.\n",
"This `information_extraction` function is defined [here](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/chains/openai_functions/extraction.py) and returns a dict.\n",
"\n",
"We can see the `dict` in the model output:\n",
"```\n",
" {\n",
" \"info\": [\n",
" {\n",
" \"name\": \"Alex\",\n",
" \"height\": 5,\n",
" \"hair_color\": \"blonde\"\n",
" },\n",
" {\n",
" \"name\": \"Claudia\",\n",
" \"height\": 6,\n",
" \"hair_color\": \"brunette\"\n",
" }\n",
" ]\n",
" }\n",
"```\n",
"\n",
"The `create_extraction_chain` then parses the raw LLM output for us using [`JsonKeyOutputFunctionsParser`](https://github.com/langchain-ai/langchain/blob/f81e613086d211327b67b0fb591fd4d5f9a85860/libs/langchain/langchain/chains/openai_functions/extraction.py#L62).\n",
"\n",
"This results in the list of JSON objects returned by the chain above:\n",
"A convenient way to express desired output schemas is via Pydantic. The above example specified the desired output schema via `Person`, a Pydantic model. Such schemas can be easily combined together to generate richer output formats:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "c1c8fe71-0ae4-466a-b32f-001c59b62bb3",
"metadata": {},
"outputs": [],
"source": [
"### Multiple entity types\n",
"from typing import Sequence\n",
"\n",
"\n",
"class People(BaseModel):\n",
" \"\"\"Identifying information about all people in a text.\"\"\"\n",
"\n",
"We can extend this further.\n",
" people: Sequence[Person]\n",
"\n",
"Let's say we want to differentiate between dogs and people.\n",
"\n",
"We can add `person_` and `dog_` prefixes for each property"
"inp = \"\"\"Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\n",
"Willow is a German Shepherd that likes to play with other dogs and can always be found playing with Milo, a border collie that lives close by.\"\"\"\n",
"\n",
"chain.run(inp)"
"inp = \"Alex is 5 feet tall. He is blond.\"\n",
"runnable.invoke(inp)"
]
},
{
@ -284,11 +278,9 @@
"id": "34f3b958",
"metadata": {},
"source": [
"### Extra information\n",
"#### Extra information\n",
"\n",
"The power of functions (relative to using parsers alone) lies in the ability to perform semantic extraction.\n",
"\n",
"In particular, `we can ask for things that are not explicitly enumerated in the schema`.\n",
"Runnables constructed via `create_structured_output_runnable` generally are capable of semantic extraction, such that they can populate information that is not explicitly enumerated in the schema.\n",
"\n",
"Suppose we want unspecified additional information about dogs. \n",
"\n",
@ -297,44 +289,53 @@
},
{
"cell_type": "code",
"execution_count": 10,
"id": "40c7b26f",
"execution_count": 8,
"id": "0ed3b5e6-a7f3-453e-be61-d94fc665c16b",
"metadata": {},
"outputs": [],
"source": [
"inp = \"\"\"Alex is 5 feet tall and has blond hair.\n",
"Claudia is 1 feet taller Alex and jumps higher than him.\n",
"Claudia is a brunette and has a beagle named Harry.\n",
"Harry likes to play with other dogs and can always be found\n",
"playing with Milo, a border collie that lives close by.\"\"\""
" 'dog_extra_info': 'likes to play with other dogs'},\n",
" {'dog_name': 'Milo',\n",
" 'dog_breed': 'border collie',\n",
" 'dog_extra_info': 'lives close by'}]"
"People(people=[Person(person_name='Alex', person_height=60, person_hair_color='blond', dog_breed=None, dog_name=None, dog_extra_info=None), Person(person_name='Claudia', person_height=72, person_hair_color='brunette', dog_breed='beagle', dog_name='Harry', dog_extra_info='likes to play with other dogs and can always be found playing with Milo, a border collie that lives close by.')])"
"Pydantic is a data validation and settings management library for Python. \n",
"`create_structured_output_runnable` supports varying implementations of the underlying extraction under the hood, which are configured via the `mode` parameter. This parameter can be one of `\"openai-functions\"`, `\"openai-tools\"`, or `\"openai-json\"`."
]
},
{
"cell_type": "markdown",
"id": "7c8e0b00-d6e6-432d-b9b0-8d0a3c0c6572",
"metadata": {},
"source": [
"#### OpenAI Functions and Tools"
]
},
{
"cell_type": "markdown",
"id": "07ccdbb1-cbe5-45af-87e4-dde42baee5eb",
"metadata": {},
"source": [
"Some LLMs are fine-tuned to support the invocation of functions or tools. If they are given an input schema for a tool and recognize an occasion to use it, they may emit JSON output conforming to that schema. We can leverage this to drive structured data extraction from natural language.\n",
"\n",
"It allows you to create data classes with attributes that are automatically validated when you instantiate an object.\n",
"OpenAI originally released this via a [`functions` parameter in its chat completions API](https://openai.com/blog/function-calling-and-other-api-updates). This has since been deprecated in favor of a [`tools` parameter](https://platform.openai.com/docs/guides/function-calling), which can include (multiple) functions."
"The corresponding [LangSmith trace](https://smith.langchain.com/public/04cc37a7-7a1c-4bae-b972-1cb1a642568c/r) illustrates the tool call that generated our structured output.\n",
"Some LLMs support generating JSON more generally. OpenAI implements this via a [`response_format` parameter](https://platform.openai.com/docs/guides/text-generation/json-mode) in its chat completions API.\n",
"\n",
"Note that this method may require explicit prompting (e.g., OpenAI requires that input messages contain the word \"json\" in some form when using this parameter)."
"inp = \"\"\"Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\"\"\"\n",
"chain.run(inp)"
"runnable.invoke({\"input\": inp})"
]
},
{
"cell_type": "markdown",
"id": "07a0351a",
"id": "b22d8262-a9b8-415c-a142-d0ee4db7ec2b",
"metadata": {},
"source": [
"As we can see from the [trace](https://smith.langchain.com/public/fed50ae6-26bb-4235-a254-e0b7a229d10f/r), we use the function `information_extraction`, as above, with the Pydantic schema. "
"### Few-shot examples"
]
},
{
"cell_type": "markdown",
"id": "cbd9f121",
"id": "a01c75f6-99d7-4d7b-a58f-b0ea7e8f338a",
"metadata": {},
"source": [
"## Option 2: Parsing\n",
"\n",
"[Output parsers](/docs/modules/model_io/output_parsers/) are classes that help structure language model responses. \n",
"Suppose we want to tune the behavior of our extractor. There are a few options available. For example, if we want to redact names but retain other information, we could adjust the system prompt:"
"Just as a above, let's parse a generation based on a Pydantic data class."
"runnable.invoke({\"input\": inp})"
]
},
{
"cell_type": "markdown",
"id": "be611688-1224-4d5a-9e34-a158b3c04296",
"metadata": {},
"source": [
"Few-shot examples are another, effective way to illustrate intended behavior. For instance, if we want to redact names with a specific character string, a one-shot example will convey this. We can use a `FewShotChatMessagePromptTemplate` to easily accommodate both a fixed set of examples as well as the dynamic selection of examples based on the input."
"Here, the [LangSmith trace](https://smith.langchain.com/public/6fe5e694-9c04-48f7-83ff-e541da764781/r) for the chat model call shows how the one-shot example is formatted into the prompt.\n",
"[Output parsers](/docs/modules/model_io/output_parsers/) are classes that help structure language model responses. \n",
"\n",
"As shown above, they are used to parse the output of the runnable created by `create_structured_output_runnable`.\n",
"\n",
"They can also be used more generally, if a LLM is instructed to emit its output in a certain format. Parsers include convenience methods for generating formatting instructions for use in prompts.\n",
"We can see from the [LangSmith trace](https://smith.langchain.com/public/8e3aa858-467e-46a5-aa49-5db65f0a2b9a/r) that we get the same output as above.\n",
"We can see from the [LangSmith trace](https://smith.langchain.com/public/aec42dd3-d471-4d34-801b-20dd88444931/r) that we get the same output as above.\n",
"As we can see, we get an output of the `Joke` class, which respects our originally desired schema: 'setup' and 'punchline'.\n",
"\n",
"We can look at the [LangSmith trace](https://smith.langchain.com/public/69f11d41-41be-4319-93b0-6d0eda66e969/r) to see exactly what is going on under the hood.\n",
"We can look at the [LangSmith trace](https://smith.langchain.com/public/557ad630-af35-43e9-b043-93800539025f/r) to see exactly what is going on under the hood.\n",
ccurme
4 months ago
committed by
GitHub
