"We can use [grammars](https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md) to constrain model outputs and sample tokens based on the rules defined in them.\n",
"\n",
"\n",
"We can specify [grammars](https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md) to constrain model outputs.\n",
"To demonstrate this concept, we've included [sample grammar files](https://github.com/langchain-ai/langchain/tree/master/libs/langchain/langchain/llms/grammars), that will be used in the examples below.\n",
"\n",
"\n",
"This will sample tokens according to the grammar.\n",
"Creating gbnf grammar files can be time-consuming, but if you have a use-case where output schemas are important, there are two tools that can help:\n",
" \n",
"- [Online grammar generator app](https://grammar.intrinsiclabs.ai/) that converts TypeScript interface definitions to gbnf file.\n",
"For example, supply the path to the specifed `json.gbnf` file in order to produce JSON."
"- [Python script](https://github.com/ggerganov/llama.cpp/blob/master/examples/json-schema-to-grammar.py) for converting json schema to gbnf file. You can for example create `pydantic` object, generate its JSON schema using `.schema_json()` method, and then use this script to convert it to gbnf file."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the first example, supply the path to the specifed `json.gbnf` file in order to produce JSON:"
]
]
},
},
{
{
@ -612,7 +621,7 @@
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {},
"metadata": {},
"source": [
"source": [
"We can also supply `list.gbnf` to return a list."
"We can also supply `list.gbnf` to return a list:"