docs: `ElasticsearchCache` in cache integrations documentation (#20790)

The package for LangChain integrations with Elasticsearch
https://github.com/langchain-ai/langchain-elastic is going to contain a
LLM cache integration in the next release (see
https://github.com/langchain-ai/langchain-elastic/pull/14). This is the
documentation contribution on the page dedicated to cache integrations
pull/20254/head
Giacomo Berardi 3 weeks ago committed by GitHub
parent 8c085fc697
commit 61f14f00d7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

@ -1428,6 +1428,15 @@
},
{
"cell_type": "code",
"execution_count": 82,
"id": "14ca942820e8140c",
"metadata": {
"ExecuteTime": {
"end_time": "2024-03-12T00:12:57.462226Z",
"start_time": "2024-03-12T00:12:55.166201Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
@ -1439,7 +1448,9 @@
},
{
"data": {
"text/plain": "'\\n\\nWhy was the math book sad? Because it had too many problems.'"
"text/plain": [
"'\\n\\nWhy was the math book sad? Because it had too many problems.'"
]
},
"execution_count": 82,
"metadata": {},
@ -1450,16 +1461,7 @@
"%%time\n",
"# The first time, it is not yet in cache, so it should take longer\n",
"llm(\"Tell me a joke\")"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-03-12T00:12:57.462226Z",
"start_time": "2024-03-12T00:12:55.166201Z"
}
},
"id": "14ca942820e8140c",
"execution_count": 82
]
},
{
"cell_type": "code",
@ -1482,7 +1484,9 @@
},
{
"data": {
"text/plain": "'\\n\\nWhy was the math book sad? Because it had too many problems.'"
"text/plain": [
"'\\n\\nWhy was the math book sad? Because it had too many problems.'"
]
},
"execution_count": 83,
"metadata": {},
@ -1495,6 +1499,141 @@
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "markdown",
"id": "306ff47b",
"metadata": {},
"source": [
"## `Elasticsearch` Cache\n",
"A caching layer for LLMs that uses Elasticsearch.\n",
"\n",
"First install the LangChain integration with Elasticsearch."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9ee5cd3e",
"metadata": {},
"outputs": [],
"source": [
"%pip install -U langchain-elasticsearch"
]
},
{
"cell_type": "markdown",
"id": "9e70b0a0",
"metadata": {},
"source": [
"Use the class `ElasticsearchCache`.\n",
"\n",
"Simple example:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1762c9c1",
"metadata": {},
"outputs": [],
"source": [
"from elasticsearch import Elasticsearch\n",
"from langchain.globals import set_llm_cache\n",
"from langchain_elasticsearch import ElasticsearchCache\n",
"\n",
"es_client = Elasticsearch(hosts=\"http://localhost:9200\")\n",
"set_llm_cache(\n",
" ElasticsearchCache(\n",
" es_connection=es_client,\n",
" index_name=\"llm-chat-cache\",\n",
" metadata={\"project\": \"my_chatgpt_project\"},\n",
" )\n",
")"
]
},
{
"cell_type": "markdown",
"id": "d4fac5d6",
"metadata": {},
"source": [
"The `index_name` parameter can also accept aliases. This allows to use the \n",
"[ILM: Manage the index lifecycle](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html)\n",
"that we suggest to consider for managing retention and controlling cache growth.\n",
"\n",
"Look at the class docstring for all parameters."
]
},
{
"cell_type": "markdown",
"id": "eaf9dfd7",
"metadata": {},
"source": [
"### Index the generated text\n",
"\n",
"The cached data won't be searchable by default.\n",
"The developer can customize the building of the Elasticsearch document in order to add indexed text fields,\n",
"where to put, for example, the text generated by the LLM.\n",
"\n",
"This can be done by subclassing end overriding methods.\n",
"The new cache class can be applied also to a pre-existing cache index:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5104c2c0",
"metadata": {},
"outputs": [],
"source": [
"import json\n",
"from typing import Any, Dict, List\n",
"\n",
"from elasticsearch import Elasticsearch\n",
"from langchain.globals import set_llm_cache\n",
"from langchain_core.caches import RETURN_VAL_TYPE\n",
"from langchain_elasticsearch import ElasticsearchCache\n",
"\n",
"\n",
"class SearchableElasticsearchCache(ElasticsearchCache):\n",
" @property\n",
" def mapping(self) -> Dict[str, Any]:\n",
" mapping = super().mapping\n",
" mapping[\"mappings\"][\"properties\"][\"parsed_llm_output\"] = {\n",
" \"type\": \"text\",\n",
" \"analyzer\": \"english\",\n",
" }\n",
" return mapping\n",
"\n",
" def build_document(\n",
" self, prompt: str, llm_string: str, return_val: RETURN_VAL_TYPE\n",
" ) -> Dict[str, Any]:\n",
" body = super().build_document(prompt, llm_string, return_val)\n",
" body[\"parsed_llm_output\"] = self._parse_output(body[\"llm_output\"])\n",
" return body\n",
"\n",
" @staticmethod\n",
" def _parse_output(data: List[str]) -> List[str]:\n",
" return [\n",
" json.loads(output)[\"kwargs\"][\"message\"][\"kwargs\"][\"content\"]\n",
" for output in data\n",
" ]\n",
"\n",
"\n",
"es_client = Elasticsearch(hosts=\"http://localhost:9200\")\n",
"set_llm_cache(\n",
" SearchableElasticsearchCache(es_connection=es_client, index_name=\"llm-chat-cache\")\n",
")"
]
},
{
"cell_type": "markdown",
"id": "db0dea73",
"metadata": {},
"source": [
"When overriding the mapping and the document building, \n",
"please only make additive modifications, keeping the base mapping intact."
]
},
{
"cell_type": "markdown",
"id": "0c69d84d",

Loading…
Cancel
Save