From 61f14f00d79762598f8920595ecae0c3be338348 Mon Sep 17 00:00:00 2001 From: Giacomo Berardi Date: Sat, 27 Apr 2024 01:43:58 +0300 Subject: [PATCH] docs: `ElasticsearchCache` in cache integrations documentation (#20790) The package for LangChain integrations with Elasticsearch https://github.com/langchain-ai/langchain-elastic is going to contain a LLM cache integration in the next release (see https://github.com/langchain-ai/langchain-elastic/pull/14). This is the documentation contribution on the page dedicated to cache integrations --- docs/docs/integrations/llms/llm_caching.ipynb | 163 ++++++++++++++++-- 1 file changed, 151 insertions(+), 12 deletions(-) diff --git a/docs/docs/integrations/llms/llm_caching.ipynb b/docs/docs/integrations/llms/llm_caching.ipynb index 504a771ea2..55d557b8b0 100644 --- a/docs/docs/integrations/llms/llm_caching.ipynb +++ b/docs/docs/integrations/llms/llm_caching.ipynb @@ -1428,6 +1428,15 @@ }, { "cell_type": "code", + "execution_count": 82, + "id": "14ca942820e8140c", + "metadata": { + "ExecuteTime": { + "end_time": "2024-03-12T00:12:57.462226Z", + "start_time": "2024-03-12T00:12:55.166201Z" + }, + "collapsed": false + }, "outputs": [ { "name": "stdout", @@ -1439,7 +1448,9 @@ }, { "data": { - "text/plain": "'\\n\\nWhy was the math book sad? Because it had too many problems.'" + "text/plain": [ + "'\\n\\nWhy was the math book sad? Because it had too many problems.'" + ] }, "execution_count": 82, "metadata": {}, @@ -1450,16 +1461,7 @@ "%%time\n", "# The first time, it is not yet in cache, so it should take longer\n", "llm(\"Tell me a joke\")" - ], - "metadata": { - "collapsed": false, - "ExecuteTime": { - "end_time": "2024-03-12T00:12:57.462226Z", - "start_time": "2024-03-12T00:12:55.166201Z" - } - }, - "id": "14ca942820e8140c", - "execution_count": 82 + ] }, { "cell_type": "code", @@ -1482,7 +1484,9 @@ }, { "data": { - "text/plain": "'\\n\\nWhy was the math book sad? Because it had too many problems.'" + "text/plain": [ + "'\\n\\nWhy was the math book sad? Because it had too many problems.'" + ] }, "execution_count": 83, "metadata": {}, @@ -1495,6 +1499,141 @@ "llm(\"Tell me a joke\")" ] }, + { + "cell_type": "markdown", + "id": "306ff47b", + "metadata": {}, + "source": [ + "## `Elasticsearch` Cache\n", + "A caching layer for LLMs that uses Elasticsearch.\n", + "\n", + "First install the LangChain integration with Elasticsearch." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9ee5cd3e", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -U langchain-elasticsearch" + ] + }, + { + "cell_type": "markdown", + "id": "9e70b0a0", + "metadata": {}, + "source": [ + "Use the class `ElasticsearchCache`.\n", + "\n", + "Simple example:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1762c9c1", + "metadata": {}, + "outputs": [], + "source": [ + "from elasticsearch import Elasticsearch\n", + "from langchain.globals import set_llm_cache\n", + "from langchain_elasticsearch import ElasticsearchCache\n", + "\n", + "es_client = Elasticsearch(hosts=\"http://localhost:9200\")\n", + "set_llm_cache(\n", + " ElasticsearchCache(\n", + " es_connection=es_client,\n", + " index_name=\"llm-chat-cache\",\n", + " metadata={\"project\": \"my_chatgpt_project\"},\n", + " )\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "d4fac5d6", + "metadata": {}, + "source": [ + "The `index_name` parameter can also accept aliases. This allows to use the \n", + "[ILM: Manage the index lifecycle](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html)\n", + "that we suggest to consider for managing retention and controlling cache growth.\n", + "\n", + "Look at the class docstring for all parameters." + ] + }, + { + "cell_type": "markdown", + "id": "eaf9dfd7", + "metadata": {}, + "source": [ + "### Index the generated text\n", + "\n", + "The cached data won't be searchable by default.\n", + "The developer can customize the building of the Elasticsearch document in order to add indexed text fields,\n", + "where to put, for example, the text generated by the LLM.\n", + "\n", + "This can be done by subclassing end overriding methods.\n", + "The new cache class can be applied also to a pre-existing cache index:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5104c2c0", + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "from typing import Any, Dict, List\n", + "\n", + "from elasticsearch import Elasticsearch\n", + "from langchain.globals import set_llm_cache\n", + "from langchain_core.caches import RETURN_VAL_TYPE\n", + "from langchain_elasticsearch import ElasticsearchCache\n", + "\n", + "\n", + "class SearchableElasticsearchCache(ElasticsearchCache):\n", + " @property\n", + " def mapping(self) -> Dict[str, Any]:\n", + " mapping = super().mapping\n", + " mapping[\"mappings\"][\"properties\"][\"parsed_llm_output\"] = {\n", + " \"type\": \"text\",\n", + " \"analyzer\": \"english\",\n", + " }\n", + " return mapping\n", + "\n", + " def build_document(\n", + " self, prompt: str, llm_string: str, return_val: RETURN_VAL_TYPE\n", + " ) -> Dict[str, Any]:\n", + " body = super().build_document(prompt, llm_string, return_val)\n", + " body[\"parsed_llm_output\"] = self._parse_output(body[\"llm_output\"])\n", + " return body\n", + "\n", + " @staticmethod\n", + " def _parse_output(data: List[str]) -> List[str]:\n", + " return [\n", + " json.loads(output)[\"kwargs\"][\"message\"][\"kwargs\"][\"content\"]\n", + " for output in data\n", + " ]\n", + "\n", + "\n", + "es_client = Elasticsearch(hosts=\"http://localhost:9200\")\n", + "set_llm_cache(\n", + " SearchableElasticsearchCache(es_connection=es_client, index_name=\"llm-chat-cache\")\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "db0dea73", + "metadata": {}, + "source": [ + "When overriding the mapping and the document building, \n", + "please only make additive modifications, keeping the base mapping intact." + ] + }, { "cell_type": "markdown", "id": "0c69d84d",