Documentation Update for Upstash Semantic Caching (#25114)

Thank you for contributing to LangChain!

- [ ] **PR title**: "Documentation Update : Semantic Caching Update for
Upstash"
 - Docs, llm caching integrations update

- **Description:** Upstash supports semantic caching, and we would like
to inform you about this
- **Twitter handle:** You can mention eray_eroglu_ if you want to post a
tweet about the PR

---------

Co-authored-by: Chester Curme <chester.curme@gmail.com>
This commit is contained in:
Eray Eroğlu 2024-08-07 17:02:07 +03:00 committed by GitHub
parent 7e7fcf5b1f
commit a2e9910268
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -334,6 +334,121 @@
"llm.invoke(\"Tell me a joke\")"
]
},
{
"cell_type": "markdown",
"id": "b29dd776",
"metadata": {},
"source": [
"### Semantic Cache\n",
"Use [Upstash Vector](https://upstash.com/docs/vector/overall/whatisvector) to do a semantic similarity search and cache the most similar response in the database. The vectorization is automatically done by the selected embedding model while creating Upstash Vector database. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b37fb3c9",
"metadata": {},
"outputs": [],
"source": [
"%pip install upstash-semantic-cache"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "8470eedc",
"metadata": {},
"outputs": [],
"source": [
"from langchain.globals import set_llm_cache\n",
"from upstash_semantic_cache import SemanticCache"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "16b9fb03",
"metadata": {},
"outputs": [],
"source": [
"UPSTASH_VECTOR_REST_URL = \"<UPSTASH_VECTOR_REST_URL>\"\n",
"UPSTASH_VECTOR_REST_TOKEN = \"<UPSTASH_VECTOR_REST_TOKEN>\"\n",
"\n",
"cache = SemanticCache(\n",
" url=UPSTASH_VECTOR_REST_URL, token=UPSTASH_VECTOR_REST_TOKEN, min_proximity=0.7\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "8d37104b",
"metadata": {},
"outputs": [],
"source": [
"set_llm_cache(cache)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "926a08b3",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 28.4 ms, sys: 3.93 ms, total: 32.3 ms\n",
"Wall time: 1.89 s\n"
]
},
{
"data": {
"text/plain": [
"'\\n\\nNew York City is the most crowded city in the USA.'"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"llm.invoke(\"Which city is the most crowded city in the USA?\")"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "0ce37d57",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 3.22 ms, sys: 940 μs, total: 4.16 ms\n",
"Wall time: 97.7 ms\n"
]
},
{
"data": {
"text/plain": [
"'\\n\\nNew York City is the most crowded city in the USA.'"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"llm.invoke(\"Which city has the highest population in the USA?\")"
]
},
{
"cell_type": "markdown",
"id": "278ad7ae",
@ -2684,7 +2799,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.5"
"version": "3.12.3"
}
},
"nbformat": 4,