2023-08-10 15:15:30 +00:00
{
"cells": [
{
"cell_type": "markdown",
"id": "bf4061ce",
"metadata": {},
"source": [
"# Caching Embeddings\n",
"\n",
"Embeddings can be stored or temporarily cached to avoid needing to recompute them.\n",
"\n",
2023-08-10 19:30:29 +00:00
"Caching embeddings can be done using a `CacheBackedEmbeddings`.\n",
2023-08-10 15:15:30 +00:00
"\n",
"The cache backed embedder is a wrapper around an embedder that caches\n",
"embeddings in a key-value store. \n",
"\n",
"The text is hashed and the hash is used as the key in the cache.\n",
"\n",
"\n",
2023-08-10 19:30:29 +00:00
"The main supported way to initialized a `CacheBackedEmbeddings` is `from_bytes_store`. This takes in the following parameters:\n",
2023-08-10 15:15:30 +00:00
"\n",
"- underlying_embedder: The embedder to use for embedding.\n",
"- document_embedding_cache: The cache to use for storing document embeddings.\n",
"- namespace: (optional, defaults to `\"\"`) The namespace to use for document cache. This namespace is used to avoid collisions with other caches. For example, set it to the name of the embedding model used.\n",
"\n",
"**Attention**: Be sure to set the `namespace` parameter to avoid collisions of the same text embedded using different embeddings models."
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 4,
2023-08-10 15:15:30 +00:00
"id": "a463c3c2-749b-40d1-a433-84f68a1cd1c7",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
2023-08-10 19:30:29 +00:00
"from langchain.storage import InMemoryStore, LocalFileStore, RedisStore\n",
"from langchain.embeddings import OpenAIEmbeddings, CacheBackedEmbeddings"
]
},
{
"cell_type": "markdown",
"id": "9ddf07dd-3e72-41de-99d4-78e9521e272f",
"metadata": {},
"source": [
"## Using with a vectorstore\n",
"\n",
"First, let's see an example that uses the local file system for storing embeddings and uses FAISS vectorstore for retrieval."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "9e4314d8-88ef-4f52-81ae-0be771168bb6",
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import TextLoader\n",
"from langchain.embeddings.openai import OpenAIEmbeddings\n",
"from langchain.text_splitter import CharacterTextSplitter\n",
"from langchain.vectorstores import FAISS"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "3e751f26-9b5b-4c10-843a-d784b5ea8538",
"metadata": {},
"outputs": [],
"source": [
"underlying_embeddings = OpenAIEmbeddings()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "30743664-38f5-425d-8216-772b64e7f348",
"metadata": {},
"outputs": [],
"source": [
"fs = LocalFileStore(\"./cache/\")\n",
"\n",
"cached_embedder = CacheBackedEmbeddings.from_bytes_store(\n",
" underlying_embeddings, fs, namespace=underlying_embeddings.model\n",
")"
]
},
{
"cell_type": "markdown",
"id": "f8cdf33c-321d-4d2c-b76b-d6f5f8b42a92",
"metadata": {},
"source": [
"The cache is empty prior to embedding"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "f9ad627f-ced2-4277-b336-2434f22f2c8a",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[]"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(fs.yield_keys())"
]
},
{
"cell_type": "markdown",
"id": "a4effe04-b40f-42f8-a449-72fe6991cf20",
"metadata": {},
"source": [
"Load the document, split it into chunks, embed each chunk and load it into the vector store."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "cf958ac2-e60e-4668-b32c-8bb2d78b3c61",
"metadata": {},
"outputs": [],
"source": [
"raw_documents = TextLoader(\"../state_of_the_union.txt\").load()\n",
"text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
"documents = text_splitter.split_documents(raw_documents)"
]
},
{
"cell_type": "markdown",
"id": "f526444b-93f8-423f-b6d1-dab539450921",
"metadata": {},
"source": [
"create the vectorstore"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "3a1d7bb8-3b72-4bb5-9013-cf7729caca61",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 608 ms, sys: 58.9 ms, total: 667 ms\n",
"Wall time: 1.3 s\n"
]
}
],
"source": [
"%%time\n",
"db = FAISS.from_documents(documents, cached_embedder)"
]
},
{
"cell_type": "markdown",
"id": "64fc53f5-d559-467f-bf62-5daef32ffbc0",
"metadata": {},
"source": [
"If we try to create the vectostore again, it'll be much faster since it does not need to re-compute any embeddings."
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "714cb2e2-77ba-41a8-bb83-84e75342af2d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 33.6 ms, sys: 3.96 ms, total: 37.6 ms\n",
"Wall time: 36.8 ms\n"
]
}
],
"source": [
"%%time\n",
"db2 = FAISS.from_documents(documents, cached_embedder)"
]
},
{
"cell_type": "markdown",
"id": "1acc76b9-9c70-4160-b593-5f932c75e2b6",
"metadata": {},
"source": [
"And here are some of the embeddings that got created:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "f2ca32dd-3712-4093-942b-4122f3dc8a8e",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['text-embedding-ada-002614d7cf6-46f1-52fa-9d3a-740c39e7a20e',\n",
" 'text-embedding-ada-0020fc1ede2-407a-5e14-8f8f-5642214263f5',\n",
" 'text-embedding-ada-002e4ad20ef-dfaa-5916-9459-f90c6d8e8159',\n",
" 'text-embedding-ada-002a5ef11e4-0474-5725-8d80-81c91943b37f',\n",
" 'text-embedding-ada-00281426526-23fe-58be-9e84-6c7c72c8ca9a']"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(fs.yield_keys())[:5]"
2023-08-10 15:15:30 +00:00
]
},
{
"cell_type": "markdown",
"id": "564c9801-29f0-4452-aeac-527382e2c0e8",
"metadata": {},
"source": [
"## In Memory\n",
"\n",
"This section shows how to set up an in memory cache for embeddings. This type of cache is primarily \n",
"useful for unit tests or prototyping. Do **not** use this cache if you need to actually store the embeddings."
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 14,
2023-08-10 15:15:30 +00:00
"id": "13bd1c5b-b7ba-4394-957c-7d5b5a841972",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"store = InMemoryStore()"
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 15,
2023-08-10 15:15:30 +00:00
"id": "9d99885f-99e1-498c-904d-6db539ac9466",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
2023-08-10 19:30:29 +00:00
"underlying_embeddings = OpenAIEmbeddings()\n",
"embedder = CacheBackedEmbeddings.from_bytes_store(\n",
" underlying_embeddings, store, namespace=underlying_embeddings.model\n",
2023-08-10 15:15:30 +00:00
")"
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 16,
2023-08-10 15:15:30 +00:00
"id": "682eb5d4-0b7a-4dac-b8fb-3de4ca6e421c",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2023-08-10 19:30:29 +00:00
"CPU times: user 10.9 ms, sys: 916 µs, total: 11.8 ms\n",
"Wall time: 159 ms\n"
2023-08-10 15:15:30 +00:00
]
}
],
"source": [
"%%time\n",
"embeddings = embedder.embed_documents([\"hello\", \"goodbye\"])"
]
},
{
"cell_type": "markdown",
"id": "95233026-147f-49d1-bd87-e1e8b88ebdbc",
"metadata": {},
"source": [
"The second time we try to embed the embedding time is only 2 ms because the embeddings are looked up in the cache."
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 17,
2023-08-10 15:15:30 +00:00
"id": "f819c3ff-a212-4d06-a5f7-5eb1435c1feb",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2023-08-10 19:30:29 +00:00
"CPU times: user 1.67 ms, sys: 342 µs, total: 2.01 ms\n",
"Wall time: 2.01 ms\n"
2023-08-10 15:15:30 +00:00
]
}
],
"source": [
"%%time\n",
"embeddings_from_cache = embedder.embed_documents([\"hello\", \"goodbye\"])"
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 18,
2023-08-10 15:15:30 +00:00
"id": "ec38fb72-90a9-4687-a483-c62c87d1f4dd",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
2023-08-10 19:30:29 +00:00
"execution_count": 18,
2023-08-10 15:15:30 +00:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"embeddings == embeddings_from_cache"
]
},
{
"cell_type": "markdown",
"id": "f6cbe100-8587-4830-b207-fb8b524a9854",
"metadata": {},
"source": [
"## File system\n",
"\n",
2023-08-10 19:30:29 +00:00
"This section covers how to use a file system store."
2023-08-10 15:15:30 +00:00
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 19,
2023-08-10 15:15:30 +00:00
"id": "a0070271-0809-4528-97e0-2a88216846f3",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
2023-08-10 19:30:29 +00:00
"fs = LocalFileStore(\"./test_cache/\")"
2023-08-10 15:15:30 +00:00
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 20,
2023-08-10 15:15:30 +00:00
"id": "0b20e9fe-f57f-4d7c-9f81-105c5f8726f4",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
2023-08-10 19:30:29 +00:00
"embedder2 = CacheBackedEmbeddings.from_bytes_store(\n",
" underlying_embeddings, fs, namespace=underlying_embeddings.model\n",
2023-08-10 15:15:30 +00:00
")"
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 21,
2023-08-10 15:15:30 +00:00
"id": "630515fd-bf5c-4d9c-a404-9705308f3a2c",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2023-08-10 19:30:29 +00:00
"CPU times: user 6.89 ms, sys: 4.89 ms, total: 11.8 ms\n",
"Wall time: 184 ms\n"
2023-08-10 15:15:30 +00:00
]
}
],
"source": [
"%%time\n",
"embeddings = embedder2.embed_documents([\"hello\", \"goodbye\"])"
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 22,
2023-08-10 15:15:30 +00:00
"id": "30e6bb87-42c9-4d08-88ac-0d22c9c449a1",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2023-08-10 19:30:29 +00:00
"CPU times: user 0 ns, sys: 3.24 ms, total: 3.24 ms\n",
"Wall time: 2.84 ms\n"
2023-08-10 15:15:30 +00:00
]
}
],
"source": [
"%%time\n",
"embeddings = embedder2.embed_documents([\"hello\", \"goodbye\"])"
]
},
{
"cell_type": "markdown",
"id": "12ed5a45-8352-4e0f-8583-5537397f53c0",
"metadata": {},
"source": [
2023-08-10 19:30:29 +00:00
"Here are the embeddings that have been persisted to the directory `./test_cache`. \n",
2023-08-10 15:15:30 +00:00
"\n",
"Notice that the embedder takes a namespace parameter."
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 23,
2023-08-10 15:15:30 +00:00
"id": "658e2914-05e9-44a3-a8fe-3fe17ca84039",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['text-embedding-ada-002e885db5b-c0bd-5fbc-88b1-4d1da6020aa5',\n",
" 'text-embedding-ada-0026ba52e44-59c9-5cc9-a084-284061b13c80']"
]
},
2023-08-10 19:30:29 +00:00
"execution_count": 23,
2023-08-10 15:15:30 +00:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(fs.yield_keys())"
]
},
{
"cell_type": "markdown",
2023-08-10 19:30:29 +00:00
"id": "cd5f5a96-6ffa-429d-aa82-00b3f6532871",
2023-08-10 15:15:30 +00:00
"metadata": {},
"source": [
2023-08-10 19:30:29 +00:00
"## Redis Store\n",
"\n"
2023-08-10 15:15:30 +00:00
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 24,
"id": "4879c134-141f-48a0-acfe-7d6f30253af0",
2023-08-10 15:15:30 +00:00
"metadata": {},
"outputs": [],
"source": [
2023-08-10 19:30:29 +00:00
"from langchain.storage import RedisStore"
2023-08-10 15:15:30 +00:00
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 25,
"id": "8b2bb9a0-6549-4487-8532-29ab4ab7336f",
2023-08-10 15:15:30 +00:00
"metadata": {},
"outputs": [],
"source": [
2023-08-10 19:30:29 +00:00
"# For cache isolation can use a separate DB\n",
"# Or additional namepace\n",
"store = RedisStore(redis_url=\"redis://localhost:6379\", client_kwargs={'db': 2}, namespace='embedding_caches')\n",
2023-08-10 15:15:30 +00:00
"\n",
2023-08-10 19:30:29 +00:00
"underlying_embeddings = OpenAIEmbeddings()\n",
"embedder = CacheBackedEmbeddings.from_bytes_store(\n",
" underlying_embeddings, store, namespace=underlying_embeddings.model\n",
2023-08-10 15:15:30 +00:00
")"
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 26,
"id": "eca3cb99-2bb3-49d5-81f9-1dee03da4b8c",
2023-08-10 15:15:30 +00:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2023-08-10 19:30:29 +00:00
"CPU times: user 3.99 ms, sys: 0 ns, total: 3.99 ms\n",
"Wall time: 3.5 ms\n"
2023-08-10 15:15:30 +00:00
]
}
],
"source": [
"%%time\n",
2023-08-10 19:30:29 +00:00
"embeddings = embedder.embed_documents([\"hello\", \"goodbye\"])"
2023-08-10 15:15:30 +00:00
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 27,
"id": "317ba5d8-89f9-462c-b807-ad4ef26e518b",
2023-08-10 15:15:30 +00:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2023-08-10 19:30:29 +00:00
"CPU times: user 2.47 ms, sys: 767 µs, total: 3.24 ms\n",
"Wall time: 2.75 ms\n"
2023-08-10 15:15:30 +00:00
]
}
],
"source": [
"%%time\n",
2023-08-10 19:30:29 +00:00
"embeddings = embedder.embed_documents([\"hello\", \"goodbye\"])"
2023-08-10 15:15:30 +00:00
]
},
{
2023-08-10 19:30:29 +00:00
"cell_type": "code",
"execution_count": 16,
"id": "8a540317-5142-4491-9062-a097932b56e3",
2023-08-10 15:15:30 +00:00
"metadata": {},
2023-08-10 19:30:29 +00:00
"outputs": [
{
"data": {
"text/plain": [
"['text-embedding-ada-002e885db5b-c0bd-5fbc-88b1-4d1da6020aa5',\n",
" 'text-embedding-ada-0026ba52e44-59c9-5cc9-a084-284061b13c80']"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
2023-08-10 15:15:30 +00:00
"source": [
2023-08-10 19:30:29 +00:00
"list(store.yield_keys())"
2023-08-10 15:15:30 +00:00
]
},
{
"cell_type": "code",
2023-08-10 19:30:29 +00:00
"execution_count": 17,
"id": "cd9b0d4a-f816-4dce-9dde-cde1ad9a65fb",
2023-08-10 15:15:30 +00:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2023-08-10 19:30:29 +00:00
"[b'embedding_caches/text-embedding-ada-002e885db5b-c0bd-5fbc-88b1-4d1da6020aa5',\n",
" b'embedding_caches/text-embedding-ada-0026ba52e44-59c9-5cc9-a084-284061b13c80']"
2023-08-10 15:15:30 +00:00
]
},
2023-08-10 19:30:29 +00:00
"execution_count": 17,
2023-08-10 15:15:30 +00:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2023-08-10 19:30:29 +00:00
"list(store.client.scan_iter())"
2023-08-10 15:15:30 +00:00
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}