mirror of
https://github.com/hwchase17/langchain
synced 2024-10-31 15:20:26 +00:00
331 lines
8.9 KiB
Plaintext
331 lines
8.9 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Redis\n",
|
||
"\n",
|
||
">[Redis (Remote Dictionary Server)](https://en.wikipedia.org/wiki/Redis) is an in-memory data structure store, used as a distributed, in-memory key–value database, cache and message broker, with optional durability.\n",
|
||
"\n",
|
||
"This notebook shows how to use functionality related to the [Redis vector database](https://redis.com/solutions/use-cases/vector-database/).\n",
|
||
"\n",
|
||
"As database either Redis standalone server or Redis Sentinel HA setups are supported for connections with the \"redis_url\"\n",
|
||
"parameter. More information about the different formats of the redis connection url can be found in the LangChain\n",
|
||
"[Redis Readme](/docs/integrations/vectorstores/redis) file"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Installing"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"!pip install redis"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"We want to use `OpenAIEmbeddings` so we have to get the OpenAI API Key."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import os\n",
|
||
"import getpass\n",
|
||
"\n",
|
||
"os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Example"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"from langchain.embeddings import OpenAIEmbeddings\n",
|
||
"from langchain.text_splitter import CharacterTextSplitter\n",
|
||
"from langchain.vectorstores.redis import Redis"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from langchain.document_loaders import TextLoader\n",
|
||
"\n",
|
||
"loader = TextLoader(\"../../../state_of_the_union.txt\")\n",
|
||
"documents = loader.load()\n",
|
||
"text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
|
||
"docs = text_splitter.split_documents(documents)\n",
|
||
"\n",
|
||
"embeddings = OpenAIEmbeddings()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"If you're not interested in the keys of your entries you can also create your redis instance from the documents."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"rds = Redis.from_documents(\n",
|
||
" docs, embeddings, redis_url=\"redis://localhost:6379\", index_name=\"link\"\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"If you're interested in the keys of your entries you have to split your docs in texts and metadatas"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"texts = [d.page_content for d in docs]\n",
|
||
"metadatas = [d.metadata for d in docs]\n",
|
||
"\n",
|
||
"rds, keys = Redis.from_texts_return_keys(\n",
|
||
" texts, embeddings, redis_url=\"redis://localhost:6379\", index_name=\"link\"\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"rds.index_name"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"query = \"What did the president say about Ketanji Brown Jackson\"\n",
|
||
"results = rds.similarity_search(query)\n",
|
||
"print(results[0].page_content)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"print(rds.add_texts([\"Ankush went to Princeton\"]))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"query = \"Princeton\"\n",
|
||
"results = rds.similarity_search(query)\n",
|
||
"print(results[0].page_content)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Load from existing index\n",
|
||
"rds = Redis.from_existing_index(\n",
|
||
" embeddings, redis_url=\"redis://localhost:6379\", index_name=\"link\"\n",
|
||
")\n",
|
||
"\n",
|
||
"query = \"What did the president say about Ketanji Brown Jackson\"\n",
|
||
"results = rds.similarity_search(query)\n",
|
||
"print(results[0].page_content)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Redis as Retriever\n",
|
||
"\n",
|
||
"Here we go over different options for using the vector store as a retriever.\n",
|
||
"\n",
|
||
"There are three different search methods we can use to do retrieval. By default, it will use semantic similarity."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"retriever = rds.as_retriever()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"docs = retriever.get_relevant_documents(query)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"We can also use similarity_limit as a search method. This is only return documents if they are similar enough"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"retriever = rds.as_retriever(search_type=\"similarity_limit\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Here we can see it doesn't return any results because there are no relevant documents\n",
|
||
"retriever.get_relevant_documents(\"where did ankush go to college?\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Delete keys"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"To delete your entries you have to address them by their keys."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"scrolled": true
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"Redis.delete(keys, redis_url=\"redis://localhost:6379\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"### Redis connection Url examples\n",
|
||
"\n",
|
||
"Valid Redis Url scheme are:\n",
|
||
"1. `redis://` - Connection to Redis standalone, unencrypted\n",
|
||
"2. `rediss://` - Connection to Redis standalone, with TLS encryption\n",
|
||
"3. `redis+sentinel://` - Connection to Redis server via Redis Sentinel, unencrypted\n",
|
||
"4. `rediss+sentinel://` - Connection to Redis server via Redis Sentinel, booth connections with TLS encryption\n",
|
||
"\n",
|
||
"More information about additional connection parameter can be found in the redis-py documentation at https://redis-py.readthedocs.io/en/stable/connections.html"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# connection to redis standalone at localhost, db 0, no password\n",
|
||
"redis_url = \"redis://localhost:6379\"\n",
|
||
"# connection to host \"redis\" port 7379 with db 2 and password \"secret\" (old style authentication scheme without username / pre 6.x)\n",
|
||
"redis_url = \"redis://:secret@redis:7379/2\"\n",
|
||
"# connection to host redis on default port with user \"joe\", pass \"secret\" using redis version 6+ ACLs\n",
|
||
"redis_url = \"redis://joe:secret@redis/0\"\n",
|
||
"\n",
|
||
"# connection to sentinel at localhost with default group mymaster and db 0, no password\n",
|
||
"redis_url = \"redis+sentinel://localhost:26379\"\n",
|
||
"# connection to sentinel at host redis with default port 26379 and user \"joe\" with password \"secret\" with default group mymaster and db 0\n",
|
||
"redis_url = \"redis+sentinel://joe:secret@redis\"\n",
|
||
"# connection to sentinel, no auth with sentinel monitoring group \"zone-1\" and database 2\n",
|
||
"redis_url = \"redis+sentinel://redis:26379/zone-1/2\"\n",
|
||
"\n",
|
||
"# connection to redis standalone at localhost, db 0, no password but with TLS support\n",
|
||
"redis_url = \"rediss://localhost:6379\"\n",
|
||
"# connection to redis sentinel at localhost and default port, db 0, no password\n",
|
||
"# but with TLS support for booth Sentinel and Redis server\n",
|
||
"redis_url = \"rediss+sentinel://localhost\""
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.11.3"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 4
|
||
}
|