- Reordered sections
- Applied consistent formatting
- Fixed headers (there were 2 H1 headers; this breaks CoT)
- Added `Settings` header and moved all related sections under it
"Redis vector database introduction and langchain integration guide.\n",
">[Redis vector database](https://redis.io/docs/get-started/vector-database/) introduction and langchain integration guide.\n",
"\n",
"## What is Redis?\n",
"\n",
"Most developers from a web services background are probably familiar with Redis. At it's core, Redis is an open-source key-value store that can be used as a cache, message broker, and database. Developers choose Redis because it is fast, has a large ecosystem of client libraries, and has been deployed by major enterprises for years.\n",
"Most developers from a web services background are familiar with `Redis`. At its core, `Redis` is an open-source key-value store that is used as a cache, message broker, and database. Developers choose `Redis` because it is fast, has a large ecosystem of client libraries, and has been deployed by major enterprises for years.\n",
"\n",
"On top of these traditional use cases, Redis provides additional capabilities like the Search and Query capability that allows users to create secondary index structures within Redis. This allows Redis to be a Vector Database, at the speed of a cache. \n",
"On top of these traditional use cases, `Redis` provides additional capabilities like the Search and Query capability that allows users to create secondary index structures within `Redis`. This allows `Redis` to be a Vector Database, at the speed of a cache. \n",
"\n",
"\n",
"## Redis as a Vector Database\n",
"\n",
"Redis uses compressed, inverted indexes for fast indexing with a low memory footprint. It also supports a number of advanced features such as:\n",
"`Redis` uses compressed, inverted indexes for fast indexing with a low memory footprint. It also supports a number of advanced features such as:\n",
"\n",
"* Indexing of multiple fields in Redis hashes and JSON\n",
"* Vector similarity search (with HNSW (ANN) or FLAT (KNN))\n",
"* Indexing of multiple fields in Redis hashes and `JSON`\n",
"* Vector similarity search (with `HNSW` (ANN) or `FLAT` (KNN))\n",
"* Vector Range Search (e.g. find all vectors within a radius of a query vector)\n",
"* Incremental indexing without performance loss\n",
"* Document ranking (using [tf-idf](https://en.wikipedia.org/wiki/Tf%E2%80%93idf), with optional user-provided weights)\n",
"* Field weighting\n",
"* Complex boolean queries with AND, OR, and NOT operators\n",
"* Complex boolean queries with `AND`, `OR`, and `NOT` operators\n",
"* Prefix matching, fuzzy matching, and exact-phrase queries\n",
"* Support for [double-metaphone phonetic matching](https://redis.io/docs/stack/search/reference/phonetic_matching/)\n",
"* Geospatial searches using Redis geospatial indexing\n",
"* A powerful aggregations engine\n",
"* Supports for all utf-8 encoded text\n",
"* Supports for all `utf-8` encoded text\n",
"* Retrieve full documents, selected fields, or only the document IDs\n",
"* Sorting results (for example, by creation date)\n",
"\n",
@ -42,7 +42,7 @@
"\n",
"## Clients\n",
"\n",
"Since redis is much more than just a vector database, there are often use cases that demand usage of a Redis client besides just the langchain integration. You can use any standard Redis client library to run Search and Query commands, but it's easiest to use a library that wraps the Search and Query API. Below are a few examples, but you can find more client libraries [here](https://redis.io/resources/clients/).\n",
"Since `Redis` is much more than just a vector database, there are often use cases that demand the usage of a `Redis` client besides just the `LangChain` integration. You can use any standard `Redis` client library to run Search and Query commands, but it's easiest to use a library that wraps the Search and Query API. Below are a few examples, but you can find more client libraries [here](https://redis.io/resources/clients/).\n",
"There are many ways to deploy Redis with RediSearch. The easiest way to get started is to use Docker, but there are are many potential options for deployment such as\n",
"\n",
@ -98,7 +98,7 @@
"- Kubernetes: [Redis Enterprise Software on Kubernetes](https://docs.redis.com/latest/kubernetes/)\n",
"\n",
"\n",
"## Examples\n",
"## Additional examples\n",
"\n",
"Many examples can be found in the [Redis AI team's GitHub](https://github.com/RedisVentures/)\n",
"\n",
@ -108,7 +108,7 @@
"- [Vector Search on Azure](https://learn.microsoft.com/azure/azure-cache-for-redis/cache-tutorial-vector-similarity) - Vector search on Azure using Azure Cache for Redis and Azure OpenAI\n",
"\n",
"\n",
"## More Resources\n",
"## More resources\n",
"\n",
"For more information on how to use Redis as a vector database, check out the following resources:\n",
"\n",
@ -122,9 +122,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Install Redis Python Client\n",
"## Setting up\n",
"\n",
"Redis-py is the officially supported client by Redis. Recently released is the RedisVL client which is purpose-built for the Vector Database use cases. Both can be installed with pip."
"\n",
"### Install Redis Python client\n",
"\n",
"`Redis-py` is the officially supported client by Redis. Recently released is the `RedisVL` client which is purpose-built for the Vector Database use cases. Both can be installed with pip."
]
},
{
@ -172,7 +175,63 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Sample Data\n",
"### Deploy Redis locally\n",
"\n",
"To locally deploy Redis, run:\n",
"\n",
"```console\n",
"docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest\n",
"```\n",
"If things are running correctly you should see a nice Redis UI at `http://localhost:8001`. See the [Deployment options](#deployment-options) section above for other ways to deploy.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Redis connection Url schemas\n",
"\n",
"Valid Redis Url schemas are:\n",
"1. `redis://` - Connection to Redis standalone, unencrypted\n",
"2. `rediss://` - Connection to Redis standalone, with TLS encryption\n",
"3. `redis+sentinel://` - Connection to Redis server via Redis Sentinel, unencrypted\n",
"4. `rediss+sentinel://` - Connection to Redis server via Redis Sentinel, booth connections with TLS encryption\n",
"\n",
"More information about additional connection parameters can be found in the [redis-py documentation](https://redis-py.readthedocs.io/en/stable/connections.html)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# connection to redis standalone at localhost, db 0, no password\n",
"redis_url = \"redis://localhost:6379\"\n",
"# connection to host \"redis\" port 7379 with db 2 and password \"secret\" (old style authentication scheme without username / pre 6.x)\n",
"redis_url = \"redis://:secret@redis:7379/2\"\n",
"# connection to host redis on default port with user \"joe\", pass \"secret\" using redis version 6+ ACLs\n",
"redis_url = \"redis://joe:secret@redis/0\"\n",
"\n",
"# connection to sentinel at localhost with default group mymaster and db 0, no password\n",
"# connection to redis standalone at localhost, db 0, no password but with TLS support\n",
"redis_url = \"rediss://localhost:6379\"\n",
"# connection to redis sentinel at localhost and default port, db 0, no password\n",
"# but with TLS support for booth Sentinel and Redis server\n",
"redis_url = \"rediss+sentinel://localhost\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Sample data\n",
"\n",
"First we will describe some sample data so that the various attributes of the Redis vector store can be demonstrated."
]
@ -222,13 +281,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initializing Redis\n",
"\n",
"To locally deploy Redis, run:\n",
"```console\n",
"docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest\n",
"```\n",
"If things are running correctly you should see a nice Redis UI at http://localhost:8001. See the [Deployment Options](#deployment-options) section above for other ways to deploy.\n",
"### Create Redis vector store\n",
"\n",
"The Redis VectorStore instance can be initialized in a number of ways. There are multiple class methods that can be used to initialize a Redis VectorStore instance.\n",
"\n",
@ -284,7 +337,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Inspecting the Created Index\n",
"## Inspecting the created Index\n",
"\n",
"Once the ``Redis`` VectorStore object has been constructed, an index will have been created in Redis if it did not already exist. The index can be inspected with both the ``rvl``and the ``redis-cli`` command line tool. If you installed ``redisvl`` above, you can use the ``rvl`` command line tool to inspect the index."
]
@ -615,7 +668,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Connect to an Existing Index\n",
"## Connect to an existing Index\n",
"\n",
"In order to have the same metadata indexed when using the ``Redis`` VectorStore. You will need to have the same ``index_schema`` passed in either as a path to a yaml file or as a dictionary. The following shows how to obtain the schema from an index and connect to an existing index."
]
@ -730,9 +783,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Custom Metadata Indexing\n",
"## Custom metadata indexing\n",
"\n",
"In some cases, you may want to control what fields the metadata maps to. For example, you may want the ``credit_score`` field to be a categorical field instead of a text field (which is the default behavior for all string fields). In this case, you can use the ``index_schema`` parameter in each of the initialization methods above to specify the schema for the index. Custom index schema can either be passed as a dictionary or as a path to a yaml file.\n",
"In some cases, you may want to control what fields the metadata maps to. For example, you may want the ``credit_score`` field to be a categorical field instead of a text field (which is the default behavior for all string fields). In this case, you can use the ``index_schema`` parameter in each of the initialization methods above to specify the schema for the index. Custom index schema can either be passed as a dictionary or as a path to a YAML file.\n",
"\n",
"All arguments in the schema have defaults besides the name, so you can specify only the fields you want to change. All the names correspond to the snake/lowercase versions of the arguments you would use on the command line with ``redis-cli`` or in ``redis-py``. For more on the arguments for each field, see the [documentation](https://redis.io/docs/interact/search-and-query/basic-constructs/field-and-type-options/)\n",
"\n",
@ -749,7 +802,7 @@
" - name: age\n",
"```\n",
"\n",
"In Python this would look like:\n",
"In Python, this would look like:\n",
"\n",
"```python\n",
"\n",
@ -810,16 +863,16 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Hybrid Filtering\n",
"## Hybrid filtering\n",
"\n",
"With the Redis Filter Expression language built into langchain, you can create arbitrarily long chains of hybrid filters\n",
"With the Redis Filter Expression language built into LangChain, you can create arbitrarily long chains of hybrid filters\n",
"that can be used to filter your search results. The expression language is derived from the [RedisVL Expression Syntax](https://redisvl.com)\n",
"and is designed to be easy to use and understand.\n",
"\n",
"The following are the available filter types:\n",
"- ``RedisText``: Filter by full-text search against metadata fields. Supports exact, fuzzy, and wildcard matching.\n",
"- ``RedisNum``: Filter by numeric range against metadata fields.\n",
"- ``RedisTag``: Filter by exact match against string based categorical metadata fields. Multiple tags can be specified like \"tag1,tag2,tag3\".\n",
"- ``RedisTag``: Filter by the exact match against string-based categorical metadata fields. Multiple tags can be specified like \"tag1,tag2,tag3\".\n",
"\n",
"The following are examples of utilizing these filters.\n",
"The following are examples of using hybrid filter for search"
"The following are examples of using a hybrid filter for search"
]
},
{
@ -1150,7 +1203,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Delete keys"
"## Delete keys and index"
]
},
{
@ -1209,48 +1262,6 @@
" redis_url=\"redis://localhost:6379\",\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Redis connection Url examples\n",
"\n",
"Valid Redis Url scheme are:\n",
"1. `redis://` - Connection to Redis standalone, unencrypted\n",
"2. `rediss://` - Connection to Redis standalone, with TLS encryption\n",
"3. `redis+sentinel://` - Connection to Redis server via Redis Sentinel, unencrypted\n",
"4. `rediss+sentinel://` - Connection to Redis server via Redis Sentinel, booth connections with TLS encryption\n",
"\n",
"More information about additional connection parameter can be found in the redis-py documentation at https://redis-py.readthedocs.io/en/stable/connections.html"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [],
"source": [
"# connection to redis standalone at localhost, db 0, no password\n",
"redis_url = \"redis://localhost:6379\"\n",
"# connection to host \"redis\" port 7379 with db 2 and password \"secret\" (old style authentication scheme without username / pre 6.x)\n",
"redis_url = \"redis://:secret@redis:7379/2\"\n",
"# connection to host redis on default port with user \"joe\", pass \"secret\" using redis version 6+ ACLs\n",
"redis_url = \"redis://joe:secret@redis/0\"\n",
"\n",
"# connection to sentinel at localhost with default group mymaster and db 0, no password\n",
Leonid Ganeline
8 months ago
committed by
GitHub