"Start a version of Redis with RediSearch (Redis Stack) by running the following docker command\n",
"\n",
"```bash\n",
"$ docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest\n",
"$ cd redis\n",
"$ docker compose up -d\n",
"```\n",
"This also includes the [RedisInsight](https://redis.com/redis-enterprise/redis-insight/) GUI for managing your Redis database which you can view at [http://localhost:8001](http://localhost:8001) once you start the docker container.\n",
"\n",
@ -1254,7 +1255,7 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 134,
"id": "d2ce669a",
"metadata": {},
"outputs": [
@ -1264,7 +1265,7 @@
"True"
]
},
"execution_count": 12,
"execution_count": 134,
"metadata": {},
"output_type": "execute_result"
}
@ -1311,7 +1312,7 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": 135,
"id": "a7c64cb9",
"metadata": {},
"outputs": [],
@ -1326,7 +1327,7 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": 136,
"id": "d95fcd06",
"metadata": {},
"outputs": [],
@ -1356,7 +1357,7 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 137,
"id": "7418480d",
"metadata": {},
"outputs": [],
@ -1386,7 +1387,7 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 138,
"id": "e98d63ad",
"metadata": {},
"outputs": [],
@ -1409,7 +1410,7 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": 139,
"id": "098d3c5a",
"metadata": {},
"outputs": [
@ -1442,7 +1443,7 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": 140,
"id": "508d1f89",
"metadata": {},
"outputs": [],
@ -1483,7 +1484,7 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": 142,
"id": "1f0eef07",
"metadata": {},
"outputs": [
@ -1494,25 +1495,25 @@
"0. Museum of Modern Art (Score: 0.875)\n",
"1. Western Europe (Score: 0.867)\n",
"2. Renaissance art (Score: 0.864)\n",
"3. Pop art (Score: 0.861)\n",
"3. Pop art (Score: 0.86)\n",
"4. Northern Europe (Score: 0.855)\n",
"5. Hellenistic art (Score: 0.853)\n",
"6. Modernist literature (Score: 0.847)\n",
"7. Art film (Score: 0.843)\n",
"8. Central Europe (Score: 0.843)\n",
"9. Art (Score: 0.842)\n"
"9. European (Score: 0.841)\n"
]
}
],
"source": [
"# For using OpenAI to generate query embedding\n",
- [Product Recommendations with DocArray / Jina](https://github.com/jina-ai/product-recommendation-redis-docarray) - Content-based product recommendations example with Redis and DocArray.
- [Redis VSS in RecSys](https://github.com/RedisVentures/Redis-Recsys) - 3 end-to-end Redis & NVIDIA Merlin Recommendation System Architectures.
- [Azure OpenAI Embeddings Q&A](https://github.com/ruoccofabrizio/azure-open-ai-embeddings-qna) - OpenAI and Redis as a Q&A service on Azure.
- [ArXiv Paper Search](https://github.com/RedisVentures/redis-arXiv-search) - Semantic search over arXiv scholarly papers
### More Resources
For more information on how to use Redis as a vector database, check out the following resources:
"Most developers from a web services background are probably familiar with Redis. At it's core, Redis is an open-source key-value store that can be used as a cache, message broker, and database. Developers choice Redis because it is fast, has a large ecosystem of client libraries, and has been deployed by major enterprises for years.\n",
"Most developers from a web services background are probably familiar with Redis. At it's core, Redis is an open-source key-value store that can be used as a cache, message broker, and database. Developers choice Redis because it is fast, has a large ecosystem of client libraries, and has been deployed by major enterprises for years.\n",
"\n",
"In addition to the traditional uses of Redis. Redis also provides [Redis Modules](https://redis.io/modules) which are a way to extend Redis with new data types and commands. Example modules include [RedisJSON](https://redis.io/docs/stack/json/), [RedisTimeSeries](https://redis.io/docs/stack/timeseries/), [RedisBloom](https://redis.io/docs/stack/bloom/) and [RediSearch](https://redis.io/docs/stack/search/).\n",
"\n",
"### What is RediSearch?\n",
"\n",
"RediSearch is a [Redis module](https://redis.io/modules) that provides querying, secondary indexing, full-text search and vector search for Redis. To use RediSearch, you first declare indexes on your Redis data. You can then use the RediSearch query language to query that data. For more information on the feature set of RediSearch, see the [README](./README.md) or the [RediSearch documentation](https://redis.io/docs/stack/search/).\n",
"RediSearch is a [Redis module](https://redis.io/modules) that provides querying, secondary indexing, full-text search and vector search for Redis. To use RediSearch, you first declare indexes on your Redis data. You can then use the RediSearch clients to query that data. For more information on the feature set of RediSearch, see the [README](./README.md) or the [RediSearch documentation](https://redis.io/docs/stack/search/).\n",
"\n",
"### Deployment options\n",
"\n",
@ -52,7 +52,7 @@
"To keep this example simple, we will use the Redis Stack docker container which we can start as follows\n",
"\n",
"```bash\n",
"$ docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest\n",
"$ docker compose up -d\n",
"```\n",
"\n",
"This also includes the [RedisInsight](https://redis.com/redis-enterprise/redis-insight/) GUI for managing your Redis database which you can view at [http://localhost:8001](http://localhost:8001) once you start the docker container.\n",
@ -73,45 +73,10 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": null,
"id": "2b04113f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: redis in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (4.3.4)\n",
"Requirement already satisfied: wget in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (3.2)\n",
"Requirement already satisfied: pandas in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (1.5.1)\n",
"Requirement already satisfied: openai in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (0.26.4)\n",
"Requirement already satisfied: packaging>=20.4 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from redis) (21.3)\n",
"Requirement already satisfied: async-timeout>=4.0.2 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from redis) (4.0.2)\n",
"Requirement already satisfied: deprecated>=1.2.3 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from redis) (1.2.13)\n",
"Requirement already satisfied: pytz>=2020.1 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from pandas) (2022.6)\n",
"Requirement already satisfied: python-dateutil>=2.8.1 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from pandas) (2.8.2)\n",
"Requirement already satisfied: numpy>=1.20.3 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from pandas) (1.22.4)\n",
"Requirement already satisfied: requests>=2.20 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from openai) (2.28.2)\n",
"Requirement already satisfied: aiohttp in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from openai) (3.8.3)\n",
"Requirement already satisfied: tqdm in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from openai) (4.64.1)\n",
"Requirement already satisfied: wrapt<2,>=1.10 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from deprecated>=1.2.3->redis) (1.14.1)\n",
"Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from packaging>=20.4->redis) (3.0.9)\n",
"Requirement already satisfied: six>=1.5 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from python-dateutil>=2.8.1->pandas) (1.16.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from requests>=2.20->openai) (2022.12.7)\n",
"Requirement already satisfied: idna<4,>=2.5 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from requests>=2.20->openai) (3.4)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from requests>=2.20->openai) (2.1.1)\n",
"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from requests>=2.20->openai) (1.26.14)\n",
"Requirement already satisfied: yarl<2.0,>=1.0 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from aiohttp->openai) (1.8.2)\n",
"Requirement already satisfied: multidict<7.0,>=4.5 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from aiohttp->openai) (6.0.4)\n",
"Requirement already satisfied: frozenlist>=1.1.1 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from aiohttp->openai) (1.3.3)\n",
"Requirement already satisfied: aiosignal>=1.1.2 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from aiohttp->openai) (1.3.1)\n",
"Requirement already satisfied: attrs>=17.3.0 in /Users/sam.partee/.virtualenvs/redisvl2/lib/python3.8/site-packages (from aiohttp->openai) (22.1.0)\n",
"\n",
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip available: \u001b[0m\u001b[31;49m22.2.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.0\u001b[0m\n",
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
]
}
],
"outputs": [],
"source": [
"!pip install redis wget pandas openai"
]
@ -134,7 +99,7 @@
},
{
"cell_type": "code",
"execution_count": 25,
"execution_count": 2,
"id": "88be138c",
"metadata": {},
"outputs": [
@ -153,7 +118,7 @@
"import openai\n",
"\n",
"# Note. alternatively you can set a temporary env variable like this:\n",
"'In Europe, after the Middle Ages, there was a \"Renaissance\" which means \"rebirth\". People rediscovered science and artists were allowed to paint subjects other than religious subjects. People like Michelangelo and Leonardo da Vinci still painted religious pictures, but they also now could paint mythological pictures too. These artists also invented perspective where things in the distance look smaller in the picture. This was new because in the Middle Ages people would paint all the figures close up and just overlapping each other. These artists used nudity regularly in their art.'"
]
},
"execution_count": 94,
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
@ -627,12 +680,14 @@
"\n",
"Up until now, we've been using the ``FLAT`` or \"brute-force\" index to run our queries. Redis also supports the ``HNSW`` index which is a fast, approximate index. The ``HNSW`` index is a graph-based index that uses a hierarchical navigable small world graph to store vectors. The ``HNSW`` index is a good choice for large datasets where you want to run approximate queries.\n",
"\n",
"``HNSW`` will take longer to build and consume more memory for most cases than ``FLAT`` but will be faster to run queries on, especially for large datasets.\n",
"\n",
"The following cells will show how to create an ``HNSW`` index and run queries with it using the same data as before."