"# Running Hybrid VSS Queries with Redis and OpenAI\n",
"\n",
"This notebook provides an introduction to using Redis as a vector database with OpenAI embeddings and running hybrid queries that combine VSS and lexical search using Redis Search and Query capability. Redis is a scalable, real-time database that can be used as a vector database when using the [RediSearch Module](https://oss.redislabs.com/redisearch/). The RediSearch module allows you to index and search for vectors in Redis. This notebook will show you how to use the RediSearch module to index and search for vectors created by using the OpenAI API and stored in Redis.\n",
"This notebook provides an introduction to using Redis as a vector database with OpenAI embeddings and running hybrid queries that combine VSS and lexical search using Redis Query and Search capability. Redis is a scalable, real-time database that can be used as a vector database when using the [RediSearch Module](https://oss.redislabs.com/redisearch/). The Redis Query and Search capability allows you to index and search for vectors in Redis. This notebook will show you how to use the Redis Query and Search to index and search for vectors created by using the OpenAI API and stored in Redis.\n",
"\n",
"### What is Redis?\n",
"\n",
"Most developers from a web services background are probably familiar with Redis. At it's core, Redis is an open-source key-value store that can be used as a cache, message broker, and database. Developers choice Redis because it is fast, has a large ecosystem of client libraries, and has been deployed by major enterprises for years.\n",
"\n",
"In addition to the traditional uses of Redis. Redis also provides [Redis Modules](https://redis.io/modules) which are a way to extend Redis with new data types and commands. Example modules include [RedisJSON](https://redis.io/docs/stack/json/), [RedisTimeSeries](https://redis.io/docs/stack/timeseries/), [RedisBloom](https://redis.io/docs/stack/bloom/) and [RediSearch](https://redis.io/docs/stack/search/).\n",
"\n",
"### What is RediSearch?\n",
"\n",
"RediSearch is a [Redis module](https://redis.io/modules) that provides querying, secondary indexing, full-text search and vector search for Redis. To use RediSearch, you first declare indexes on your Redis data. You can then use the RediSearch clients to query that data. For more information on the feature set of RediSearch, see the [README](./README.md) or the [RediSearch documentation](https://redis.io/docs/stack/search/).\n",
"\n",
"### Deployment options\n",
"\n",
"There are a number of ways to deploy Redis. For local development, the quickest method is to use the [Redis Stack docker container](https://hub.docker.com/r/redis/redis-stack) which we will use here. Redis Stack contains a number of Redis modules that can be used together to create a fast, multi-model data store and query engine.\n",
"\n",
"For production use cases, The easiest way to get started is to use the [Redis Cloud](https://redislabs.com/redis-enterprise-cloud/overview/) service. Redis Cloud is a fully managed Redis service. You can also deploy Redis on your own infrastructure using [Redis Enterprise](https://redislabs.com/redis-enterprise/overview/). Redis Enterprise is a fully managed Redis service that can be deployed in kubernetes, on-premises or in the cloud.\n",
"\n",
"Additionally, every major cloud provider ([AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-e6y7ork67pjwg?sr=0-2&ref_=beagle&applicationId=AWSMPContessa), [Google Marketplace](https://console.cloud.google.com/marketplace/details/redislabs-public/redis-enterprise?pli=1), or [Azure Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/garantiadata.redis_enterprise_1sp_public_preview?tab=Overview)) offers Redis Enterprise in a marketplace offering.\n",
"\n"
"Hybrid queries combine vector similarity with traditional Redis Query and Search filtering capabilities on GEO, NUMERIC, TAG or TEXT data simplifying application code. A common example of a hybrid query in an e-commerce use case if to find items visually similar to a given query image limited to items available in a GEO location and within a price range."
"# For using OpenAI to generate query embedding\n",
"# Execute a simple vector search in Redis\n",
"results = search_redis(redis_client, 'man blue jeans', k=10)"
]
},
{
"cell_type": "markdown",
"id": "2007be48",
"metadata": {},
"source": [
"## Hybrid Queries with Redis\n",
"\n",
"The previous examples showed how run vector search queries with RediSearch. In this section, we will show how to combine vector search with other RediSearch fields for hybrid search. In the example below, we will combine vector search with full text search."
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "93c4a696",
"execution_count": 14,
"id": "0c4f4d0f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0. John Players Men Blue Jeans (Score: 0.739)\n",
"1. Lee Men Tino Blue Jeans (Score: 0.72)\n",
"2. Peter England Men Party Blue Jeans (Score: 0.718)\n",
"3. Denizen Women Blue Jeans (Score: 0.715)\n",
"4. Jealous 21 Women Washed Blue Jeans (Score: 0.708)\n",
"5. Jealous 21 Women Washed Blue Jeans (Score: 0.708)\n",
"6. Levis Kids Blue Solid Jean (Score: 0.706)\n",
"7. French Connection Men Blue Jeans (Score: 0.705)\n",
"8. Lee Men Blue Chicago Fit Jeans (Score: 0.705)\n",
"9. Lee Men Blue Chicago Fit Jeans (Score: 0.705)\n"
"0. John Players Men Blue Jeans (Score: 0.791)\n",
"1. Lee Men Tino Blue Jeans (Score: 0.775)\n",
"2. Peter England Men Party Blue Jeans (Score: 0.763)\n",
"3. French Connection Men Blue Jeans (Score: 0.74)\n",
"4. Locomotive Men Washed Blue Jeans (Score: 0.739)\n",
"5. Locomotive Men Washed Blue Jeans (Score: 0.739)\n",
"6. Palm Tree Kids Boy Washed Blue Jeans (Score: 0.732)\n",
"7. Denizen Women Blue Jeans (Score: 0.725)\n",
"8. Jealous 21 Women Washed Blue Jeans (Score: 0.713)\n",
"9. Jealous 21 Women Washed Blue Jeans (Score: 0.713)\n"
"The previous examples showed how run vector search queries with RediSearch. In this section, we will show how to combine vector search with other RediSearch fields for hybrid search. In the example below, we will combine vector search with full text search."
"# improve search quality by adding hybrid query for \"man blue jeans\" in the product vector combined with a phrase search for \"blue jeans\"\n",