Add elasticsearch examples to vector databases folder (#622)

* Add Elasticsearch to vector databases, add notebooks

* Update prompt

* Make intro verbiage more neutral

* Add semantic search notebook outputs

* Add RAG notebook output

* Update query

* Remove unreadable vector output
pull/665/head
Liam Thompson 10 months ago committed by GitHub
parent 169f5e02c8
commit 31b4de22a3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -1,6 +1,6 @@
# Vector Databases
This section of the OpenAI Cookbook showcases many of the vector databases available to support your semantic search use cases.
This section of the OpenAI Cookbook showcases many of the vector databases available to support your semantic search use cases.
Vector databases can be a great accompaniment for knowledge retrieval applications, which reduce hallucinations by providing the LLM with the relevant context to answer questions.
@ -9,6 +9,7 @@ Each provider has their own named directory, with a standard notebook to introdu
## Guides & deep dives
- [AnalyticDB](https://www.alibabacloud.com/help/en/analyticdb-for-postgresql/latest/get-started-with-analyticdb-for-postgresql)
- [Chroma](https://docs.trychroma.com/getting-started)
- [Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html)
- [Hologres](https://www.alibabacloud.com/help/en/hologres/latest/procedure-to-use-hologres)
- [Kusto](https://learn.microsoft.com/en-us/azure/data-explorer/web-query-data)
- [Milvus](https://milvus.io/docs/example_code.md)

@ -0,0 +1,31 @@
# Elasticsearch
Elasticsearch is a popular search/analytics engine and [vector database](https://www.elastic.co/elasticsearch/vector-database).
Elasticsearch offers an efficient way to create, store, and search vector embeddings at scale.
For technical details, refer to the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html).
The [`elasticsearch-labs`](https://github.com/elastic/elasticsearch-labs) repo contains executable Python notebooks, sample apps, and resources for testing out the Elastic platform.
## OpenAI cookbook notebooks 📒
Check out our notebooks in this repo for working with OpenAI, using Elasticsearch as your vector database.
### [Semantic search](https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb)
In this notebook you'll learn how to:
- Index the OpenAI Wikipedia embeddings dataset into Elasticsearch
- Encode a question with the `openai ada-02` model
- Perform a semantic search
<hr>
### [Retrieval augmented generation](https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb)
This notebooks builds on the semantic search notebook by:
- Selecting the top hit from a semantic search
- Sending that result to the OpenAI [Chat Completions](https://platform.openai.com/docs/guides/gpt/chat-completions-api) API endpoint for retrieval augmented generation (RAG)

File diff suppressed because one or more lines are too long
Loading…
Cancel
Save