langchain/tests/integration_tests/retrievers
Jason Fan 8effd90be0
Add new types of document transformers (#7379)
- Description: Add two new document transformers that translates
documents into different languages and converts documents into q&a
format to improve vector search results. Uses OpenAI function calling
via the [doctran](https://github.com/psychic-api/doctran/tree/main)
library.
  - Issue: N/A
  - Dependencies: `doctran = "^0.0.5"`
  - Tag maintainer: @rlancemartin @eyurtsev @hwchase17 
  - Twitter handle: @psychicapi or @jfan001

Notes
- Adheres to the `DocumentTransformer` abstraction set by @dev2049 in
#3182
- refactored `EmbeddingsRedundantFilter` to put it in a file under a new
`document_transformers` module
- Added basic docs for `DocumentInterrogator`, `DocumentTransformer` as
well as the existing `EmbeddingsRedundantFilter`

---------

Co-authored-by: Lance Martin <lance@langchain.dev>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-07-12 23:53:30 -04:00
..
docarray DocArray as a Retriever (#6031) 2023-06-17 09:09:33 -07:00
document_compressors Add new types of document transformers (#7379) 2023-07-12 23:53:30 -04:00
__init__.py
test_arxiv.py
test_azure_cognitive_search.py
test_contextual_compression.py Rm retriever kwargs (#7013) 2023-07-02 08:22:24 -06:00
test_merger_retriever.py Add serialized object to retriever start callback (#7074) 2023-07-05 18:04:43 +01:00
test_pupmed.py Harrison/pubmed integration (#5664) 2023-06-03 16:25:28 -07:00
test_weaviate_hybrid_search.py Remove unnecessary comment (#4845) 2023-05-17 11:53:03 -04:00
test_wikipedia.py