13 Commits (9c88037dbc6cfc99c1be8263ec7a0a196da47724)

Author	SHA1	Message	Date
ChengZi	a03141ac51	partners[milvus]: fix integration test issues (#26136) ... fix some integration test issues: https://github.com/langchain-ai/langchain/actions/runs/10688447230/job/29628412258 Signed-off-by: ChengZi <chen.zhang@zilliz.com> Co-authored-by: Erick Friis <erick@langchain.dev>	2 weeks ago
Ohad Eytan	b5d670498f	partners/milvus: allow creating a vectorstore with sparse embeddings (#25284) ... # Description Milvus (and `pymilvus`) recently added the option to use [sparse vectors](https://milvus.io/docs/sparse_vector.md#Sparse-Vector) with appropriate search methods (e.g., `SPARSE_INVERTED_INDEX`) and embeddings (e.g., `BM25`, `SPLADE`). This PR allow creating a vector store using langchain's `Milvus` class, setting the matching vector field type to `DataType.SPARSE_FLOAT_VECTOR` and the default index type to `SPARSE_INVERTED_INDEX`. It is only extending functionality, and backward compatible. ## Note I also interested in extending the Milvus class further to support multi vector search (aka hybrid search). Will be happy to discuss that. See [here](https://github.com/langchain-ai/langchain/discussions/19955), [here](https://github.com/langchain-ai/langchain/pull/20375), and [here](https://github.com/langchain-ai/langchain/discussions/22886) similar needs. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	4 weeks ago
ChengZi	37f5ba416e	partners[milvus]: fix issue when metadata_schema is None (#25836) ... fix issue when metadata_schema is None Signed-off-by: ChengZi <chen.zhang@zilliz.com>	4 weeks ago
Rohit Gupta	aff50a1e6f	milvus: add array data type for collection create (#23219) ... Add array data type for milvus vector store collection create Thank you for contributing to LangChain! - [x] **PR title**: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [x] **PR message**: ***Delete this entire checklist*** and replace with - **Description:** a description of the change - **Issue:** the issue # it fixes, if applicable - **Dependencies:** any dependencies required for this change - **Twitter handle:** if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [x] **Add tests and docs**: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17. --------- Signed-off-by: ChengZi <chen.zhang@zilliz.com> Co-authored-by: Rohit Gupta <rohit.gupta2@walmart.com> Co-authored-by: ChengZi <chen.zhang@zilliz.com> Co-authored-by: Erick Friis <erick@langchain.dev>	4 weeks ago
venkatram-dev	ec99f0d193	milvus: add_db_milvus_connection (#25627) ... Thank you for contributing to LangChain! - [x] **PR title**: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - "libs: langchain_milvus: add db name to milvus connection check" - [x] **PR message**: ***Delete this entire checklist*** and replace with - **Description:** add db name to milvus connection check - **Issue:** https://github.com/langchain-ai/langchain/issues/25277 - [x] **Add tests and docs**: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	4 weeks ago
Christophe Bornet	7f1e444efa	partners: Use simsimd types (#25299) ... The simsimd package [now has types](https://github.com/ashvardanian/SimSIMD/releases/tag/v5.0.0)	1 month ago
Isaac Francisco	a72fddbf8d	[docs]: vector store integration pages (#24858) ... Co-authored-by: Erick Friis <erick@langchain.dev>	2 months ago
Stuart Marsh	16bd0697dc	milvus: fixed bug when using partition key and dynamic fields together (#25028) ... **Description:** This PR fixes a bug where if `enable_dynamic_field` and `partition_key_field` are enabled at the same time, a pymilvus error occurs. Milvus requires the partition key field to be a full schema defined field, and not a dynamic one, so it will throw the error "the specified partition key field {field} not exist" when creating the collection. When `enabled_dynamic_field` is set to `True`, all schema field creation based on `metadatas` is skipped. This code now checks if `partition_key_field` is set, and creates the field. Integration test added. **Twitter handle:** StuartMarshUK --------- Co-authored-by: Stuart Marsh <stuart.marsh@qumata.com> Co-authored-by: Erick Friis <erick@langchain.dev>	2 months ago
ChengZi	29a3b3a711	partners[milvus]: add dynamic field (#24544) ... add dynamic field feature to langchain_milvus more unittest, more robustic plan to deprecate the `metadata_field` in the future, because it's function is the same as `enable_dynamic_field`, but the latter one is a more advanced concept in milvus Signed-off-by: ChengZi <chen.zhang@zilliz.com> Co-authored-by: Erick Friis <erick@langchain.dev>	2 months ago
Leonid Ganeline	f0a7581b50	milvus: docstring (#23151) ... Added missed docstrings. Format docstrings to the consistent format (used in the API Reference) --------- Co-authored-by: Isaac Francisco <78627776+isahers1@users.noreply.github.com> Co-authored-by: isaac hershenson <ihershenson@hmc.edu> Co-authored-by: Erick Friis <erick@langchain.dev>	2 months ago
Eugene Yurtsev	8858846607	milvus[patch]: Fix Milvus vectorstore for newer versions of langchain-core (#24152) ... Fix for: https://github.com/langchain-ai/langchain/issues/24116 This keeps the old behavior of add_documents and add_texts	2 months ago
Joydeep Banik Roy	3796672c67	community, milvus, pinecone, qdrant, mongo: Broadcast operation failure while using simsimd beyond v3.7.7 (#22271) ... - [ ] **Packages affected**: - community: fix `cosine_similarity` to support simsimd beyond 3.7.7 - partners/milvus: fix `cosine_similarity` to support simsimd beyond 3.7.7 - partners/mongodb: fix `cosine_similarity` to support simsimd beyond 3.7.7 - partners/pinecone: fix `cosine_similarity` to support simsimd beyond 3.7.7 - partners/qdrant: fix `cosine_similarity` to support simsimd beyond 3.7.7 - [ ] **Broadcast operation failure while using simsimd beyond v3.7.7**: - **Description:** I was using simsimd 4.3.1 and the unsupported operand type issue popped up. When I checked out the repo and ran the tests, they failed as well (have attached a screenshot for that). Looks like it is a variant of https://github.com/langchain-ai/langchain/issues/18022 . Prior to 3.7.7, simd.cdist returned an ndarray but now it returns simsimd.DistancesTensor which is ineligible for a broadcast operation with numpy. With this change, it also remove the need to explicitly cast `Z` to numpy array - **Issue:** #19905 - **Dependencies:** No - **Twitter handle:** https://x.com/GetzJoydeep <img width="1622" alt="Screenshot 2024-05-29 at 2 50 00 PM" src="https://github.com/langchain-ai/langchain/assets/31132555/fb27b383-a9ae-4a6f-b355-6d503b72db56"> - [ ] **Considerations**: 1. I started with community but since similar changes were there in Milvus, MongoDB, Pinecone, and QDrant so I modified their files as well. If touching multiple packages in one PR is not the norm, then I can remove them from this PR and raise separate ones 2. I have run and verified that the tests work. Since, only MongoDB had tests, I ran theirs and verified it works as well. Screenshots attached : <img width="1573" alt="Screenshot 2024-05-29 at 2 52 13 PM" src="https://github.com/langchain-ai/langchain/assets/31132555/ce87d1ea-19b6-4900-9384-61fbc1a30de9"> <img width="1614" alt="Screenshot 2024-05-29 at 3 33 51 PM" src="https://github.com/langchain-ai/langchain/assets/31132555/6ce1d679-db4c-4291-8453-01028ab2dca5"> I have added a test for simsimd. I feel it may not go well with the CI/CD setup as installing simsimd is not a dependency requirement. I have just imported simsimd to ensure simsimd cosine similarity is invoked. However, its not a good approach. Suggestions are welcome and I can make the required changes on the PR. Please provide guidance on the same as I am new to the community. --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	4 months ago
ChengZi	404d92ded0	milvus: New langchain_milvus package and new milvus features (#21077) ... New features: - New langchain_milvus package in partner - Milvus collection hybrid search retriever - Zilliz cloud pipeline retriever - Milvus Local guid - Rag-milvus template --------- Signed-off-by: ChengZi <chen.zhang@zilliz.com> Signed-off-by: Jael Gu <mengjia.gu@zilliz.com> Co-authored-by: Jael Gu <mengjia.gu@zilliz.com> Co-authored-by: Jackson <jacksonxie612@gmail.com> Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: Erick Friis <erickfriis@gmail.com>	4 months ago