4871 Commits (89436de7a7682b627c449ea3fa54885a9894af1c)

 

All Branches   

Search

Author	SHA1	Message	Date
Michael Feil	94e31647bd	Support for Gradient.ai embedding (#10968) ... Adds support for gradient.ai's embedding model. This will remain a Draft, as the code will likely be refactored with the `pip install gradientai` python sdk.	12 months ago
Bagatur	5fd13c22ad	redirect mrkl (#10979)	12 months ago
C.J. Jameson	05d5fcfdf8	fix make-coverage local invocation #10941 (#10974) ... Fix the invocation of `make coverage` in `libs/langchain` Fixes #10941	12 months ago
Bagatur	040d436b3f	Add vertex scheduled test (#10958)	12 months ago
Piyush Jain	8602a32b7e	Fixes error with providers that don't have model_id (#10966) ... ## Description Fixes error with using the chain for providers that don't have `model_id` field. 	12 months ago
Nuno Campos	7b13292e35	Remove python eval from vector sql db chain (#10937) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	12 months ago
Richard Wang	b809c243af	Fix bug in `index` api (#10614) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> - **Description:** a fix for `index`. - **Issue:** Not applicable. - **Dependencies:** None - **Tag maintainer:** - **Twitter handle:** richarddwang # Problem Replication code ```python from pprint import pprint from langchain.embeddings import OpenAIEmbeddings from langchain.indexes import SQLRecordManager, index from langchain.schema import Document from langchain.vectorstores import Qdrant from langchain_setup.qdrant import pprint_qdrant_documents, create_inmemory_empty_qdrant # Documents metadata1 = {"source": "fullhell.alchemist"} doc1_1 = Document(page_content="1-1 I have a dog~", metadata=metadata1) doc1_2 = Document(page_content="1-2 I have a daugter~", metadata=metadata1) doc1_3 = Document(page_content="1-3 Ahh! O..Oniichan", metadata=metadata1) doc2 = Document(page_content="2 Lancer died again.", metadata={"source": "fate.docx"}) # Create empty vectorstore collection_name = "secret_of_D_disk" vectorstore: Qdrant = create_inmemory_empty_qdrant() # Create record Manager import tempfile from pathlib import Path record_manager = SQLRecordManager( namespace="qdrant/{collection_name}", db_url=f"sqlite:///{Path(tempfile.gettempdir())/collection_name}.sql", ) record_manager.create_schema() # 必須 sync_result = index( [doc1_1, doc1_2, doc1_2, doc2], record_manager, vectorstore, cleanup="full", source_id_key="source", ) print(sync_result, end="\n\n") pprint_qdrant_documents(vectorstore) ``` <details> <summary>Code of helper functions `pprint_qdrant_documents` and `create_inmemory_empty_qdrant`</summary> ```python def create_inmemory_empty_qdrant(**from_texts_kwargs): # Qdrant requires vector size, which can be only know after applying embedder vectorstore = Qdrant.from_texts(["dummy"], location=":memory:", embedding=OpenAIEmbeddings(), **from_texts_kwargs) dummy_document_id = vectorstore.client.scroll(vectorstore.collection_name)[0][0].id vectorstore.delete([dummy_document_id]) return vectorstore def pprint_qdrant_documents(vectorstore, limit: int = 100, **scroll_kwargs): document_ids, documents = [], [] for record in vectorstore.client.scroll( vectorstore.collection_name, limit=100, **scroll_kwargs )[0]: document_ids.append(record.id) documents.append( Document( page_content=record.payload["page_content"], metadata=record.payload["metadata"] or {}, ) ) pprint_documents(documents, document_ids=document_ids) def pprint_document(document: Document = None, document_id=None, return_string=False): displayed_text = "" if document_id: displayed_text += f"Document {document_id}:\n\n" displayed_text += f"{document.page_content}\n\n" metadata_text = pformat(document.metadata, indent=1) if "\n" in metadata_text: displayed_text += f"Metadata:\n{metadata_text}" else: displayed_text += f"Metadata:{metadata_text}" if return_string: return displayed_text else: print(displayed_text) def pprint_documents(documents, document_ids=None): if not document_ids: document_ids = [i + 1 for i in range(len(documents))] displayed_texts = [] for document_id, document in zip(document_ids, documents): displayed_text = pprint_document( document_id=document_id, document=document, return_string=True ) displayed_texts.append(displayed_text) print(f"\n{'-' * 100}\n".join(displayed_texts)) ``` </details> You will get ``` {'num_added': 3, 'num_updated': 0, 'num_skipped': 0, 'num_deleted': 0} Document 1b19816e-b802-53c0-ad60-5ff9d9b9b911: 1-2 I have a daugter~ Metadata:{'source': 'fullhell.alchemist'} ---------------------------------------------------------------------------------------------------- Document 3362f9bc-991a-5dd5-b465-c564786ce19c: 1-1 I have a dog~ Metadata:{'source': 'fullhell.alchemist'} ---------------------------------------------------------------------------------------------------- Document a4d50169-2fda-5339-a196-249b5f54a0de: 1-2 I have a daugter~ Metadata:{'source': 'fullhell.alchemist'} ``` This is not correct. We should be able to expect that the vectorsotre now includes doc1_1, doc1_2, and doc2, but not doc1_1, doc1_2, and doc1_2. # Reason In `index`, the original code is ```python uids = [] docs_to_index = [] for doc, hashed_doc, doc_exists in zip(doc_batch, hashed_docs, exists_batch): if doc_exists: # Must be updated to refresh timestamp. record_manager.update([hashed_doc.uid], time_at_least=index_start_dt) num_skipped += 1 continue uids.append(hashed_doc.uid) docs_to_index.append(doc) ``` In the aforementioned example, `len(doc_batch) == 4`, but `len(hashed_docs) == len(exists_batch) == 3`. This is because the deduplication of input documents [doc1_1, doc1_2, doc1_2, doc2] is [doc1_1, doc1_2, doc2]. So `index` insert doc1_1, doc1_2, doc1_2 with the uid of doc1_1, doc1_2, doc2. --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	12 months ago
Joshua Sundance Bailey	d67b120a41	Make anthropic_api_key a secret str (#10724) ... This PR makes `ChatAnthropic.anthropic_api_key` a `pydantic.SecretStr` to avoid inadvertently exposing API keys when the `ChatAnthropic` object is represented as a str.	12 months ago
Bagatur	1b65779905	fix integration tests (#10952)	12 months ago
Bagatur	6f781902ae	vercel fix (#10951)	12 months ago
Bagatur	f0408c347f	llm feat table revision (#10947)	12 months ago
Harrison Chase	9062e36722	Harrison/agents structured (#10911)	12 months ago
C.J. Jameson	b4d2663beb	CONTRIBUTING.md Quick Start: focus on langchain core; clarify docs and experimental are separate (#10906) ... follow up to https://github.com/langchain-ai/langchain/pull/7959 , explaining better to focus just on langchain core no dependencies twitter @cjcjameson	12 months ago
Michael Landis	f30b4697d4	fix: broken link in libs/langchain README (#10920) ... **Description** Fixes broken link to `CONTRIBUTING.md` in `libs/langchain/README.md`. Because`libs/langchain/README.md` was copied from the top level README, and because the README contains a link to `.github/CONTRIBUTING.md`, the copied README's link relative path must be updated. This commit fixes that link.	12 months ago
Bagatur	3cb460d5d8	bump 300 (#10940)	12 months ago
Bagatur	281a332784	table fix (#10944)	12 months ago
Bagatur	5336d87c15	update feat table (#10939)	12 months ago
Nuno Campos	3d5e92e3ef	Accept run name arg for non-chain runs (#10935)	12 months ago
Nuno Campos	aac2d4dcef	In MergerRetriever async call all retrievers in parallel (#10938)	12 months ago
German Martin	66d5a7e7cf	Add async support to multi-query retriever. (#10873) ... Added async support to the MultiQueryRetriever class. --------- Co-authored-by: Nuno Campos <nuno@boringbits.io>	12 months ago
Greg Richardson	4eee789dd3	Docs: Using SupabaseVectorStore with existing documents (#10907) ... ## Description Adds additional docs on how to use `SupabaseVectorStore` with existing data in your DB (vs inserting new documents each time).	12 months ago
Leonid Kuligin	9d4b710a48	small fixes to Vertex (#10934) ... Fixed tests, updated the required version of the SDK and a few minor changes after the recent improvement (https://github.com/langchain-ai/langchain/pull/10910)	12 months ago
wo0d	4e58b78102	Fix chat_history message order (#10869) ... Not all databases uses id as default order, so add it explicitly sqlite uses rawid as default order in select statement: [https://www.sqlite.org/lang_createtable.html#rowid](https://www.sqlite.org/lang_createtable.html#rowid), but some other databases like postgresql not behaves like this. since this class supports multiple db engine. we should have an order.	12 months ago
Roman Shaptala	3d40de75c5	Fix default refine prompt template bug (#10928) ... **Description:** Default refine template does not actually use the refine template defined above, it uses a string with the variable name. @baskaryan, @eyurtsev, @hwchase17	12 months ago
Bagatur	cab55e9bc1	add vertex prod features (#10910) ... - chat vertex async - vertex stream - vertex full generation info - vertex use server-side stopping - model garden async - update docs for all the above in follow up will add [] chat vertex full generation info [] chat vertex retries [] scheduled tests	12 months ago
Bagatur	dccc20b402	add model feat table (#10921)	12 months ago
William FH	ee8653f62c	Wfh/allow nonparallel (#10914)	12 months ago
Harrison Chase	bb3e6cb427	lcel benefits (#10898)	12 months ago
Leonid Kuligin	95e1d1fae6	fix in the docstring (#10902) ... Description: A fix in the documentation on how to use `GoogleSearchAPIWrapper`.	12 months ago
Bagatur	af41bc84e6	bump 299 (#10904)	12 months ago
Bagatur	9a858a9107	Bagatur/arxiv kwargs (#10903) ... support all arXiv api wrapper kwargs in loader	12 months ago
Maksym Diabin	697efd9757	JSONLoader Documentation Fix (#10505) ... - Description: Updated JSONLoader usage documentation which was making it unusable - Issue: JSONLoader if used with the documented arguments was failing on various JSON documents. - Dependencies: no dependencies - Twitter handle: @TheSlnArchitect	12 months ago
niklas	e5f420d2bc	Fix typo in URL document loader example (#10585) ... - **Description:** Fix typo in URL document loader example - **Issue:** N/A - **Dependencies:** N/A - **Tag maintainer:** not urgent	12 months ago
Nuno Campos	ea26c12b23	Fix Runnable.transform() for false-y inputs (#10893) ... --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	12 months ago
Nuno Campos	fcb5aba9f0	Add `Runnable.astream_log()` (#10374) ... --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	12 months ago
Harrison Chase	a1ade48e8f	update agent docs (#10894)	12 months ago
Stefano Lottini	40e836c67e	added Cassandra caches to the llm_caching notebook doc (#10889) ... This adds a section on usage of `CassandraCache` and `CassandraSemanticCache` to the doc notebook about caching LLMs, as suggested in [this comment](https://github.com/langchain-ai/langchain/pull/9772/#issuecomment-1710544100) on a previous merged PR. I also spotted what looks like a mismatch between different executions and propose a fix (line 98). Being the result of several runs, the cell execution numbers are scrambled somewhat, so I volunteer to refine this PR by (manually) re-numbering the cells to restore the appearance of a single, smooth running (for the sake of orderly execution :)	12 months ago
Bagatur	d37ce48e60	sep base url and loaded url in sub link extraction (#10895)	12 months ago
Bagatur	24cb5cd379	bump 298 (#10892)	12 months ago
Bagatur	c1f9cc0bc5	recursive loader add status check (#10891)	12 months ago
Matvey Arye	6e02c45ca4	Add integration for Timescale Vector(Postgres) (#10650) ... **Description:** This commit adds a vector store for the Postgres-based vector database (`TimescaleVector`). Timescale Vector(https://www.timescale.com/ai) is PostgreSQL++ for AI applications. It enables you to efficiently store and query billions of vector embeddings in `PostgreSQL`: - Enhances `pgvector` with faster and more accurate similarity search on 1B+ vectors via DiskANN inspired indexing algorithm. - Enables fast time-based vector search via automatic time-based partitioning and indexing. - Provides a familiar SQL interface for querying vector embeddings and relational data. Timescale Vector scales with you from POC to production: - Simplifies operations by enabling you to store relational metadata, vector embeddings, and time-series data in a single database. - Benefits from rock-solid PostgreSQL foundation with enterprise-grade feature liked streaming backups and replication, high-availability and row-level security. - Enables a worry-free experience with enterprise-grade security and compliance. Timescale Vector is available on Timescale, the cloud PostgreSQL platform. (There is no self-hosted version at this time.) LangChain users get a 90-day free trial for Timescale Vector. --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Avthar Sewrathan <avthar@timescale.com>	12 months ago
Michael Feil	55570e54e1	gradient.ai LLM intregration (#10800) ... - **Description:** This PR implements a new LLM API to https://gradient.ai - **Issue:** Feature request for LLM #10745 - **Dependencies**: No additional dependencies are introduced. - **Tag maintainer:** I am opening this PR for visibility, once ready for review I'll tag. - ```make format && make lint && make test``` is running. - added a `integration` and `mock unit` test. Co-authored-by: michaelfeil <me@michaelfeil.eu> Co-authored-by: Bagatur <baskaryan@gmail.com>	12 months ago
Bagatur	5097007407	cleanup recursive url session (#10863)	12 months ago
Harrison Chase	777b33b873	fix experimental imports (#10875)	12 months ago
Harrison Chase	808caca607	beef up agent docs (#10866)	12 months ago
Bagatur	4b558c9e17	update guide imports (#10865)	12 months ago
Sharath Rajasekar	96023f94d9	Add Javelin integration (#10275) ... We are introducing the py integration to Javelin AI Gateway www.getjavelin.io. Javelin is an enterprise-scale fast llm router & gateway. Could you please review and let us know if there is anything missing. Javelin AI Gateway wraps Embedding, Chat and Completion LLMs. Uses javelin_sdk under the covers (pip install javelin_sdk). Author: Sharath Rajasekar, Twitter: @sharathr, @javelinai Thanks!!	12 months ago
Bagatur	957956ba6d	bump 297 (#10861)	12 months ago
Harrison Chase	1bc3244db9	fix loading of sql chain (#10860) ... Closing #6889	12 months ago
Harrison Chase	4074ea4c41	fix databricks docs (#10858)	12 months ago