langchain

Commit Graph

Author	SHA1	Message	Date
Anar	ff732e10f8	LLMRails Embedding (#10959 ) LLMRails Embedding Integration This PR provides integration with LLMRails. Implemented here are: langchain/embeddings/llm_rails.py docs/extras/integrations/text_embedding/llm_rails.ipynb Hi @hwchase17 after adding our vectorstore integration to langchain with confirmation of you and @baskaryan, now we want to add our embedding integration --------- Co-authored-by: Anar Aliyev <aaliyev@mgmt.cloudnet.services> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Michael Feil	94e31647bd	Support for Gradient.ai embedding (#10968 ) Adds support for gradient.ai's embedding model. This will remain a Draft, as the code will likely be refactored with the `pip install gradientai` python sdk.	1 year ago
Bagatur	5fd13c22ad	redirect mrkl (#10979 )	1 year ago
C.J. Jameson	05d5fcfdf8	fix make-coverage local invocation #10941 (#10974 ) Fix the invocation of `make coverage` in `libs/langchain` Fixes #10941	1 year ago
Bagatur	040d436b3f	Add vertex scheduled test (#10958 )	1 year ago
Piyush Jain	8602a32b7e	Fixes error with providers that don't have model_id (#10966 ) ## Description Fixes error with using the chain for providers that don't have `model_id` field. ![image](https://github.com/langchain-ai/langchain/assets/289369/a86074cf-6c99-4390-a135-b3af7a4f0827)	1 year ago
Nuno Campos	7b13292e35	Remove python eval from vector sql db chain (#10937 ) <!-- Thank you for contributing to LangChain! Replace this entire comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	1 year ago
Richard Wang	b809c243af	Fix bug in `index` api (#10614 ) <!-- Thank you for contributing to LangChain! Replace this entire comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> - Description: a fix for `index`. - Issue: Not applicable. - Dependencies: None - Tag maintainer: - Twitter handle: richarddwang # Problem Replication code ```python from pprint import pprint from langchain.embeddings import OpenAIEmbeddings from langchain.indexes import SQLRecordManager, index from langchain.schema import Document from langchain.vectorstores import Qdrant from langchain_setup.qdrant import pprint_qdrant_documents, create_inmemory_empty_qdrant # Documents metadata1 = {"source": "fullhell.alchemist"} doc1_1 = Document(page_content="1-1 I have a dog~", metadata=metadata1) doc1_2 = Document(page_content="1-2 I have a daugter~", metadata=metadata1) doc1_3 = Document(page_content="1-3 Ahh! O..Oniichan", metadata=metadata1) doc2 = Document(page_content="2 Lancer died again.", metadata={"source": "fate.docx"}) # Create empty vectorstore collection_name = "secret_of_D_disk" vectorstore: Qdrant = create_inmemory_empty_qdrant() # Create record Manager import tempfile from pathlib import Path record_manager = SQLRecordManager( namespace="qdrant/{collection_name}", db_url=f"sqlite:///{Path(tempfile.gettempdir())/collection_name}.sql", ) record_manager.create_schema() # 必須 sync_result = index( [doc1_1, doc1_2, doc1_2, doc2], record_manager, vectorstore, cleanup="full", source_id_key="source", ) print(sync_result, end="\n\n") pprint_qdrant_documents(vectorstore) ``` <details> <summary>Code of helper functions `pprint_qdrant_documents` and `create_inmemory_empty_qdrant`</summary> ```python def create_inmemory_empty_qdrant(from_texts_kwargs): # Qdrant requires vector size, which can be only know after applying embedder vectorstore = Qdrant.from_texts(["dummy"], location=":memory:", embedding=OpenAIEmbeddings(), from_texts_kwargs) dummy_document_id = vectorstore.client.scroll(vectorstore.collection_name)[0][0].id vectorstore.delete([dummy_document_id]) return vectorstore def pprint_qdrant_documents(vectorstore, limit: int = 100, scroll_kwargs): document_ids, documents = [], [] for record in vectorstore.client.scroll( vectorstore.collection_name, limit=100, scroll_kwargs )[0]: document_ids.append(record.id) documents.append( Document( page_content=record.payload["page_content"], metadata=record.payload["metadata"] or {}, ) ) pprint_documents(documents, document_ids=document_ids) def pprint_document(document: Document = None, document_id=None, return_string=False): displayed_text = "" if document_id: displayed_text += f"Document {document_id}:\n\n" displayed_text += f"{document.page_content}\n\n" metadata_text = pformat(document.metadata, indent=1) if "\n" in metadata_text: displayed_text += f"Metadata:\n{metadata_text}" else: displayed_text += f"Metadata:{metadata_text}" if return_string: return displayed_text else: print(displayed_text) def pprint_documents(documents, document_ids=None): if not document_ids: document_ids = [i + 1 for i in range(len(documents))] displayed_texts = [] for document_id, document in zip(document_ids, documents): displayed_text = pprint_document( document_id=document_id, document=document, return_string=True ) displayed_texts.append(displayed_text) print(f"\n{'-' * 100}\n".join(displayed_texts)) ``` </details> You will get ``` {'num_added': 3, 'num_updated': 0, 'num_skipped': 0, 'num_deleted': 0} Document 1b19816e-b802-53c0-ad60-5ff9d9b9b911: 1-2 I have a daugter~ Metadata:{'source': 'fullhell.alchemist'} ---------------------------------------------------------------------------------------------------- Document 3362f9bc-991a-5dd5-b465-c564786ce19c: 1-1 I have a dog~ Metadata:{'source': 'fullhell.alchemist'} ---------------------------------------------------------------------------------------------------- Document a4d50169-2fda-5339-a196-249b5f54a0de: 1-2 I have a daugter~ Metadata:{'source': 'fullhell.alchemist'} ``` This is not correct. We should be able to expect that the vectorsotre now includes doc1_1, doc1_2, and doc2, but not doc1_1, doc1_2, and doc1_2. # Reason In `index`, the original code is ```python uids = [] docs_to_index = [] for doc, hashed_doc, doc_exists in zip(doc_batch, hashed_docs, exists_batch): if doc_exists: # Must be updated to refresh timestamp. record_manager.update([hashed_doc.uid], time_at_least=index_start_dt) num_skipped += 1 continue uids.append(hashed_doc.uid) docs_to_index.append(doc) ``` In the aforementioned example, `len(doc_batch) == 4`, but `len(hashed_docs) == len(exists_batch) == 3`. This is because the deduplication of input documents [doc1_1, doc1_2, doc1_2, doc2] is [doc1_1, doc1_2, doc2]. So `index` insert doc1_1, doc1_2, doc1_2 with the uid of doc1_1, doc1_2, doc2. --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	1 year ago
Joshua Sundance Bailey	d67b120a41	Make anthropic_api_key a secret str (#10724 ) This PR makes `ChatAnthropic.anthropic_api_key` a `pydantic.SecretStr` to avoid inadvertently exposing API keys when the `ChatAnthropic` object is represented as a str.	1 year ago
Bagatur	1b65779905	fix integration tests (#10952 )	1 year ago
Bagatur	6f781902ae	vercel fix (#10951 )	1 year ago
Bagatur	f0408c347f	llm feat table revision (#10947 )	1 year ago
Harrison Chase	9062e36722	Harrison/agents structured (#10911 )	1 year ago
C.J. Jameson	b4d2663beb	CONTRIBUTING.md Quick Start: focus on langchain core; clarify docs and experimental are separate (#10906 ) follow up to https://github.com/langchain-ai/langchain/pull/7959 , explaining better to focus just on langchain core no dependencies twitter @cjcjameson	1 year ago
Michael Landis	f30b4697d4	fix: broken link in libs/langchain README (#10920 ) Description Fixes broken link to `CONTRIBUTING.md` in `libs/langchain/README.md`. Because`libs/langchain/README.md` was copied from the top level README, and because the README contains a link to `.github/CONTRIBUTING.md`, the copied README's link relative path must be updated. This commit fixes that link.	1 year ago
Bagatur	3cb460d5d8	bump 300 (#10940 )	1 year ago
Bagatur	281a332784	table fix (#10944 )	1 year ago
Bagatur	5336d87c15	update feat table (#10939 )	1 year ago
Nuno Campos	3d5e92e3ef	Accept run name arg for non-chain runs (#10935 )	1 year ago
Nuno Campos	aac2d4dcef	In MergerRetriever async call all retrievers in parallel (#10938 )	1 year ago
German Martin	66d5a7e7cf	Add async support to multi-query retriever. (#10873 ) Added async support to the MultiQueryRetriever class. --------- Co-authored-by: Nuno Campos <nuno@boringbits.io>	1 year ago
Greg Richardson	4eee789dd3	Docs: Using SupabaseVectorStore with existing documents (#10907 ) ## Description Adds additional docs on how to use `SupabaseVectorStore` with existing data in your DB (vs inserting new documents each time).	1 year ago
Leonid Kuligin	9d4b710a48	small fixes to Vertex (#10934 ) Fixed tests, updated the required version of the SDK and a few minor changes after the recent improvement (https://github.com/langchain-ai/langchain/pull/10910)	1 year ago
wo0d	4e58b78102	Fix chat_history message order (#10869 ) Not all databases uses id as default order, so add it explicitly sqlite uses rawid as default order in select statement: [https://www.sqlite.org/lang_createtable.html#rowid](https://www.sqlite.org/lang_createtable.html#rowid), but some other databases like postgresql not behaves like this. since this class supports multiple db engine. we should have an order.	1 year ago
Roman Shaptala	3d40de75c5	Fix default refine prompt template bug (#10928 ) Description: Default refine template does not actually use the refine template defined above, it uses a string with the variable name. @baskaryan, @eyurtsev, @hwchase17	1 year ago
Bagatur	cab55e9bc1	add vertex prod features (#10910 ) - chat vertex async - vertex stream - vertex full generation info - vertex use server-side stopping - model garden async - update docs for all the above in follow up will add [] chat vertex full generation info [] chat vertex retries [] scheduled tests	1 year ago
Bagatur	dccc20b402	add model feat table (#10921 )	1 year ago
William FH	ee8653f62c	Wfh/allow nonparallel (#10914 )	1 year ago
Harrison Chase	bb3e6cb427	lcel benefits (#10898 )	1 year ago
Leonid Kuligin	95e1d1fae6	fix in the docstring (#10902 ) Description: A fix in the documentation on how to use `GoogleSearchAPIWrapper`.	1 year ago
Bagatur	af41bc84e6	bump 299 (#10904 )	1 year ago
Bagatur	9a858a9107	Bagatur/arxiv kwargs (#10903 ) support all arXiv api wrapper kwargs in loader	1 year ago
Maksym Diabin	697efd9757	JSONLoader Documentation Fix (#10505 ) - Description: Updated JSONLoader usage documentation which was making it unusable - Issue: JSONLoader if used with the documented arguments was failing on various JSON documents. - Dependencies: no dependencies - Twitter handle: @TheSlnArchitect	1 year ago
niklas	e5f420d2bc	Fix typo in URL document loader example (#10585 ) - Description: Fix typo in URL document loader example - Issue: N/A - Dependencies: N/A - Tag maintainer: not urgent	1 year ago
Nuno Campos	ea26c12b23	Fix Runnable.transform() for false-y inputs (#10893 ) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Nuno Campos	fcb5aba9f0	Add `Runnable.astream_log()` (#10374 ) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Harrison Chase	a1ade48e8f	update agent docs (#10894 )	1 year ago
Stefano Lottini	40e836c67e	added Cassandra caches to the llm_caching notebook doc (#10889 ) This adds a section on usage of `CassandraCache` and `CassandraSemanticCache` to the doc notebook about caching LLMs, as suggested in [this comment](https://github.com/langchain-ai/langchain/pull/9772/#issuecomment-1710544100) on a previous merged PR. I also spotted what looks like a mismatch between different executions and propose a fix (line 98). Being the result of several runs, the cell execution numbers are scrambled somewhat, so I volunteer to refine this PR by (manually) re-numbering the cells to restore the appearance of a single, smooth running (for the sake of orderly execution :)	1 year ago
Bagatur	d37ce48e60	sep base url and loaded url in sub link extraction (#10895 )	1 year ago
Bagatur	24cb5cd379	bump 298 (#10892 )	1 year ago
Bagatur	c1f9cc0bc5	recursive loader add status check (#10891 )	1 year ago
Matvey Arye	6e02c45ca4	Add integration for Timescale Vector(Postgres) (#10650 ) Description: This commit adds a vector store for the Postgres-based vector database (`TimescaleVector`). Timescale Vector(https://www.timescale.com/ai) is PostgreSQL++ for AI applications. It enables you to efficiently store and query billions of vector embeddings in `PostgreSQL`: - Enhances `pgvector` with faster and more accurate similarity search on 1B+ vectors via DiskANN inspired indexing algorithm. - Enables fast time-based vector search via automatic time-based partitioning and indexing. - Provides a familiar SQL interface for querying vector embeddings and relational data. Timescale Vector scales with you from POC to production: - Simplifies operations by enabling you to store relational metadata, vector embeddings, and time-series data in a single database. - Benefits from rock-solid PostgreSQL foundation with enterprise-grade feature liked streaming backups and replication, high-availability and row-level security. - Enables a worry-free experience with enterprise-grade security and compliance. Timescale Vector is available on Timescale, the cloud PostgreSQL platform. (There is no self-hosted version at this time.) LangChain users get a 90-day free trial for Timescale Vector. --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Avthar Sewrathan <avthar@timescale.com>	1 year ago
Michael Feil	55570e54e1	gradient.ai LLM intregration (#10800 ) - Description: This PR implements a new LLM API to https://gradient.ai - Issue: Feature request for LLM #10745 - Dependencies: No additional dependencies are introduced. - Tag maintainer: I am opening this PR for visibility, once ready for review I'll tag. - ```make format && make lint && make test``` is running. - added a `integration` and `mock unit` test. Co-authored-by: michaelfeil <me@michaelfeil.eu> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	5097007407	cleanup recursive url session (#10863 )	1 year ago
Harrison Chase	777b33b873	fix experimental imports (#10875 )	1 year ago
Harrison Chase	808caca607	beef up agent docs (#10866 )	1 year ago
Bagatur	4b558c9e17	update guide imports (#10865 )	1 year ago
Sharath Rajasekar	96023f94d9	Add Javelin integration (#10275 ) We are introducing the py integration to Javelin AI Gateway www.getjavelin.io. Javelin is an enterprise-scale fast llm router & gateway. Could you please review and let us know if there is anything missing. Javelin AI Gateway wraps Embedding, Chat and Completion LLMs. Uses javelin_sdk under the covers (pip install javelin_sdk). Author: Sharath Rajasekar, Twitter: @sharathr, @javelinai Thanks!!	1 year ago
Bagatur	957956ba6d	bump 297 (#10861 )	1 year ago
Harrison Chase	1bc3244db9	fix loading of sql chain (#10860 ) Closing #6889	1 year ago

1 2 3 4 5 ...

4722 Commits (33da8bd711224bd64106b55dadcd594b4b572b7a) All Branches Search

4722 Commits (33da8bd711224bd64106b55dadcd594b4b572b7a)

All Branches