langchain

Commit Graph

Author	SHA1	Message	Date
Mike Lambert	e94a5d753f	Move from test to supported claude-instant-1 model (#9066 ) Moves from "test" model to "claude-instant-1" model which is supported and has actual capacity	1 year ago
Eugene Yurtsev	b7bc8ec87f	Add excludes to FileSystemBlobLoader (#9064 ) Add option to specify exclude patterns. https://github.com/langchain-ai/langchain/discussions/9059	1 year ago
Eugene Yurtsev	6c70f491ba	ChatPromptTemplate pending deprecation proposal (#9004 ) Pending deprecations for ChatPromptTemplate proposals	1 year ago
TRY-ER	2431eca700	Agent vector store tool doc (#9029 ) I was initially confused weather to use create_vectorstore_agent or create_vectorstore_router_agent due to lack of documentation so I created a simple documentation for each of the function about their different usecase. Replace this comment with: - Description: Added the doc_strings in create_vectorstore_agent and create_vectorstore_router_agent to point out the difference in their usecase - Tag maintainer: @rlancemartin, @eyurtsev --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Alvaro Bartolome	08a0741d82	Update `ArgillaCallbackHandler` as of latest `argilla` release (#9043 ) Hi @agola11, or whoever is reviewing this PR 😄 ## What's in this PR? As of the latest Argilla release, we'll change and refactor some things to make some workflows easier, one of those is how everything's pushed to Argilla, so that now there's no need to call `push_to_argilla` over a `FeedbackDataset` when either `push_to_argilla` is called for the first time, or `from_argilla` is called; among others. We also add some class variables to make sure those are easy to update in case we update those internally in the future, also to make the `warnings.warn` message lighter from the code view. P.S. Regarding the Twitter/X mention feel free to do so at either https://twitter.com/argilla_io or https://twitter.com/alvarobartt, or both if applicable, otherwise, just the first Twitter/X handle.	1 year ago
Blake (Yung Cher Ho)	8d351bfc20	Takeoff integration (#9045 ) ## Description: This PR adds the Titan Takeoff Server to the available LLMs in LangChain. Titan Takeoff is an inference server created by [TitanML](https://www.titanml.co/) that allows you to deploy large language models locally on your hardware in a single command. Most generative model architectures are included, such as Falcon, Llama 2, GPT2, T5 and many more. Read more about Titan Takeoff here: - [Blog](https://medium.com/@TitanML/introducing-titan-takeoff-6c30e55a8e1e) - [Docs](https://docs.titanml.co/docs/titan-takeoff/getting-started) #### Testing As Titan Takeoff runs locally on port 8000 by default, no network access is needed. Responses are mocked for testing. - [x] Make Lint - [x] Make Format - [x] Make Test #### Dependencies No new dependencies are introduced. However, users will need to install the titan-iris package in their local environment and start the Titan Takeoff inferencing server in order to use the Titan Takeoff integration. Thanks for your help and please let me know if you have any questions. cc: @hwchase17 @baskaryan	1 year ago
Nuno Campos	3bdc273ab3	Implement .transform() in RunnablePassthrough() (#9032 ) - This ensures passthrough doesnt break streaming --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	206f809366	fix sched ci (more) (#9056 )	1 year ago
Ismail Pelaseyed	abb1264edf	Fix issue with Metaphor Search Tool throwing error on missing keys in API response (#9051 ) - Description: Fixes an issue with Metaphor Search Tool throwing when missing keys in API response. - Issue: #9048 - Tag maintainer: @hinthornw @hwchase17 - Twitter handle: @pelaseyed	1 year ago
Eugene Yurtsev	5e05ba2140	Add embeddings cache (#8976 ) This PR adds the ability to temporarily cache or persistently store embeddings. A notebook has been included showing how to set up the cache and how to use it with a vectorstore.	1 year ago
Bagatur	6e14f9548b	bump 261 (#9041 )	1 year ago
Eugene Yurtsev	d21333d710	Add redis storage (#8980 ) Add a redis implementation of a BaseStore	1 year ago
Bagatur	434a96415b	make runnable dir (#9016 ) Co-authored-by: Nuno Campos <nuno@boringbits.io>	1 year ago
Nuno Campos	c7a489ae0d	Small improvements for tracer and debug output of runnables (#8683 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
EricFan	618cf5241e	Open file in UTF-8 encoding (#6919 ) (#8943 ) FileCallbackHandler cannot handle some language, for example: Chinese. Open file using UTF-8 encoding can fix it. @agola11 Issue: #6919 Dependencies: NO dependencies, --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
colegottdank	f4a47ec717	Add optional model kwargs to ChatAnthropic to allow overrides (#9013 ) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Kaizen	bbbd2b076f	DirectoryLoader slicing (#8994 ) DirectoryLoader can now return a random sample of files in a directory. Parameters added are: sample_size randomize_sample sample_seed @rlancemartin, @eyurtsev --------- Co-authored-by: Andrew Oseen <amovfx@protonmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
IanRogers-101Ways	d248481f13	skip over empty google spreadsheets (#8974 ) - Description: Allow GoogleDriveLoader to handle empty spreadsheets - Issue: Currently GoogleDriveLoader will crash if it tries to load a spreadsheet with an empty sheet - Dependencies: n/a - Tag maintainer: @rlancemartin, @eyurtsev	1 year ago
Eugene Yurtsev	efa02ed768	Suppress divide by zero wranings for cosine similarity (#9006 ) Suppress run time warnings for divide by zero as the downstream code handles the scenario (handling inf and nan)	1 year ago
Leonid Ganeline	5454591b0a	docstrings cleanup (#8993 ) Added/Updated docstrings @baskaryan	1 year ago
Massimiliano Pronesti	c72da53c10	Add logprobs to SamplingParameters in vllm (#9010 ) This PR aims at amending #8806 , that I opened a few days ago, adding the extra `logprobs` parameter that I accidentally forgot	1 year ago
Bagatur	8dd071ad08	import airbyte loaders (#9009 )	1 year ago
Bagatur	96d064e305	bump 260 (#9002 )	1 year ago
Nuno Campos	808248049d	Implement a router for openai functions (#8589 )	1 year ago
Eugene Yurtsev	a6e6e9bb86	Fix airbyte loader (#8998 ) Fix airbyte loader https://github.com/langchain-ai/langchain/issues/8996	1 year ago
William FH	90579021f8	Update Key Check (#8948 ) In eval loop. It needn't be done unless you are creating the corresponding evaluators	1 year ago
Jerzy Czopek	539672a7fd	Feature/fix azureopenai model mappings (#8621 ) This pull request aims to ensure that the `OpenAICallbackHandler` can properly calculate the total cost for Azure OpenAI chat models. The following changes have resolved this issue: - The `model_name` has been added to the ChatResult llm_output. Without this, the default values of `gpt-35-turbo` were applied. This was causing the total cost for Azure OpenAI's GPT-4 to be significantly inaccurate. - A new parameter `model_version` has been added to `AzureChatOpenAI`. Azure does not include the model version in the response. With the addition of `model_name`, this is not a significant issue for GPT-4 models, but it's an issue for GPT-3.5-Turbo. Version 0301 (default) of GPT-3.5-Turbo on Azure has a flat rate of 0.002 per 1k tokens for both prompt and completion. However, version 0613 introduced a split in pricing for prompt and completion tokens. - The `OpenAICallbackHandler` implementation has been updated with the proper model names, versions, and cost per 1k tokens. Unit tests have been added to ensure the functionality works as expected; the Azure ChatOpenAI notebook has been updated with examples. Maintainers: @hwchase17, @baskaryan Twitter handle: @jjczopek --------- Co-authored-by: Jerzy Czopek <jerzy.czopek@avanade.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
shibuiwilliam	3adb1e12ca	make trajectory eval chain stricter and add unit tests (#8909 ) - update trajectory eval logic to be stricter - add tests to trajectory eval chain	1 year ago
Nuno Campos	b8df15cd64	Adds transform support for runnables (#8762 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: jacoblee93 <jacoblee93@gmail.com> Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Harrison Chase	4d72288487	async output parser (#8894 ) Co-authored-by: Nuno Campos <nuno@boringbits.io>	1 year ago
Bagatur	3c6eccd701	bump 259 (#8951 )	1 year ago
Harrison Chase	7de6a1b78e	parent document retriever (#8941 )	1 year ago
Aarav Borthakur	3f64b8a761	Integrate Rockset as a chat history store (#8940 ) Description: Adds Rockset as a chat history store Dependencies: no changes Tag maintainer: @hwchase17 This PR passes linting and testing. I added a test for the integration and an example notebook showing its use.	1 year ago
William FH	e3056340da	Add id in error in tracer (#8944 )	1 year ago
Bagatur	95cf7de112	scheduled tests GHA (#8879 ) Adding scheduled daily GHA that runs marked integration tests. To start just marking some tests in test_openai	1 year ago
Joe Reuter	8f0cd91d57	Airbyte based loaders (#8586 ) This PR adds 8 new loaders: * `AirbyteCDKLoader` This reader can wrap and run all python-based Airbyte source connectors. * Separate loaders for the most commonly used APIs: * `AirbyteGongLoader` * `AirbyteHubspotLoader` * `AirbyteSalesforceLoader` * `AirbyteShopifyLoader` * `AirbyteStripeLoader` * `AirbyteTypeformLoader` * `AirbyteZendeskSupportLoader` ## Documentation and getting started I added the basic shape of the config to the notebooks. This increases the maintenance effort a bit, but I think it's worth it to make sure people can get started quickly with these important connectors. This is also why I linked the spec and the documentation page in the readme as these two contain all the information to configure a source correctly (e.g. it won't suggest using oauth if that's avoidable even if the connector supports it). ## Document generation The "documents" produced by these loaders won't have a text part (instead, all the record fields are put into the metadata). If a text is required by the use case, the caller needs to do custom transformation suitable for their use case. ## Incremental sync All loaders support incremental syncs if the underlying streams support it. By storing the `last_state` from the reader instance away and passing it in when loading, it will only load updated records. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Eugene Yurtsev	15f650ae8c	Add base storage interface, 2 implementations and utility encoder (#8895 ) This PR defines an abstract interface for key value stores. It provides 2 implementations: 1. Local File System 2. In memory -- used to facilitate testing It also provides an encoder utility to help take care of serialization from arbitrary data to data that can be stored by the given store	1 year ago
Harrison Chase	7543a3d70e	Harrison/image (#845 ) Co-authored-by: Ashutosh Sanzgiri <sanzgiri@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	ab193338aa	bump 258 (#8932 )	1 year ago
Eugene Yurtsev	bb12184551	Internal code deprecation API (#8763 ) Proposal for an internal API to deprecate LangChain code. This PR is heavily based on: https://github.com/matplotlib/matplotlib/blob/main/lib/matplotlib/_api/deprecation.py This PR only includes deprecation functionality (no renaming etc.). Additional functionality can be added on a need basis (e.g., renaming parameters), but best to roll out as an MVP to test this out. DeprecationWarnings are ignored by default. We can change the policy for the deprecation warnings, but we'll need to make sure we're not creating noise for users due to internal code invoking deprecated functionality.	1 year ago
Leonid Ganeline	33a2f58fbf	`tensoflow_datasets` document loader (#8721 ) This PR adds `tensoflow_datasets` document loader	1 year ago
Holt Skinner	fad26e79a3	fix: Resolve `AttributeError` in Google Cloud Enterprise Search retriever (#8872 ) - Reverting some of the changes made in https://github.com/langchain-ai/langchain/pull/8369	1 year ago
William FH	b2eb4ff0fc	Relax Validation in Eval (#8902 ) Just check for missing keys	1 year ago
Leonid Ganeline	2d078c7767	`PubMed` document loader (#8893 ) - added `PubMed Document Loader` artifacts; ut-s; examples - fixed `PubMed utility`; ut-s @hwchase17	1 year ago
Ofer Mendelevitch	a7824f16f2	Added consistent timeout for Vectara calls (#8892 ) - Description: consistent timeout at 60s for all calls to Vectara API - Tag maintainer: @rlancemartin, @eyurtsev --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	642b57c7ff	nit (#8927 )	1 year ago
manmax31	4a07fba9f0	Improve query prompt of BGE embeddings (#8908 ) Replace this comment with: - Description: Improved query of BGE embeddings after talking with the devs of BGE embeddings , - Dependencies: any dependencies required for this change, - Tag maintainer: @hwchase17 , - Twitter handle: @ManabChetia3 --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	1 year ago
Chris Pappalardo	beab637f04	added filter kwarg to VectorStoreIndexWrapper query and query_with_so… (#8844 ) - Description: added filter to query methods in VectorStoreIndexWrapper for filtering by metadata (i.e. search_kwargs) - Tag maintainer: @rlancemartin, @eyurtsev Updated the doc snippet on this topic as well. It took me a long while to figure out how to filter the vectorstore by filename, so this might help someone else out. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
David vonThenen	bf4a112aa6	Fixes to the Nebula LLM Integration (#8918 ) This addresses some issues with introducing the Nebula LLM to LangChain in this PR: https://github.com/langchain-ai/langchain/pull/8876 This fixes the following: - Removes `SYMBLAI` from variable names - Fixes bug with `Bearer` for the API KEY Thanks again in advance for your help! cc: @hwchase17, @baskaryan --------- Co-authored-by: dvonthenen <david.vonthenen@gmail.com>	1 year ago
Marie-Philippe Gill	6b9f266837	Add user_context to AmazonKendraRetriever (#8869 ) ### Description Now, we can pass information like a JWT token using user_context: ```python self.retriever = AmazonKendraRetriever(index_id=kendraIndexId, user_context={"Token": jwt_token}) ``` - [x] `make lint` - [x] `make format` - [x] `make test` Also tested by pip installing in my own project, and it allows access through the token. ### Maintainers @rlancemartin, @eyurtsev ### My twitter handle [girlknowstech](https://twitter.com/girlknowstech)	1 year ago
GitHub-L	67718c1d6b	Update OpenAPI code to fetch use the requestBody - Description: The API doc passed to LLM only included the content of responses but did not include the content of requestBody, causing the agent to be unable to construct the correct request parameters based on the requestBody information. Add two lines of code fixed the bug, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: @hinthornw , - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!	1 year ago
Leonid Kuligin	52d6b91c18	Fixed a source for documents uploaded from GCS (#8912 ) Sets source for documents uploaded from GCS to source on gcs #8911 Co-authored-by: Leonid Kuligin <kuligin@google.com>	1 year ago
Bagatur	022ef170f8	bump 257 (#8903 )	1 year ago
Jacob Lee	fa30a57034	Adds Ollama as an LLM (#8829 ) Adds Ollama as an LLM. Ollama can run various open source models locally e.g. Llama 2 and Vicuna, automatically configuring and GPU-optimizing them. @rlancemartin @hwchase17 --------- Co-authored-by: Lance Martin <lance@langchain.dev>	1 year ago
Ash Vardanian	1f9124ceaa	Add: USearch Vector Store (#8835 ) ## Description I am excited to propose an integration with USearch, a lightweight vector-search engine available for both Python and JavaScript, among other languages. ## Dependencies It introduces a new PyPi dependency - `usearch`. I am unsure if it must be added to the Poetry file, as this would make the PR too clunky. Please let me know. ## Profiles - Maintainers: @ashvardanian @davvard - Twitter handles: @ashvardanian @unum_cloud --------- Co-authored-by: Davit Vardanyan <78792753+davvard@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Leonid Kuligin	b52a3785c9	Allow to specify a custom loader for GcsFileLoader (#8868 ) Co-authored-by: Leonid Kuligin <kuligin@google.com>	1 year ago
Bruno Bornsztein	d56eff042a	Make json output parser handle newlines inside markdown code blocks (#8682 ) Update to #8528 Newlines and other special characters within markdown code blocks returned as `action_input` should be handled correctly (in particular, unescaped `"` => `\"` and `\n` => `\\n`) so they don't break JSON parsing. @baskaryan	1 year ago
Oege Dijk	cff52638b2	when encountering error during fetch return "" in web_base.py (#8753 ) when e.g. downloading a sitemap with a malformed url (e.g. "ttp://example.com/index.html" with the h omitted at the beginning of the url), this will ensure that the sitemap download does not crash, but just emits a warning. (maybe should be optional with e.g. a `skip_faulty_urls:bool=True` parameter, but this was the most straightforward fix) @rlancemartin, @eyurtsev --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bennji94	33cdb06b5c	Async RetryOutputParser, RetryWithErrorOutputParser and OutputFixingParser (#8776 ) Added async parsing functions for RetryOutputParser, RetryWithErrorOutputParser and OutputFixingParser. The async parse functions call the arun methods of the used LLMChains. Fix for #7989 --------- Co-authored-by: Benjamin May <benjamin.may94@gmail.com>	1 year ago
Joshua Sundance Bailey	7fc07ba5df	Create ChatAnyscale (#8770 ) - Description: Adds the ChatAnyscale class with llama-2 7b, llama-2 13b, and llama-2 70b on [Anyscale Endpoints](https://app.endpoints.anyscale.com/) - It inherits from ChatOpenAI and requires openai (probably unnecessary but it made for a quick and easy implementation) - Inspired by https://github.com/langchain-ai/langchain/pull/8434 (@kylehh and @baskaryan )	1 year ago
idcore	fe78aff1f2	Add new parameter forced_decoder_ids to OpenAIWhisperParserLocal + small bug fix (#8793 ) - Description: new parameter forced_decoder_ids for OpenAIWhisperParserLocal to force input language, and enable optional translate mode. Usage example: processor = WhisperProcessor.from_pretrained("openai/whisper-medium") forced_decoder_ids = processor.get_decoder_prompt_ids(language="french", task="transcribe") #forced_decoder_ids = processor.get_decoder_prompt_ids(language="french", task="translate") loader = GenericLoader(YoutubeAudioLoader(urls, save_dir), OpenAIWhisperParserLocal(lang_model="openai/whisper-medium",forced_decoder_ids=forced_decoder_ids)) - Issue #8792 - Tag maintainer: @rlancemartin, @eyurtsev --------- Co-authored-by: idcore <eugene.novozhilov@gmail.com>	1 year ago
David vonThenen	40079d4936	Introduce Nebula LLM to LangChain (#8876 ) ## Description This PR adds Nebula to the available LLMs in LangChain. Nebula is an LLM focused on conversation understanding and enables users to extract conversation insights from video, audio, text, and chat-based conversations. These conversations can occur between any mix of human or AI participants. Examples of some questions you could ask Nebula from a given conversation are: - What could be the customer’s pain points based on the conversation? - What sales opportunities can be identified from this conversation? - What best practices can be derived from this conversation for future customer interactions? You can read more about Nebula here: https://symbl.ai/blog/extract-insights-symbl-ai-generative-ai-recall-ai-meetings/ #### Integration Test An integration test is added, but it requires network access. Since Nebula is fully managed like OpenAI, network access is required to exercise the integration test. #### Linting - [x] make lint - [x] make test (TODO: there seems to be a failure in another non-related test??? Need to check on this.) - [x] make format ### Dependencies No new dependencies were introduced. ### Twitter handle [@symbldotai](https://twitter.com/symbldotai) [@dvonthenen](https://twitter.com/dvonthenen) If you have any questions, please let me know. cc: @hwchase17, @baskaryan --------- Co-authored-by: dvonthenen <david.vonthenen@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Eugene Yurtsev	f616aee35a	JsonOutputFunctionParser: Fix mutation in place bug (#8758 ) Fixes mutation in place in the JsonOutputFunctionParser. This causes issues when trying to re-use the original AI message.	1 year ago
shibuiwilliam	ab47557db3	fix evaluation parse test (#8859 ) # What - fix evaluation parse test <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Fix evaluation parse test - Issue: None - Dependencies: None - Tag maintainer: @baskaryan - Twitter handle: @MLOpsJ Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
manmax31	40096c73cd	Add BGE embeddings support (#8848 ) - Description: [BGE-large](https://huggingface.co/BAAI/bge-large-en) embeddings from BAAI are at the top of [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard). Hence adding support for it. - Tag maintainer: @baskaryan - Twitter handle: @ManabChetia3 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
shibuiwilliam	fbc83dfdbb	Fix/abstract add message (#8856 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Fix/abstract add message - Issue: None - Dependencies: None - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: @MLOpsJ Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
William FH	91be7eee66	Add concurrency support for run_on_dataset (#8841 ) Long-term, would be better to use the lower-level batch() method(s) but it may take me a bit longer to clean up. This unblocks in the meantime, though it may fail when the evaluated chain raises a `NotImplementedError` for a corresponding async method	1 year ago
Bagatur	fc2f450f2d	bump 256 (#8870 )	1 year ago
Tudor Golubenco	aeaef8f3a3	Add support for Xata as a vector store (#8822 ) This adds support for [Xata](https://xata.io) (data platform based on Postgres) as a vector store. We have recently added [Xata to Langchain.js](https://github.com/hwchase17/langchainjs/pull/2125) and would love to have the equivalent in the Python project as well. The PR includes integration tests and a Jupyter notebook as docs. Please let me know if anything else would be needed or helpful. I have added the xata python SDK as an optional dependency. ## To run the integration tests You will need to create a DB in xata (see the docs), then run something like: ``` OPENAI_API_KEY=sk-... XATA_API_KEY=xau_... XATA_DB_URL='https://....xata.sh/db/langchain' poetry run pytest tests/integration_tests/vectorstores/test_xata.py ``` <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Philip Krauss <35487337+philkra@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Leonid Kuligin	6e3fa59073	Added chat history to codey models (#8831 ) #7469 since 1.29.0, Vertex SDK supports a chat history provided to a codey chat model. Co-authored-by: Leonid Kuligin <kuligin@google.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Massimiliano Pronesti	a616e19975	feat(llms): add support for vLLM (#8806 ) Hello langchain maintainers, this PR aims at integrating [vllm](https://vllm.readthedocs.io/en/latest/#) into langchain. This PR closes #8729. This feature clearly depends on `vllm`, but I've seen other models supported here depend on packages that are not included in the pyproject.toml (e.g. `gpt4all`, `text-generation`) so I thought it was the case for this as well. @hwchase17, @baskaryan --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Bagatur	100d9ce4c7	bump 255 (#8865 )	1 year ago
Vic Cao	c9da300e4d	fix: overwrite stream for ChatOpenAI in runtime (#8288 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> @hwchase17, @baskaryan --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Nuno Campos <nuno@boringbits.io>	1 year ago
Karthik Raja A	5a9765b1b5	MultiOn client toolkit update 2.0 (#8750 ) - Updated to use newer better function interaction - Previous version had only one callback - @hinthornw @hwchase17 Can you look into this - Shout out to @MultiON_AI @DivGarg9 on twitter --------- Co-authored-by: Naman Garg <ngarg3@binghamton.edu> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Emre	454998c1fb	Fix invalid escape sequence warnings (#8771 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> Description: The lines I have changed looks like incorrectly escaped for regex. In python 3.11, I receive DeprecationWarning for these lines. You don't see any warnings unless you explicitly run python with `-W always::DeprecationWarning` flag. So, this is my attempt to fix it. Here are the warnings from log files: ``` /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:919: DeprecationWarning: invalid escape sequence '\s' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:918: DeprecationWarning: invalid escape sequence '\s' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:917: DeprecationWarning: invalid escape sequence '\s' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:916: DeprecationWarning: invalid escape sequence '\c' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:903: DeprecationWarning: invalid escape sequence '\' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:804: DeprecationWarning: invalid escape sequence '\' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:804: DeprecationWarning: invalid escape sequence '\*' ``` cc @baskaryan --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	0adc282d70	Harrison/as retriever docstring (#8840 ) Co-authored-by: Bytestorm <31070777+Bytestorm5@users.noreply.github.com>	1 year ago
Zend	bd4865b6fe	Async Recursive URL loader (#8502 ) Description: This PR improves the function of recursive_url_loader, such as limiting the depth of the access, and customizable extractors(from the raw webpage to the text of the Document object), so that users can use other tools to extract the webpage. This PR also includes the document and test for the new loader. Old PR closed due to project structure change. #7756 Because socket requests are not allowed, the old unit test was removed. Issue: N/A Dependencies: asyncio, aiohttp Tag maintainer: @rlancemartin Twitter handle: @ Zend_Nihility --------- Co-authored-by: Lance Martin <lance@langchain.dev>	1 year ago
fqassemi	485d716c21	Feature faiss delete (#8135 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: docstore had two main method: add and search, however, dealing with docstore sometimes requires deleting an entry from docstore. So I have added a simple delete method that deletes items from docstore. Additionally, I have added the delete method to faiss vectorstore for the very same reason. - Issue: NA - Dependencies: NA - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Pierre Alexandre SCHEMBRI	4a7ebb7184	Fix issue #7616 (#7617 ) Fix Issue #7616 with a simpler approach to extract function names (use `__name__` attribute) @hwchase17 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Ankur Agarwal	797c9e92c8	#8786 Fixed: Callback handler disconnect in between (#8787 ) Fixes for #8786 @agola11 - Description: The flow of callback is breaking till the last chain, as callbacks are missed in between chain along nested path. This will help get full trace and correlate parent child relationship in all nested chains. - Issue: the issue #8786 - Dependencies: NA - Tag maintainer: @agola11 - Twitter handle: Agarwal_Ankur	1 year ago
William FH	983678dedc	Add Dist Metrics for String Distance Evaluation (#8837 ) Co-authored-by: shibuiwilliam <shibuiyusuke@gmail.com>	1 year ago
William FH	f76d50d8dc	fix exception inconsistencies (#8812 ) (#8839 ) Merge #8812 with main to fix unrelated test failure Co-authored-by: shibuiwilliam <shibuiyusuke@gmail.com>	1 year ago
Bagatur	15c271e7b3	bump 254 (#8834 )	1 year ago
Bagatur	d7b613a293	Bagatur/revert revert nuclia (#8833 )	1 year ago
Bagatur	2f309a4ce6	Revert "Bagatur/nuclia (#8404 )" (#8832 )	1 year ago
Paul Hager	2111ed3c75	Improving the text of the invalid tool to list the available tools. (#8767 ) Description: When using a ReAct Agent with tools and no tool is found, the InvalidTool gets called. Previously it just asked for a different action, but I've found that if you list the available actions it improves the chances of getting a valid action in the next round. I've added a UnitTest for it also. @hinthornw	1 year ago
shibuiwilliam	d9bc46186d	Add missing test for retrievers self_query (#8783 ) # What - Add missing test for retrievers self_query - Add missing import validation <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Add missing test for retrievers self_query - Issue: None - Dependencies: None - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: @MlopsJ Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
Wilson Leao Neto	b0d0338f21	feat: expose Kendra result item id and document id as document metadata (#8796 ) - Description: we expose Kendra result item id and document id as document metadata. - Tag maintainer: @3coins @baskaryan - Twitter handle: wilsonleao Why The result item id and document id might be used to keep track of the retrieved resources.	1 year ago
Bagatur	9b86235a56	bump 253 (#8798 )	1 year ago
Bagatur	9fc9018951	Bagatur/nuclia (#8404 ) Co-authored-by: Eric BREHAULT <ebrehault@gmail.com>	1 year ago
William FH	1d68470bac	Same Project for Eval Runs (#8781 )	1 year ago
William FH	c8f3615aa6	Support evaluating runnables and arbitrary functions (#8698 ) Added a couple of "integration tests" for these that I ran. Main design point of feedback: at this point, would it just be better to have separate arguments for each type? Little confusing what is or isn't supported and what is the intended usage at this point since I try to wrap the function as runnable or pack or unpack chains/llms. ``` run_on_dataset( ... llm_or_chain_factory = None, llm = None, chain = NOne, runnable=None, function=None ): # raise error if none set ``` Downside with runnables and arbitrary function support is that you get much less helpful validation and error messages, but I don't think we should block you from this, at least.	1 year ago
liguoqinjim	d00a247da7	fix:get bilibili subtitles (#8165 ) - Description: fix the Loader 'BiliBiliLoader' - Issue: the API response was changed ![image](https://github.com/langchain-ai/langchain/assets/2113954/91216793-82f8-4c82-a018-d49f36f5f6aa) The previously used API no longer returns the "subtitle_url" property. ![image](https://github.com/langchain-ai/langchain/assets/2113954/a8ec2a7a-f40d-4c2a-b7d0-0ccdf2b327cc) We should use another API to get `subtitle_url` property. The `subtitle_url` returned by this API does not include the http schema and needs to be added. - Dependencies: Nope - Tag maintainer: @rlancemartin	1 year ago
Bagatur	f437311eef	Bagatur/runnable with fallbacks (#8543 )	1 year ago
Piyush Jain	8374367de2	Amazon Textract as document loader (#8661 ) Description: Adding support for [Amazon Textract](https://aws.amazon.com/textract/) as a PDF document loader --------- Co-authored-by: schadem <45048633+schadem@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Leonid Ganeline	82ef1f587d	fix makefile help (#8723 ) Fixed the `makefile` help. It was not up-to-date. @baskaryan	1 year ago
Neil Murphy	b0d0399d34	(issue #5163 ) Append reminder to nest multi-prompt router prompt output in JSON markdown code block, resolving JSON parsing error. (#8709 ) Resolves occasional JSON parsing error when some predictions are passed through a `MultiPromptChain`. Makes [this modification](https://github.com/langchain-ai/langchain/issues/5163#issuecomment-1652220401) to `multi_prompt_prompt.py`, which is much cleaner than appending an entire example object, which is another community-reported solution. @hwchase17, @baskaryan cc: @SimasJan	1 year ago
rjanardhan3	affaaea87b	Updates fireworks (#8765 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Updates to Fireworks Documentation, - Issue: N/A, - Dependencies: N/A, - Tag maintainer: @rlancemartin, --------- Co-authored-by: Raj Janardhan <rajjanardhan@Rajs-Laptop.attlocal.net>	1 year ago
Bagatur	e45be8b3f6	bump 252 (#8759 )	1 year ago
Bagatur	0d5a90f30a	Revert "add filter to sklearn vector store functions (#8113 )" (#8760 )	1 year ago
Bagatur	115a77142a	support for arbitrary kwargs for llamacpp (#8727 ) llamacpp params (per their own code) are unstable, so instead of adding/deleting them constantly adding a model_kwargs parameter that allows for arbitrary additional kwargs cc @jsjolund and @zacps re #8599 and #8704	1 year ago
Alec Flett	f0b0c72d98	add `load()` deserializer function that bypasses need for json serialization (#7626 ) There is already a `loads()` function which takes a JSON string and loads it using the Reviver But in the callbacks system, there is a `serialized` object that is passed in and that object is already a deserialized JSON-compatible object. This allows you to call `load(serialized)` and bypass intermediate JSON encoding. I found one other place in the code that benefited from this short-circuiting (string_run_evaluator.py) so I fixed that too. Tagging @baskaryan for general/utility stuff. <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Nuno Campos <nuno@boringbits.io>	1 year ago
Ruiqi Guo	6aee589eec	Add ScaNN support in vectorstore. (#8251 ) Description: Add ScaNN vectorstore to langchain. ScaNN is a Open Source, high performance vector similarity library optimized for AVX2-enabled CPUs. https://github.com/google-research/google-research/tree/master/scann - Dependencies: scann Python notebook to illustrate the usage: docs/extras/integrations/vectorstores/scann.ipynb Integration test: libs/langchain/tests/integration_tests/vectorstores/test_scann.py @rlancemartin, @eyurtsev for review. Thanks!	1 year ago
Moonsik Kang	5b7ff215e8	Fix load map reduce documents chain (#7915 ) This PR updates _load_reduce_documents_chain to handle `reduce_documents_chain` and `combine_documents_chain` config Please review @hwchase17, @baskaryan Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
shibuiwilliam	0f0ccfe7f6	add filter to sklearn vector store functions (#8113 ) # What - This is to add filter option to sklearn vectore store functions <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Add filter to sklearn vectore store functions. - Issue: None - Dependencies: None - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: @MlopsJ If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
shibuiwilliam	2759e2d857	add save and load tfidf vectorizer and docs for TFIDFRetriever (#8112 ) This is to add save_local and load_local to tfidf_vectorizer and docs in tfidf_retriever to make the vectorizer reusable. <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: add save_local and load_local to tfidf_vectorizer and docs in tfidf_retriever - Issue: None - Dependencies: None - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: @MlopsJ Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
aerickson-clt	0f68054401	Issue #8089 Improve painless script scoring with params.query_value. (#8086 ) This is a minor improvement that replaces the full query_vector with the reference string `params.query_value` used in the painless scripting docs. I have tested it manually and it works on an example. This makes the query about half the size and much easier to read. https://opensearch.org/docs/latest/search-plugins/knn/painless-functions/#get-started-with-k-nns-painless-scripting-functions @babbldev #8089 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
linpan	0ead8ea708	typo: ignored to ignore (#8740 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
aerickson-clt	c7ea6e9ff8	Issue 8081 Fix query results size bug. Other bug: pass vector_field param. (#8085 ) @baskaryan #8081 Likely the reason why the issue occurred is that OpenSearch's default k is 10, so it needs to be specified. Here's a similar question about its cousin ElasticSearch https://discuss.elastic.co/t/elasticsearch-returns-only-10-records-but-the-hit-is-507/136605 I tested this manually and also fixed the same issue in `_default_painless_scripting_query`. In addition, `_default_painless_scripting_query` was not passing the `vector_field` name to a sub call, so I fixed that too. ![image](https://github.com/hwchase17/langchain/assets/32244272/cfb7aad1-f701-49d9-9beb-a723aa276817) I also tested this in the aws opensearch developer tools. ![image](https://github.com/hwchase17/langchain/assets/32244272/24544682-1578-4bbb-9eb5-980463c5b41b) --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Sidchat95	812419d946	Removing score threshold parameter of faiss _similarity_search_with_r… (#8093 ) Removing score threshold parameter of faiss _similarity_search_with_relevance_scores as the thresholding part is implemented in similarity_search_with_relevance_scores method which calls this method. As this method is supposed to be a private method of faiss.py this will never receive the score threshold parameter as it is popped in the super method similarity_search_with_relevance_scores. @baskaryan @hwchase17	1 year ago
Mathias Panzenböck	873a80e496	Reduce generation of temporary objects (#7950 ) Just a tiny change to use `list.append(...)` and `list.extend(...)` instead of `list += [...]` so that no unnecessary temporary lists are created. Since its a tiny miscellaneous thing I guess @baskaryan is the maintainer to tag? --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Lance Martin	d1b95db874	Retriever that can re-phase user inputs (#8026 ) Simple retriever that applies an LLM between the user input and the query pass the to retriever. It can be used to pre-process the user input in any way. The default prompt: ``` DEFAULT_QUERY_PROMPT = PromptTemplate( input_variables=["question"], template="""You are an assistant tasked with taking a natural languge query from a user and converting it into a query for a vectorstore. In this process, you strip out information that is not relevant for the retrieval task. Here is the user query: {question} """ ) ``` --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	6c3573e7f6	Harrison/aleph alpha (#8735 ) Co-authored-by: PiotrMazurek <piotr.mazurek@aleph-alpha.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Wilson Leao Neto	179a39954d	Provides access to a Document page_content formatter in the AmazonKendraRetriever (#8034 ) - Description: - Provides a new attribute in the AmazonKendraRetriever which processes a ResultItem and returns a string that will be used as page_content; - The excerpt metadata should not be changed, it will be kept as was retrieved. But it is cleaned when composing the page_content; - Refactors the AmazonKendraRetriever to improve code reusability; - Issue: #7787 - Tag maintainer: @3coins @baskaryan - Twitter handle: wilsonleao Why? Some use cases need to adjust the page_content by dynamically combining the ResultItem attributes depending on the context of the item.	1 year ago
Ilya	6f0bccfeb5	Add regex control over separators in character text splitter (#7933 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> #7854 Added the ability to use the `separator` ase a regex or a simple character. Fixed a bug where `start_index` was incorrectly counting from -1. Who can review? @eyurtsev @hwchase17 @mmz-001	1 year ago
Vasileios Mansolas	e68a1d73d0	Fix Issue #6650 : Enable Azure Active Directory token-based auth access for AzureChatOpenAI (#8622 ) When using AzureChatOpenAI the openai_api_type defaults to "azure". The utils' get_from_dict_or_env() function triggered by the root validator does not look for user provided values from environment variables OPENAI_API_TYPE, so other values like "azure_ad" are replaced with "azure". This does not allow the use of token-based auth. By removing the "default" value, this allows environment variables to be pulled at runtime for the openai_api_type and thus enables the other api_types which are expected to work. This fixes #6650 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Alec Flett	5d765408ce	propagate callbacks through load_summarize_chain (#7565 ) This lets you pass callbacks when you create the summarize chain: ``` summarize = load_summarize_chain(llm, chain_type="map_reduce", callbacks=[my_callbacks]) summary = summarize(documents) ``` See #5572 for a similar surgical fix. tagging @hwchase17 for callbacks work <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
Alec Flett	404d103c41	propagate RetrievalQA chain callbacks through its own LLMChain and StuffDocumentsChain (#7853 ) This is another case, similar to #5572 and #7565 where the callbacks are getting dropped during construction of the chains. tagging @hwchase17 and @agola11 for callbacks propagation <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
Bal Narendra Sapa	47eea32f6a	add serializer methods (#7914 ) Description: I have added two methods serializer and deserializer methods. There was method called save local but it saves the to the local disk. I wanted the vectorstore in the format using which i can push it to the sql database's blob field. I have used this while i was working on something @rlancemartin, @eyurtsev --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Ryan Sloan	b786335dd1	fix RecursiveUrlLoader (#8582 ) Description: the recursive url loader does not fully crawl for all urls under base url Maintainer: @baskaryan	1 year ago
William FH	f81e613086	Fix Async Retry Event Handling (#8659 ) It fails currently because the event loop is already running. The `retry` decorator alraedy infers an `AsyncRetrying` handler for coroutines (see [tenacity line](`aa6f8f0a24/tenacity/__init__.py (L535)`)) However before_sleep always gets called synchronously (see [tenacity line](`aa6f8f0a24/tenacity/__init__.py (L338)`)). Instead, check for a running loop and use that it exists. Of course, it's running an async method synchronously which is not _nice_. Given how important LLMs are, it may make sense to have a task list or something but I'd want to chat with @nfcampos on where that would live. This PR also fixes the unit tests to check the handler is called and to make sure the async test is run (it looks like it's just been being skipped). It would have failed prior to the proposed fixes but passes now.	1 year ago
ruze	8ef7e14a85	RSS Feed / OPML loader (#8694 ) Replace this comment with: - Description: added a document loader for a list of RSS feeds or OPML. It iterates through the list and uses NewsURLLoader to load each article. - Issue: N/A - Dependencies: feedparser, listparser - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: @ruze --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
sumandeng	53e4148a1b	add model_revison parameter to ModelScopeEmbeddings (#8669 ) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Yoshi	4e8f11b36a	Deterministic Fake Embedding Model (#8706 ) Solves #8644 This embedding models output identical random embedding vectors, given the input texts are identical. Useful when used in unittest. @baskaryan	1 year ago
Leonid Kuligin	2928a1a3c9	added minimum expected version of SDK to the error description (#8712 ) #7932 Co-authored-by: Leonid Kuligin <kuligin@google.com>	1 year ago
Harrison Chase	814faa9de5	relax deps for yaml (#8713 ) context: https://github.com/yaml/pyyaml/issues/724 I think this is fine? I don't think we use yaml too heavily	1 year ago
Holt Skinner	8a8917e0d9	feat: Add Spell Correction Spec to Google Cloud Enterprise Search connector (#8705 )	1 year ago
Bagatur	b2b71b0d35	Bagatur/eden llm (#8670 ) Co-authored-by: RedhaWassim <rwasssim@gmail.com> Co-authored-by: KyrianC <ckyrian@protonmail.com> Co-authored-by: sam <melaine.samy@gmail.com>	1 year ago
William FH	8022293124	lint (#8702 )	1 year ago
William FH	a137492b53	Permit none key in chain mapper (#8696 )	1 year ago
Bagatur	e283dc8d50	bump 251 (#8690 )	1 year ago
Eugene Yurtsev	81e0cbf2d5	Minor typo fix (#8657 ) Fix typo in doc-string.	1 year ago
Harrison Chase	43dffe39fb	Harrison/conversational retrieval agent (#8639 ) Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
ruze	71f98db2fe	Newspaper (#8647 ) - Description: Added newspaper3k based news article loader. Provide a list of urls. - Issue: N/A - Dependencies: newspaper3k, - Tag maintainer: @rlancemartin , @eyurtsev - Twitter handle: @ruze --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
shibuiwilliam	f68f3b23d7	add missing RemoteLangChainRetriever _get_relevant_documents test (#8628 ) # What - Add missing RemoteLangChainRetriever _get_relevant_documents test --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
William FH	206901fa01	Use salt instead of datetime (#8653 ) If you want to kick off two runs at the same time it'll cause errors. Use a uuid instead	1 year ago
William FH	7ea2b08d1f	Use call directly for chain (#8655 ) for run_on_dataset since the `run()` method requires a single output	1 year ago
William FH	368aa4ede7	fix enum error message (#8652 ) could be a string so don't directly call value	1 year ago
Erick Friis	96b0ff182e	Enterprise support form wording (#8641 )	1 year ago
Will Thompson	ee1d13678e	🐛 Docs Fixes [2 one-liners, examples broken] (#8519 ) ## Description: 1)Map reduce example in docs is missing an important import statement. Figured other people would benefit from being able to copy 🍝 the code. 2)RefineDocumentsChain example also broken. ## Issue: None ## Dependencies: None. One liner. ## Tag maintainer: @baskaryan ## Twitter handle: I mean, it's a one line fix lol. But @will_thompson_k is my twitter handle.	1 year ago
Kacper Łukawski	16551536e3	Refactor Qdrant integration (#8634 ) This small PR introduces new parameters into Qdrant (`on_disk`), fixes some tests and changes the error message to be more clear. Tagging: @baskaryan, @rlancemartin, @eyurtsev	1 year ago
Erick Friis	c5fb3b6069	Enterprise support form in airtable (#8607 )	1 year ago
Eugene Yurtsev	1ec0b18379	Re-add __add__ functionality for messages (revert #8245 ) (#8489 ) This PR reverts #8245, so `__add__` is defined on base messages. Resolves issue: https://github.com/langchain-ai/langchain/issues/8472	1 year ago
Bagatur	f31047a394	bump 250 (#8632 )	1 year ago
Comendeiro	5c516945d0	Add local support for audio models (PR #7329 ) (#7591 ) - Description: run the poetry dependencies - Issue: #7329 - Dependencies: any dependencies required for this change, - Tag maintainer: @rlancemartin --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Naveen Tatikonda	d2adec3818	[Opensearch] : Fix the service validation in http_auth (#8609 ) ### Description OpenSearch supports validation using both Master Credentials (Username and password) and IAM. For Master Credentials users will not pass the argument `service` in `http_auth` and the existing code will break. To fix this, I have updated the condition to check if service attribute is present in http_auth before accessing it. ### Maintainers @baskaryan @navneet1v Signed-off-by: Naveen Tatikonda <navtat@amazon.com>	1 year ago
Harrison Chase	7c5c0557cb	cast to string when measuring token length (#8617 )	1 year ago
rjanardhan3	68113348cc	Fireworks integration (#8322 ) Description - Integrates Fireworks within Langchain LLMs to allow users to use Fireworks models with Langchain, mainly for summarization. Issue - Not applicable Dependencies - None Tag maintainer - @rlancemartin --------- Co-authored-by: Raj Janardhan <rajjanardhan@Rajs-Laptop.attlocal.net>	1 year ago
Bagatur	b574507c51	normalized openai embeddings embed_query (#8604 ) we weren't normalizing when embedding queries	1 year ago
Neil Murphy	31820a31e4	Add firestore_client param to FirestoreChatMessageHistory if caller already has one; also lets them specify GCP project, etc. (#8601 ) Existing implementation requires that you install `firebase-admin` package, and prevents you from using an existing Firestore client instance if available. This adds optional `firestore_client` param to `FirestoreChatMessageHistory`, so users can just use their existing client/settings. If not passed, existing logic executes to initialize a `firestore_client`. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago

1 2 3 4 5 ...

420 Commits (de8dfde7f75f0b6ade0629647e67f53b595230b5)