langchain

Commit Graph

Author	SHA1	Message	Date
GitHub-L	67718c1d6b	Update OpenAPI code to fetch use the requestBody - Description: The API doc passed to LLM only included the content of responses but did not include the content of requestBody, causing the agent to be unable to construct the correct request parameters based on the requestBody information. Add two lines of code fixed the bug, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: @hinthornw , - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!	1 year ago
Leonid Kuligin	52d6b91c18	Fixed a source for documents uploaded from GCS (#8912 ) Sets source for documents uploaded from GCS to source on gcs #8911 Co-authored-by: Leonid Kuligin <kuligin@google.com>	1 year ago
Bagatur	022ef170f8	bump 257 (#8903 )	1 year ago
Jacob Lee	fa30a57034	Adds Ollama as an LLM (#8829 ) Adds Ollama as an LLM. Ollama can run various open source models locally e.g. Llama 2 and Vicuna, automatically configuring and GPU-optimizing them. @rlancemartin @hwchase17 --------- Co-authored-by: Lance Martin <lance@langchain.dev>	1 year ago
Ash Vardanian	1f9124ceaa	Add: USearch Vector Store (#8835 ) ## Description I am excited to propose an integration with USearch, a lightweight vector-search engine available for both Python and JavaScript, among other languages. ## Dependencies It introduces a new PyPi dependency - `usearch`. I am unsure if it must be added to the Poetry file, as this would make the PR too clunky. Please let me know. ## Profiles - Maintainers: @ashvardanian @davvard - Twitter handles: @ashvardanian @unum_cloud --------- Co-authored-by: Davit Vardanyan <78792753+davvard@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Leonid Kuligin	b52a3785c9	Allow to specify a custom loader for GcsFileLoader (#8868 ) Co-authored-by: Leonid Kuligin <kuligin@google.com>	1 year ago
Bruno Bornsztein	d56eff042a	Make json output parser handle newlines inside markdown code blocks (#8682 ) Update to #8528 Newlines and other special characters within markdown code blocks returned as `action_input` should be handled correctly (in particular, unescaped `"` => `\"` and `\n` => `\\n`) so they don't break JSON parsing. @baskaryan	1 year ago
Oege Dijk	cff52638b2	when encountering error during fetch return "" in web_base.py (#8753 ) when e.g. downloading a sitemap with a malformed url (e.g. "ttp://example.com/index.html" with the h omitted at the beginning of the url), this will ensure that the sitemap download does not crash, but just emits a warning. (maybe should be optional with e.g. a `skip_faulty_urls:bool=True` parameter, but this was the most straightforward fix) @rlancemartin, @eyurtsev --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bennji94	33cdb06b5c	Async RetryOutputParser, RetryWithErrorOutputParser and OutputFixingParser (#8776 ) Added async parsing functions for RetryOutputParser, RetryWithErrorOutputParser and OutputFixingParser. The async parse functions call the arun methods of the used LLMChains. Fix for #7989 --------- Co-authored-by: Benjamin May <benjamin.may94@gmail.com>	1 year ago
Joshua Sundance Bailey	7fc07ba5df	Create ChatAnyscale (#8770 ) - Description: Adds the ChatAnyscale class with llama-2 7b, llama-2 13b, and llama-2 70b on [Anyscale Endpoints](https://app.endpoints.anyscale.com/) - It inherits from ChatOpenAI and requires openai (probably unnecessary but it made for a quick and easy implementation) - Inspired by https://github.com/langchain-ai/langchain/pull/8434 (@kylehh and @baskaryan )	1 year ago
idcore	fe78aff1f2	Add new parameter forced_decoder_ids to OpenAIWhisperParserLocal + small bug fix (#8793 ) - Description: new parameter forced_decoder_ids for OpenAIWhisperParserLocal to force input language, and enable optional translate mode. Usage example: processor = WhisperProcessor.from_pretrained("openai/whisper-medium") forced_decoder_ids = processor.get_decoder_prompt_ids(language="french", task="transcribe") #forced_decoder_ids = processor.get_decoder_prompt_ids(language="french", task="translate") loader = GenericLoader(YoutubeAudioLoader(urls, save_dir), OpenAIWhisperParserLocal(lang_model="openai/whisper-medium",forced_decoder_ids=forced_decoder_ids)) - Issue #8792 - Tag maintainer: @rlancemartin, @eyurtsev --------- Co-authored-by: idcore <eugene.novozhilov@gmail.com>	1 year ago
David vonThenen	40079d4936	Introduce Nebula LLM to LangChain (#8876 ) ## Description This PR adds Nebula to the available LLMs in LangChain. Nebula is an LLM focused on conversation understanding and enables users to extract conversation insights from video, audio, text, and chat-based conversations. These conversations can occur between any mix of human or AI participants. Examples of some questions you could ask Nebula from a given conversation are: - What could be the customer’s pain points based on the conversation? - What sales opportunities can be identified from this conversation? - What best practices can be derived from this conversation for future customer interactions? You can read more about Nebula here: https://symbl.ai/blog/extract-insights-symbl-ai-generative-ai-recall-ai-meetings/ #### Integration Test An integration test is added, but it requires network access. Since Nebula is fully managed like OpenAI, network access is required to exercise the integration test. #### Linting - [x] make lint - [x] make test (TODO: there seems to be a failure in another non-related test??? Need to check on this.) - [x] make format ### Dependencies No new dependencies were introduced. ### Twitter handle [@symbldotai](https://twitter.com/symbldotai) [@dvonthenen](https://twitter.com/dvonthenen) If you have any questions, please let me know. cc: @hwchase17, @baskaryan --------- Co-authored-by: dvonthenen <david.vonthenen@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Eugene Yurtsev	f616aee35a	JsonOutputFunctionParser: Fix mutation in place bug (#8758 ) Fixes mutation in place in the JsonOutputFunctionParser. This causes issues when trying to re-use the original AI message.	1 year ago
shibuiwilliam	ab47557db3	fix evaluation parse test (#8859 ) # What - fix evaluation parse test <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Fix evaluation parse test - Issue: None - Dependencies: None - Tag maintainer: @baskaryan - Twitter handle: @MLOpsJ Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
manmax31	40096c73cd	Add BGE embeddings support (#8848 ) - Description: [BGE-large](https://huggingface.co/BAAI/bge-large-en) embeddings from BAAI are at the top of [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard). Hence adding support for it. - Tag maintainer: @baskaryan - Twitter handle: @ManabChetia3 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
shibuiwilliam	fbc83dfdbb	Fix/abstract add message (#8856 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Fix/abstract add message - Issue: None - Dependencies: None - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: @MLOpsJ Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
William FH	91be7eee66	Add concurrency support for run_on_dataset (#8841 ) Long-term, would be better to use the lower-level batch() method(s) but it may take me a bit longer to clean up. This unblocks in the meantime, though it may fail when the evaluated chain raises a `NotImplementedError` for a corresponding async method	1 year ago
Bagatur	fc2f450f2d	bump 256 (#8870 )	1 year ago
Tudor Golubenco	aeaef8f3a3	Add support for Xata as a vector store (#8822 ) This adds support for [Xata](https://xata.io) (data platform based on Postgres) as a vector store. We have recently added [Xata to Langchain.js](https://github.com/hwchase17/langchainjs/pull/2125) and would love to have the equivalent in the Python project as well. The PR includes integration tests and a Jupyter notebook as docs. Please let me know if anything else would be needed or helpful. I have added the xata python SDK as an optional dependency. ## To run the integration tests You will need to create a DB in xata (see the docs), then run something like: ``` OPENAI_API_KEY=sk-... XATA_API_KEY=xau_... XATA_DB_URL='https://....xata.sh/db/langchain' poetry run pytest tests/integration_tests/vectorstores/test_xata.py ``` <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Philip Krauss <35487337+philkra@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Leonid Kuligin	6e3fa59073	Added chat history to codey models (#8831 ) #7469 since 1.29.0, Vertex SDK supports a chat history provided to a codey chat model. Co-authored-by: Leonid Kuligin <kuligin@google.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Massimiliano Pronesti	a616e19975	feat(llms): add support for vLLM (#8806 ) Hello langchain maintainers, this PR aims at integrating [vllm](https://vllm.readthedocs.io/en/latest/#) into langchain. This PR closes #8729. This feature clearly depends on `vllm`, but I've seen other models supported here depend on packages that are not included in the pyproject.toml (e.g. `gpt4all`, `text-generation`) so I thought it was the case for this as well. @hwchase17, @baskaryan --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Bagatur	100d9ce4c7	bump 255 (#8865 )	1 year ago
Vic Cao	c9da300e4d	fix: overwrite stream for ChatOpenAI in runtime (#8288 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> @hwchase17, @baskaryan --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Nuno Campos <nuno@boringbits.io>	1 year ago
Karthik Raja A	5a9765b1b5	MultiOn client toolkit update 2.0 (#8750 ) - Updated to use newer better function interaction - Previous version had only one callback - @hinthornw @hwchase17 Can you look into this - Shout out to @MultiON_AI @DivGarg9 on twitter --------- Co-authored-by: Naman Garg <ngarg3@binghamton.edu> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Emre	454998c1fb	Fix invalid escape sequence warnings (#8771 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> Description: The lines I have changed looks like incorrectly escaped for regex. In python 3.11, I receive DeprecationWarning for these lines. You don't see any warnings unless you explicitly run python with `-W always::DeprecationWarning` flag. So, this is my attempt to fix it. Here are the warnings from log files: ``` /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:919: DeprecationWarning: invalid escape sequence '\s' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:918: DeprecationWarning: invalid escape sequence '\s' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:917: DeprecationWarning: invalid escape sequence '\s' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:916: DeprecationWarning: invalid escape sequence '\c' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:903: DeprecationWarning: invalid escape sequence '\' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:804: DeprecationWarning: invalid escape sequence '\' /usr/local/lib/python3.11/site-packages/langchain/text_splitter.py:804: DeprecationWarning: invalid escape sequence '\*' ``` cc @baskaryan --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	0adc282d70	Harrison/as retriever docstring (#8840 ) Co-authored-by: Bytestorm <31070777+Bytestorm5@users.noreply.github.com>	1 year ago
Zend	bd4865b6fe	Async Recursive URL loader (#8502 ) Description: This PR improves the function of recursive_url_loader, such as limiting the depth of the access, and customizable extractors(from the raw webpage to the text of the Document object), so that users can use other tools to extract the webpage. This PR also includes the document and test for the new loader. Old PR closed due to project structure change. #7756 Because socket requests are not allowed, the old unit test was removed. Issue: N/A Dependencies: asyncio, aiohttp Tag maintainer: @rlancemartin Twitter handle: @ Zend_Nihility --------- Co-authored-by: Lance Martin <lance@langchain.dev>	1 year ago
fqassemi	485d716c21	Feature faiss delete (#8135 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: docstore had two main method: add and search, however, dealing with docstore sometimes requires deleting an entry from docstore. So I have added a simple delete method that deletes items from docstore. Additionally, I have added the delete method to faiss vectorstore for the very same reason. - Issue: NA - Dependencies: NA - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Pierre Alexandre SCHEMBRI	4a7ebb7184	Fix issue #7616 (#7617 ) Fix Issue #7616 with a simpler approach to extract function names (use `__name__` attribute) @hwchase17 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Ankur Agarwal	797c9e92c8	#8786 Fixed: Callback handler disconnect in between (#8787 ) Fixes for #8786 @agola11 - Description: The flow of callback is breaking till the last chain, as callbacks are missed in between chain along nested path. This will help get full trace and correlate parent child relationship in all nested chains. - Issue: the issue #8786 - Dependencies: NA - Tag maintainer: @agola11 - Twitter handle: Agarwal_Ankur	1 year ago
William FH	983678dedc	Add Dist Metrics for String Distance Evaluation (#8837 ) Co-authored-by: shibuiwilliam <shibuiyusuke@gmail.com>	1 year ago
William FH	f76d50d8dc	fix exception inconsistencies (#8812 ) (#8839 ) Merge #8812 with main to fix unrelated test failure Co-authored-by: shibuiwilliam <shibuiyusuke@gmail.com>	1 year ago
Bagatur	15c271e7b3	bump 254 (#8834 )	1 year ago
Bagatur	d7b613a293	Bagatur/revert revert nuclia (#8833 )	1 year ago
Bagatur	2f309a4ce6	Revert "Bagatur/nuclia (#8404 )" (#8832 )	1 year ago
Paul Hager	2111ed3c75	Improving the text of the invalid tool to list the available tools. (#8767 ) Description: When using a ReAct Agent with tools and no tool is found, the InvalidTool gets called. Previously it just asked for a different action, but I've found that if you list the available actions it improves the chances of getting a valid action in the next round. I've added a UnitTest for it also. @hinthornw	1 year ago
shibuiwilliam	d9bc46186d	Add missing test for retrievers self_query (#8783 ) # What - Add missing test for retrievers self_query - Add missing import validation <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Add missing test for retrievers self_query - Issue: None - Dependencies: None - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: @MlopsJ Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
Wilson Leao Neto	b0d0338f21	feat: expose Kendra result item id and document id as document metadata (#8796 ) - Description: we expose Kendra result item id and document id as document metadata. - Tag maintainer: @3coins @baskaryan - Twitter handle: wilsonleao Why The result item id and document id might be used to keep track of the retrieved resources.	1 year ago
Bagatur	9b86235a56	bump 253 (#8798 )	1 year ago
Bagatur	9fc9018951	Bagatur/nuclia (#8404 ) Co-authored-by: Eric BREHAULT <ebrehault@gmail.com>	1 year ago
William FH	1d68470bac	Same Project for Eval Runs (#8781 )	1 year ago
William FH	c8f3615aa6	Support evaluating runnables and arbitrary functions (#8698 ) Added a couple of "integration tests" for these that I ran. Main design point of feedback: at this point, would it just be better to have separate arguments for each type? Little confusing what is or isn't supported and what is the intended usage at this point since I try to wrap the function as runnable or pack or unpack chains/llms. ``` run_on_dataset( ... llm_or_chain_factory = None, llm = None, chain = NOne, runnable=None, function=None ): # raise error if none set ``` Downside with runnables and arbitrary function support is that you get much less helpful validation and error messages, but I don't think we should block you from this, at least.	1 year ago
liguoqinjim	d00a247da7	fix:get bilibili subtitles (#8165 ) - Description: fix the Loader 'BiliBiliLoader' - Issue: the API response was changed ![image](https://github.com/langchain-ai/langchain/assets/2113954/91216793-82f8-4c82-a018-d49f36f5f6aa) The previously used API no longer returns the "subtitle_url" property. ![image](https://github.com/langchain-ai/langchain/assets/2113954/a8ec2a7a-f40d-4c2a-b7d0-0ccdf2b327cc) We should use another API to get `subtitle_url` property. The `subtitle_url` returned by this API does not include the http schema and needs to be added. - Dependencies: Nope - Tag maintainer: @rlancemartin	1 year ago
Bagatur	f437311eef	Bagatur/runnable with fallbacks (#8543 )	1 year ago
Piyush Jain	8374367de2	Amazon Textract as document loader (#8661 ) Description: Adding support for [Amazon Textract](https://aws.amazon.com/textract/) as a PDF document loader --------- Co-authored-by: schadem <45048633+schadem@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Leonid Ganeline	82ef1f587d	fix makefile help (#8723 ) Fixed the `makefile` help. It was not up-to-date. @baskaryan	1 year ago
Neil Murphy	b0d0399d34	(issue #5163 ) Append reminder to nest multi-prompt router prompt output in JSON markdown code block, resolving JSON parsing error. (#8709 ) Resolves occasional JSON parsing error when some predictions are passed through a `MultiPromptChain`. Makes [this modification](https://github.com/langchain-ai/langchain/issues/5163#issuecomment-1652220401) to `multi_prompt_prompt.py`, which is much cleaner than appending an entire example object, which is another community-reported solution. @hwchase17, @baskaryan cc: @SimasJan	1 year ago
rjanardhan3	affaaea87b	Updates fireworks (#8765 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Updates to Fireworks Documentation, - Issue: N/A, - Dependencies: N/A, - Tag maintainer: @rlancemartin, --------- Co-authored-by: Raj Janardhan <rajjanardhan@Rajs-Laptop.attlocal.net>	1 year ago
Bagatur	e45be8b3f6	bump 252 (#8759 )	1 year ago
Bagatur	0d5a90f30a	Revert "add filter to sklearn vector store functions (#8113 )" (#8760 )	1 year ago
Bagatur	115a77142a	support for arbitrary kwargs for llamacpp (#8727 ) llamacpp params (per their own code) are unstable, so instead of adding/deleting them constantly adding a model_kwargs parameter that allows for arbitrary additional kwargs cc @jsjolund and @zacps re #8599 and #8704	1 year ago
Alec Flett	f0b0c72d98	add `load()` deserializer function that bypasses need for json serialization (#7626 ) There is already a `loads()` function which takes a JSON string and loads it using the Reviver But in the callbacks system, there is a `serialized` object that is passed in and that object is already a deserialized JSON-compatible object. This allows you to call `load(serialized)` and bypass intermediate JSON encoding. I found one other place in the code that benefited from this short-circuiting (string_run_evaluator.py) so I fixed that too. Tagging @baskaryan for general/utility stuff. <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Nuno Campos <nuno@boringbits.io>	1 year ago
Ruiqi Guo	6aee589eec	Add ScaNN support in vectorstore. (#8251 ) Description: Add ScaNN vectorstore to langchain. ScaNN is a Open Source, high performance vector similarity library optimized for AVX2-enabled CPUs. https://github.com/google-research/google-research/tree/master/scann - Dependencies: scann Python notebook to illustrate the usage: docs/extras/integrations/vectorstores/scann.ipynb Integration test: libs/langchain/tests/integration_tests/vectorstores/test_scann.py @rlancemartin, @eyurtsev for review. Thanks!	1 year ago
Moonsik Kang	5b7ff215e8	Fix load map reduce documents chain (#7915 ) This PR updates _load_reduce_documents_chain to handle `reduce_documents_chain` and `combine_documents_chain` config Please review @hwchase17, @baskaryan Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
shibuiwilliam	0f0ccfe7f6	add filter to sklearn vector store functions (#8113 ) # What - This is to add filter option to sklearn vectore store functions <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Add filter to sklearn vectore store functions. - Issue: None - Dependencies: None - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: @MlopsJ If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
shibuiwilliam	2759e2d857	add save and load tfidf vectorizer and docs for TFIDFRetriever (#8112 ) This is to add save_local and load_local to tfidf_vectorizer and docs in tfidf_retriever to make the vectorizer reusable. <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: add save_local and load_local to tfidf_vectorizer and docs in tfidf_retriever - Issue: None - Dependencies: None - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: @MlopsJ Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
aerickson-clt	0f68054401	Issue #8089 Improve painless script scoring with params.query_value. (#8086 ) This is a minor improvement that replaces the full query_vector with the reference string `params.query_value` used in the painless scripting docs. I have tested it manually and it works on an example. This makes the query about half the size and much easier to read. https://opensearch.org/docs/latest/search-plugins/knn/painless-functions/#get-started-with-k-nns-painless-scripting-functions @babbldev #8089 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
linpan	0ead8ea708	typo: ignored to ignore (#8740 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
aerickson-clt	c7ea6e9ff8	Issue 8081 Fix query results size bug. Other bug: pass vector_field param. (#8085 ) @baskaryan #8081 Likely the reason why the issue occurred is that OpenSearch's default k is 10, so it needs to be specified. Here's a similar question about its cousin ElasticSearch https://discuss.elastic.co/t/elasticsearch-returns-only-10-records-but-the-hit-is-507/136605 I tested this manually and also fixed the same issue in `_default_painless_scripting_query`. In addition, `_default_painless_scripting_query` was not passing the `vector_field` name to a sub call, so I fixed that too. ![image](https://github.com/hwchase17/langchain/assets/32244272/cfb7aad1-f701-49d9-9beb-a723aa276817) I also tested this in the aws opensearch developer tools. ![image](https://github.com/hwchase17/langchain/assets/32244272/24544682-1578-4bbb-9eb5-980463c5b41b) --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Sidchat95	812419d946	Removing score threshold parameter of faiss _similarity_search_with_r… (#8093 ) Removing score threshold parameter of faiss _similarity_search_with_relevance_scores as the thresholding part is implemented in similarity_search_with_relevance_scores method which calls this method. As this method is supposed to be a private method of faiss.py this will never receive the score threshold parameter as it is popped in the super method similarity_search_with_relevance_scores. @baskaryan @hwchase17	1 year ago
Mathias Panzenböck	873a80e496	Reduce generation of temporary objects (#7950 ) Just a tiny change to use `list.append(...)` and `list.extend(...)` instead of `list += [...]` so that no unnecessary temporary lists are created. Since its a tiny miscellaneous thing I guess @baskaryan is the maintainer to tag? --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Lance Martin	d1b95db874	Retriever that can re-phase user inputs (#8026 ) Simple retriever that applies an LLM between the user input and the query pass the to retriever. It can be used to pre-process the user input in any way. The default prompt: ``` DEFAULT_QUERY_PROMPT = PromptTemplate( input_variables=["question"], template="""You are an assistant tasked with taking a natural languge query from a user and converting it into a query for a vectorstore. In this process, you strip out information that is not relevant for the retrieval task. Here is the user query: {question} """ ) ``` --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	6c3573e7f6	Harrison/aleph alpha (#8735 ) Co-authored-by: PiotrMazurek <piotr.mazurek@aleph-alpha.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Wilson Leao Neto	179a39954d	Provides access to a Document page_content formatter in the AmazonKendraRetriever (#8034 ) - Description: - Provides a new attribute in the AmazonKendraRetriever which processes a ResultItem and returns a string that will be used as page_content; - The excerpt metadata should not be changed, it will be kept as was retrieved. But it is cleaned when composing the page_content; - Refactors the AmazonKendraRetriever to improve code reusability; - Issue: #7787 - Tag maintainer: @3coins @baskaryan - Twitter handle: wilsonleao Why? Some use cases need to adjust the page_content by dynamically combining the ResultItem attributes depending on the context of the item.	1 year ago
Ilya	6f0bccfeb5	Add regex control over separators in character text splitter (#7933 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> #7854 Added the ability to use the `separator` ase a regex or a simple character. Fixed a bug where `start_index` was incorrectly counting from -1. Who can review? @eyurtsev @hwchase17 @mmz-001	1 year ago
Vasileios Mansolas	e68a1d73d0	Fix Issue #6650 : Enable Azure Active Directory token-based auth access for AzureChatOpenAI (#8622 ) When using AzureChatOpenAI the openai_api_type defaults to "azure". The utils' get_from_dict_or_env() function triggered by the root validator does not look for user provided values from environment variables OPENAI_API_TYPE, so other values like "azure_ad" are replaced with "azure". This does not allow the use of token-based auth. By removing the "default" value, this allows environment variables to be pulled at runtime for the openai_api_type and thus enables the other api_types which are expected to work. This fixes #6650 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Alec Flett	5d765408ce	propagate callbacks through load_summarize_chain (#7565 ) This lets you pass callbacks when you create the summarize chain: ``` summarize = load_summarize_chain(llm, chain_type="map_reduce", callbacks=[my_callbacks]) summary = summarize(documents) ``` See #5572 for a similar surgical fix. tagging @hwchase17 for callbacks work <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
Alec Flett	404d103c41	propagate RetrievalQA chain callbacks through its own LLMChain and StuffDocumentsChain (#7853 ) This is another case, similar to #5572 and #7565 where the callbacks are getting dropped during construction of the chains. tagging @hwchase17 and @agola11 for callbacks propagation <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
Bal Narendra Sapa	47eea32f6a	add serializer methods (#7914 ) Description: I have added two methods serializer and deserializer methods. There was method called save local but it saves the to the local disk. I wanted the vectorstore in the format using which i can push it to the sql database's blob field. I have used this while i was working on something @rlancemartin, @eyurtsev --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Ryan Sloan	b786335dd1	fix RecursiveUrlLoader (#8582 ) Description: the recursive url loader does not fully crawl for all urls under base url Maintainer: @baskaryan	1 year ago
William FH	f81e613086	Fix Async Retry Event Handling (#8659 ) It fails currently because the event loop is already running. The `retry` decorator alraedy infers an `AsyncRetrying` handler for coroutines (see [tenacity line](`aa6f8f0a24/tenacity/__init__.py (L535)`)) However before_sleep always gets called synchronously (see [tenacity line](`aa6f8f0a24/tenacity/__init__.py (L338)`)). Instead, check for a running loop and use that it exists. Of course, it's running an async method synchronously which is not _nice_. Given how important LLMs are, it may make sense to have a task list or something but I'd want to chat with @nfcampos on where that would live. This PR also fixes the unit tests to check the handler is called and to make sure the async test is run (it looks like it's just been being skipped). It would have failed prior to the proposed fixes but passes now.	1 year ago
ruze	8ef7e14a85	RSS Feed / OPML loader (#8694 ) Replace this comment with: - Description: added a document loader for a list of RSS feeds or OPML. It iterates through the list and uses NewsURLLoader to load each article. - Issue: N/A - Dependencies: feedparser, listparser - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: @ruze --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
sumandeng	53e4148a1b	add model_revison parameter to ModelScopeEmbeddings (#8669 ) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Yoshi	4e8f11b36a	Deterministic Fake Embedding Model (#8706 ) Solves #8644 This embedding models output identical random embedding vectors, given the input texts are identical. Useful when used in unittest. @baskaryan	1 year ago
Leonid Kuligin	2928a1a3c9	added minimum expected version of SDK to the error description (#8712 ) #7932 Co-authored-by: Leonid Kuligin <kuligin@google.com>	1 year ago
Harrison Chase	814faa9de5	relax deps for yaml (#8713 ) context: https://github.com/yaml/pyyaml/issues/724 I think this is fine? I don't think we use yaml too heavily	1 year ago
Holt Skinner	8a8917e0d9	feat: Add Spell Correction Spec to Google Cloud Enterprise Search connector (#8705 )	1 year ago
Bagatur	b2b71b0d35	Bagatur/eden llm (#8670 ) Co-authored-by: RedhaWassim <rwasssim@gmail.com> Co-authored-by: KyrianC <ckyrian@protonmail.com> Co-authored-by: sam <melaine.samy@gmail.com>	1 year ago
William FH	8022293124	lint (#8702 )	1 year ago
William FH	a137492b53	Permit none key in chain mapper (#8696 )	1 year ago
Bagatur	e283dc8d50	bump 251 (#8690 )	1 year ago
Eugene Yurtsev	81e0cbf2d5	Minor typo fix (#8657 ) Fix typo in doc-string.	1 year ago
Harrison Chase	43dffe39fb	Harrison/conversational retrieval agent (#8639 ) Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
ruze	71f98db2fe	Newspaper (#8647 ) - Description: Added newspaper3k based news article loader. Provide a list of urls. - Issue: N/A - Dependencies: newspaper3k, - Tag maintainer: @rlancemartin , @eyurtsev - Twitter handle: @ruze --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
shibuiwilliam	f68f3b23d7	add missing RemoteLangChainRetriever _get_relevant_documents test (#8628 ) # What - Add missing RemoteLangChainRetriever _get_relevant_documents test --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
William FH	206901fa01	Use salt instead of datetime (#8653 ) If you want to kick off two runs at the same time it'll cause errors. Use a uuid instead	1 year ago
William FH	7ea2b08d1f	Use call directly for chain (#8655 ) for run_on_dataset since the `run()` method requires a single output	1 year ago
William FH	368aa4ede7	fix enum error message (#8652 ) could be a string so don't directly call value	1 year ago
Erick Friis	96b0ff182e	Enterprise support form wording (#8641 )	1 year ago
Will Thompson	ee1d13678e	🐛 Docs Fixes [2 one-liners, examples broken] (#8519 ) ## Description: 1)Map reduce example in docs is missing an important import statement. Figured other people would benefit from being able to copy 🍝 the code. 2)RefineDocumentsChain example also broken. ## Issue: None ## Dependencies: None. One liner. ## Tag maintainer: @baskaryan ## Twitter handle: I mean, it's a one line fix lol. But @will_thompson_k is my twitter handle.	1 year ago
Kacper Łukawski	16551536e3	Refactor Qdrant integration (#8634 ) This small PR introduces new parameters into Qdrant (`on_disk`), fixes some tests and changes the error message to be more clear. Tagging: @baskaryan, @rlancemartin, @eyurtsev	1 year ago
Erick Friis	c5fb3b6069	Enterprise support form in airtable (#8607 )	1 year ago
Eugene Yurtsev	1ec0b18379	Re-add __add__ functionality for messages (revert #8245 ) (#8489 ) This PR reverts #8245, so `__add__` is defined on base messages. Resolves issue: https://github.com/langchain-ai/langchain/issues/8472	1 year ago
Bagatur	f31047a394	bump 250 (#8632 )	1 year ago
Comendeiro	5c516945d0	Add local support for audio models (PR #7329 ) (#7591 ) - Description: run the poetry dependencies - Issue: #7329 - Dependencies: any dependencies required for this change, - Tag maintainer: @rlancemartin --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Naveen Tatikonda	d2adec3818	[Opensearch] : Fix the service validation in http_auth (#8609 ) ### Description OpenSearch supports validation using both Master Credentials (Username and password) and IAM. For Master Credentials users will not pass the argument `service` in `http_auth` and the existing code will break. To fix this, I have updated the condition to check if service attribute is present in http_auth before accessing it. ### Maintainers @baskaryan @navneet1v Signed-off-by: Naveen Tatikonda <navtat@amazon.com>	1 year ago
Harrison Chase	7c5c0557cb	cast to string when measuring token length (#8617 )	1 year ago
rjanardhan3	68113348cc	Fireworks integration (#8322 ) Description - Integrates Fireworks within Langchain LLMs to allow users to use Fireworks models with Langchain, mainly for summarization. Issue - Not applicable Dependencies - None Tag maintainer - @rlancemartin --------- Co-authored-by: Raj Janardhan <rajjanardhan@Rajs-Laptop.attlocal.net>	1 year ago
Bagatur	b574507c51	normalized openai embeddings embed_query (#8604 ) we weren't normalizing when embedding queries	1 year ago
Neil Murphy	31820a31e4	Add firestore_client param to FirestoreChatMessageHistory if caller already has one; also lets them specify GCP project, etc. (#8601 ) Existing implementation requires that you install `firebase-admin` package, and prevents you from using an existing Firestore client instance if available. This adds optional `firestore_client` param to `FirestoreChatMessageHistory`, so users can just use their existing client/settings. If not passed, existing logic executes to initialize a `firestore_client`. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Naveen Tatikonda	13ccf202de	[OpenSearch] : Fix AOSS Initialization (#8600 ) ### Description This PR fixes the AOSS Initialization in Opensearch. ### Maintainers @rlancemartin, @eyurtsev, @navneet1v Signed-off-by: Naveen Tatikonda <navtat@amazon.com>	1 year ago
Joshua Carroll	6705928b9d	Add StreamlitChatMessageHistory (#8497 ) Add a StreamlitChatMessageHistory class that stores chat messages in [Streamlit's Session State](https://docs.streamlit.io/library/api-reference/session-state). Note: The integration test uses a currently-experimental Streamlit testing framework to simulate the execution of a Streamlit app. Marking this PR as draft until I confirm with the Streamlit team that we're comfortable supporting it. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
brettdbrewer	2de028834f	updated to use new llm_util query (#8591 ) - Description: added memgraph_graph.py which defines the MemgraphGraph class, subclassing off the existing Neo4jGraph class. This lets you query the Memgraph graph database using natural language. It leverages the Neo4j drivers and the bolt protocol. - Dependencies: since it is a subclass off of Neo4jGraph, it is dependent on it and the GraphCypherQA Chain implementations. It is dependent on the Neo4j drivers being present. It is dependent on having a running Memgraph instance to connect to. - Tag maintainer: @baskaryan - Twitter handle: @villageideate - example usage can be seen in this repo https://github.com/brettdbrewer/MemgraphGraph/ --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Tesfagabir Meharizghi	a7000ee89e	Callback handler for Amazon SageMaker Experiments (#8587 ) ## Description This PR implements a callback handler for SageMaker Experiments which is similar to that of mlflow. * When creating the callback handler, it takes the experiment's run object as an argument. All the callback outputs are then logged to the run object. * The output of each callback action (e.g., `on_llm_start`) is saved to S3 bucket as json file. * Optionally, you can also log additional information such as the LLM hyper-parameters to the same run object. * Once the callback object is no more needed, you will need to call the `flush_tracker()` method. This makes sure that any intermediate files are deleted. * A separate notebook example is provided to show how the callback is used. @3coins @agola11 --------- Co-authored-by: Tesfagabir Meharizghi <mehariz@amazon.com>	1 year ago
Harrison Chase	9c2b29a1cb	Harrison/loader bug (#8559 ) Co-authored-by: ddroghini <d.droghini@mflgroup.com> Co-authored-by: Buckler89 <Droghini.diego@gmail.com>	1 year ago
Kristelle Widjaja	f190bc3e83	Bug fix: feature/issue-7804-chroma-client_settings-bug (#8267 ) Description: Made Chroma constructor more robust when client_settings is provided. Otherwise, existing embeddings will not be loaded correctly from Chroma. Issue: #7804 Dependencies: None Tag maintainer: @rlancemartin, @eyurtsev --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
mpb159753	7df2dfc4c2	Add Support for Loading Documents from Huawei OBS (#8573 ) Description: This PR adds support for loading documents from Huawei OBS (Object Storage Service) in Langchain. OBS is a cloud-based object storage service provided by Huawei Cloud. With this enhancement, Langchain users can now easily access and load documents stored in Huawei OBS directly into the system. Key Changes: - Added a new document loader module specifically for Huawei OBS integration. - Implemented the necessary logic to authenticate and connect to Huawei OBS using access credentials. - Enabled the loading of individual documents from a specified bucket and object key in Huawei OBS. - Provided the option to specify custom authentication information or obtain security tokens from Huawei Cloud ECS for easy access. How to Test: 1. Ensure the required package "esdk-obs-python" is installed. 2. Configure the endpoint, access key, secret key, and bucket details for Huawei OBS in the Langchain settings. 3. Load documents from Huawei OBS using the updated document loader module. 4. Verify that documents are successfully retrieved and loaded into Langchain for further processing. Please review this PR and let us know if any further improvements are needed. Your feedback is highly appreciated! @rlancemartin, @eyurtsev --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Leonid Ganeline	ed9a0f8185	Docstrings: Module descriptions (#8262 ) Added/changed the module descriptions (the firs-line docstrings in the `__init__` files). Added class hierarchy info. @baskaryan	1 year ago
shibuiwilliam	465faab935	fix apparent spelling inconsistencies (#8574 ) Use ImportErrors where appropriate	1 year ago
Nuno Campos	0ec020698f	Add new run types for Runnables (#8488 ) - allow overriding run_type in on_chain_start <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
Bagatur	bd2e298468	bump 249 (#8571 )	1 year ago
William FH	e83250cc5f	Rm RunTypeEnum (#8553 ) We already support raw strings in the SDK but would like to deprecate client-side validation of run types. This removes its usage	1 year ago
Jacob Lee	2a26cc6d2b	Fix combining runnable sequences (#8557 ) Combining runnable sequences was dropping a step in the middle. @nfcampos @baskaryan	1 year ago
Mohamad Zamini	3fbb737bb3	Update combined.py (#7541 ) from my understanding, the `check_repeated_memory_variable` validator will raise an error if any of the variables in the `memories` list are repeated. However, the `load_memory_variables` method does not check for repeated variables. This means that it is possible for the `CombinedMemory` instance to return a dictionary of memory variables that contains duplicate values. This code will check for repeated variables in the `data` dictionary returned by the `load_memory_variables` method of each sub-memory. If a repeated variable is found, an error will be raised. <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
yangdihang	ff5024634e	fix: openapi controller prompt, when bot is unable to resolve an api … (#7525 ) …call, it needs retry <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> Co-authored-by: yangdihang <yangdihang@bytedance.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Kenny	1e8fca5518	Add ConcurrentLoader (#7512 ) Works just like the GenericLoader but concurrently for those who choose to optimize their workflow. @rlancemartin @eyurtsev --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Kevin Buckley	8061994c61	AzureSearch Vector Store: Moving the usage of additional_fields into context of it's definition (bug fix from python error) (#8551 ) Description: Using Azure Cognitive Search as a VectorStore. Calling the `add_texts` method throws an error if there is no metadata property specified. The `additional_fields` field is set in an `if` statement and then is used later outside the if statement. This PR just moves the declaration of `additional_fields` below and puts the usage of it in context. Issue: https://github.com/langchain-ai/langchain/issues/8544 Tagging @rlancemartin, @eyurtsev as this is related to Vector stores. `make format`, `make lint`, `make spellcheck`, and `make test` have been run	1 year ago
Pranay Chandekar	7e70cd2a28	Bug Fix - #8415 (#8417 ) - Issue: #8415 Signed-off-by: Pranay Chandekar <pranayc6@gmail.com>	1 year ago
shibuiwilliam	de61ebd9e0	add tests to redis vectorstore (#8116 ) # What - Add function to get similarity with score with threshold in Redis vector store. - Add tests to Redis vector store.	1 year ago
Bruno Bornsztein	5a490a79f4	fix issue #8357 by making json backtick regex greedy (#8528 ) - Description: Markdown code blocks in json response should not break the parser - Issue: #8357 @baskaryan @hinthornw	1 year ago
Gordon Clark	64d0a0fcc0	Updating docstings in utilities (#8411 ) Updating docstrings on utility packages @baskaryan	1 year ago
Mohammad Mohtashim	144b4c0c78	SQL Query Prompt update + added _execute method for SQLDatabase (#8100 ) - Description: This pull request (PR) includes two minor changes: 1. Updated the default prompt for SQL Query Checker: The current prompt does not clearly specify the final response that the LLM (Language Model) should provide when checking for the query if `use_query_checker` is enabled in SQLDatabase Chain. As a result, the LLM adds extra words like "Here is your updated query" to the response. However, this causes a syntax error when executing the SQL command in SQLDatabaseChain, as these additional words are also included in the SQL query. 2. Moved the query's execution part into a separate method for SQLDatabase: The purpose of this change is to provide users with more flexibility when obtaining the result of an SQL query in the original form returned by sqlalchemy. In the previous implementation, the run method returned the results as a string. By creating a distinct method for execution, users can now receive the results in original format, which proves helpful in various scenarios. For example, during the development of a tool, I found it advantageous to obtain results in original format rather than a string, as currently done by the run method. - Tag maintainer: @hinthornw --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Matthew DeGuzman	844eca98d5	Add LLaMa Formatter and AzureML Chat Endpoint (#8382 ) ## Description Microsoft and Meta recently [announced their collaboration](https://blogs.microsoft.com/blog/2023/07/18/microsoft-and-meta-expand-their-ai-partnership-with-llama-2-on-azure-and-windows/) on LLaMa2. This PR extends the current LLM wrapper and introduces a new Chat Model wrapper for AzureML to support LLaMa2. ## Dependencies No dependencies added :) ## Twitter Handles [@matthew_d13](https://twitter.com/matthew_d13) [@prakhar_in](https://twitter.com/prakhar_in) maintainers - @hwchase17, @baskaryan	1 year ago
Harrison Chase	15de57b848	fix web loader (#8538 )	1 year ago
Nuno Campos	4780156955	Rely less on positional arg order in subclasses of vector store when calling async methods (#8534 )	1 year ago
Harrison Chase	5e3b968078	router runnable (#8496 ) Co-authored-by: Nuno Campos <nuno@boringbits.io>	1 year ago
Anubhav Bindlish	913a156cff	Minor improvements to rockset vectorstore (#8416 ) This PR makes minor improvements to our python notebook, and adds support for `Rockset` workspaces in our vectorstore client. @rlancemartin, @eyurtsev --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	a8be207ea3	bump 248 (#8518 )	1 year ago
Harrison Chase	6556a8fcfd	add initial anthropic agent (#8468 ) Co-authored-by: Nuno Campos <nuno@boringbits.io>	1 year ago
os1ma	a795c3d860	Fix GitLoader to handle repeated load calls (#8412 ) Description: a description of the change In this pull request, GitLoader has been updated to handle multiple load calls, provided the same repository is being cloned. Previously, calling `load` multiple times would raise an error if a clone URL was provided. Additionally, a check has been added to raise a ValueError when attempting to clone a different repository into an existing path. New tests have also been introduced to verify the correct behavior of the GitLoader class when `load` is called multiple times. Lastly, the GitPython package, a dependency for the GitLoader class, has been added to the project dependencies (pyproject.toml and poetry.lock). Issue: the issue # it fixes (if applicable) None Dependencies: any dependencies required for this change GitPython Tag maintainer: for a quicker response, tag the relevant maintainer (see below) - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev	1 year ago
Piyush Jain	b2f8a5bae9	Fixed exports for NeptuneOpenCypherQAChain (#8439 ) ## Description The imports for `NeptuneOpenCypherQAChain` are failing. This PR adds the chain class to the `__init__.py` file to fix this issue. ## Maintainers @dev2049 @krlawrence	1 year ago
Eugene Yurtsev	e98e2b2b81	ChatPromptTemplate: clean up doc-string (#8473 ) Minor doc-string clean up --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Eugene Yurtsev	529cb2e30c	Update doc-string in few shot template (#8474 ) Partial update of doc-string, need to update other instances in documentation	1 year ago
Muneeb Ahmad	4923cf029a	Added Proper Documentation for `faiss-gpu` Installation (#8492 ) ### Description In the LangChain Documentation and Comments, I've Noticed that `pip install faiss` was mentioned, instead of `pip install faiss-gpu`, since installing `pip install faiss` results in an error. I've gone ahead and updated the Documentation, and `faiss.ipynb`. This Change will ensure ease of use for the end user, trying to install `faiss-gpu`. ### Issue: Documentation / Comments Related. ### Dependencies: No Dependencies we're changed only updated the files with the wrong reference. ### Tag maintainer: @rlancemartin, @eyurtsev (Thank You for your contributions 😄 )	1 year ago
shibuiwilliam	549720ae51	add test to ensure values in time weighted retriever are updated (#8479 ) # What - add test to ensure values in time weighted retriever are updated <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: add test to ensure values in time weighted retriever are updated - Issue: None - Dependencies: None - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: @MlopsJ Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
Harrison Chase	18a2452121	prompt cleanup (#8470 )	1 year ago
Harrison Chase	4d526c49ed	bump experimental to 008 (#8490 )	1 year ago
Harrison Chase	8f14ddefdf	add anthropic functions wrapper (#8475 ) a cheeky wrapper around claude that adds in function calling support (kind of, hence it going in experimental)	1 year ago
Nuno Campos	b65a9414bb	runnable.bind().bind() should combine kwargs, instead of nesting wrappers (#8467 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Nuno Campos	872abb4198	Implement Runnable for Tools (#8460 ) - Make _arun optional - Pass run_manager to inner chains in tools that have them <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
William FH	b7c0eb9ecb	Wfh/ref links (#8454 )	1 year ago
Harrison Chase	13b4f465e2	log output parser (#8446 )	1 year ago
William FH	d935573362	Partial formatting for chat messages (#8450 )	1 year ago
William FH	3314f54383	Update supabase docstrings (#8443 )	1 year ago
Harrison Chase	2448043b84	bump and fix (#8441 )	1 year ago
Amélie	8ee56b9a5b	Feature: Add support for meilisearch vectorstore (#7649 ) Description: Add support for Meilisearch vector store. Resolve #7603 - No external dependencies added - A notebook has been added @rlancemartin https://twitter.com/meilisearch Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bearnardd	b7d6e1909c	fix empty ids when metadatas is provided (#8127 ) Fixes https://github.com/hwchase17/langchain/issues/7865 and https://github.com/hwchase17/langchain/issues/8061 - [x] fixes returning empty ids when metadatas argument is provided @baskaryan --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
lvisdd	abe4c361f9	update get_num_tokens_from_messages model (#8431 ) (#8430) Co-authored-by: Kano Kunihiko <kkano@heroz.co.jp>	1 year ago
Jeffrey Wang	e0de62f6da	Add RoPE Scaling params from llamacpp (#8422 ) Description: Just adding parameters from `llama-python-cpp` that support RoPE scaling. @hwchase17, @baskaryan sources: papers and explanation: https://kaiokendev.github.io/context llamacpp conversation: https://github.com/ggerganov/llama.cpp/discussions/1965 Supports models like: https://huggingface.co/conceptofmind/LLongMA-2-13b	1 year ago
Harrison Chase	fab24457bc	remove code (#8425 )	1 year ago
Harrison Chase	3a78450883	update experimental (#8402 ) some changes were made to experimental, porting them over	1 year ago
Harrison Chase	af7e70d4af	expose function for converting messages to messages (#8426 )	1 year ago
Eugene Yurtsev	06bdbe06fe	PromptTemplate update documentation and expand kwarg (#8423 ) # PromptTemplate * Update documentation to highlight the classmethod for instantiating a prompt template. * Expand kwargs in the classmethod to make parameters easier to discover This PR got reverted here: https://github.com/langchain-ai/langchain/pull/8395/files	1 year ago
Eugene Yurtsev	e62a1686e2	ChatPromptTemplate: minor fix in doc string (#8424 ) Minor fix in doc-string to use `ai` rather than `assistant`	1 year ago
Eugene Yurtsev	760c278fe0	ChatPromptTemplate: Expand support for message formats and documentation (#8244 ) * Expands support for a variety of message formats in the `from_messages` classmethod. Ideally, we could deprecate the other on-ramps to reduce the amount of classmethods users need to know about. * Expand documentation with code examples.	1 year ago
Bagatur	61dd92f821	bump 246 (#8410 )	1 year ago
Harrison Chase	394b67ab92	add kwargs to llm runnables (#8388 )	1 year ago
HeTaoPKU	d5884017a9	Add Minimax llm model to langchain (#7645 ) - Description: Minimax is a great AI startup from China, recently they released their latest model and chat API, and the API is widely-spread in China. As a result, I'd like to add the Minimax llm model to Langchain. - Tag maintainer: @hwchase17, @baskaryan --------- Co-authored-by: the <tao.he@hulu.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
James Campbell	0ad2d5f27a	[nit] Add default value for ChatOpenAI client (#7939 ) Micro convenience PR to avoid warning regarding missing `client` parameter. It is always set during initialization. @baskaryan Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Harrison Chase	82df923f37	Merge branch 'master' of github.com:hwchase17/langchain	1 year ago
Harrison Chase	1b0bfa54cf	cr	1 year ago
Jeff Vestal	c7ff5f19a8	ElasticKnnSearch rewrite - bug fix - return Document (#8180 ) Fixes: https://github.com/hwchase17/langchain/issues/7117 https://github.com/hwchase17/langchain/issues/5760 Adding back `create_index` , `add_texts`, `from_texts` to ElasticKnnSearch `from_texts` matches standard `from_texts` methods as quick start up method `knn_search` and `hybrid_result` return a list of [`Document()`, `score`,] # Test `from_texts` for quick start ``` # create new index using from_text from langchain.vectorstores.elastic_vector_search import ElasticKnnSearch from langchain.embeddings import ElasticsearchEmbeddings model_id = "sentence-transformers__all-distilroberta-v1" dims = 768 es_cloud_id = "" es_user = "" es_password = "" test_index = "knn_test_index_305" embeddings = ElasticsearchEmbeddings.from_credentials( model_id, #input_field=input_field, es_cloud_id=es_cloud_id, es_user=es_user, es_password=es_password, ) # add texts and create class instance texts = ["This is a test document", "This is another test document"] knnvectorsearch = ElasticKnnSearch.from_texts( texts=texts, embedding=embeddings, index_name= test_index, vector_query_field='vector', query_field='text', model_id=model_id, dims=dims, es_cloud_id=es_cloud_id, es_user=es_user, es_password=es_password ) # Test `add_texts` method texts2 = ["Hello, world!", "Machine learning is fun.", "I love Python."] knnvectorsearch.add_texts(texts2) query = "Hello" knn_result = knnvectorsearch.knn_search(query = query, model_id= model_id, k=2) hybrid_result = knnvectorsearch.knn_hybrid_search(query = query, model_id= model_id, k=2) ``` The mapping is as follows: ``` { "knn_test_index_012": { "mappings": { "properties": { "text": { "type": "text" }, "vector": { "type": "dense_vector", "dims": 768, "index": true, "similarity": "dot_product" } } } } } ``` # Check response type ``` >>> hybrid_result [(Document(page_content='Hello, world!', metadata={}), 0.94232327), (Document(page_content='I love Python.', metadata={}), 0.5321523)] >>> hybrid_result[0] (Document(page_content='Hello, world!', metadata={}), 0.94232327) >>> hybrid_result[0][0] Document(page_content='Hello, world!', metadata={}) >>> type(hybrid_result[0][0]) <class 'langchain.schema.document.Document'> ``` # Test with existing Index ``` from langchain.vectorstores.elastic_vector_search import ElasticKnnSearch from langchain.embeddings import ElasticsearchEmbeddings ## Initialize ElasticsearchEmbeddings model_id = "sentence-transformers__all-distilroberta-v1" dims = 768 es_cloud_id = es_user = "" es_password = "" test_index = "knn_test_index_012" embeddings = ElasticsearchEmbeddings.from_credentials( model_id, es_cloud_id=es_cloud_id, es_user=es_user, es_password=es_password, ) ## Initialize ElasticKnnSearch knn_search = ElasticKnnSearch( es_cloud_id=es_cloud_id, es_user=es_user, es_password=es_password, index_name= test_index, embedding= embeddings ) ## Test adding vectors ### Test `add_texts` method when index created texts = ["Hello, world!", "Machine learning is fun.", "I love Python."] knn_search.add_texts(texts) ``` --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Harrison Chase	a221a9ced0	Harrison/sql query (#8370 ) Co-authored-by: Nuno Campos <nuno@boringbits.io>	1 year ago
Bagatur	a1a650c743	Bagatur/from texts bug fix (#8394 ) --------- Co-authored-by: Davit Buniatyan <davit@loqsh.com> Co-authored-by: Davit Buniatyan <d@activeloop.ai> Co-authored-by: adilkhan <adilkhan.sarsen@nu.edu.kz> Co-authored-by: Ivo Stranic <istranic@gmail.com>	1 year ago
Jiayi Ni	1efb9bae5f	FEAT: Integrate Xinference LLMs and Embeddings (#8171 ) - [Xorbits Inference(Xinference)](https://github.com/xorbitsai/inference) is a powerful and versatile library designed to serve language, speech recognition, and multimodal models. Xinference supports a variety of GGML-compatible models including chatglm, whisper, and vicuna, and utilizes heterogeneous hardware and a distributed architecture for seamless cross-device and cross-server model deployment. - This PR integrates Xinference models and Xinference embeddings into LangChain. - Dependencies: To install the depenedencies for this integration, run `pip install "xinference[all]"` - Example Usage: To start a local instance of Xinference, run `xinference`. To deploy Xinference in a distributed cluster, first start an Xinference supervisor using `xinference-supervisor`: `xinference-supervisor -H "${supervisor_host}"` Then, start the Xinference workers using `xinference-worker` on each server you want to run them on. `xinference-worker -e "http://${supervisor_host}:9997"` To use Xinference with LangChain, you also need to launch a model. You can use command line interface (CLI) to do so. Fo example: `xinference launch -n vicuna-v1.3 -f ggmlv3 -q q4_0`. This launches a model named vicuna-v1.3 with `model_format="ggmlv3"` and `quantization="q4_0"`. A model UID is returned for you to use. Now you can use Xinference with LangChain: ```python from langchain.llms import Xinference llm = Xinference( server_url="http://0.0.0.0:9997", # suppose the supervisor_host is "0.0.0.0" model_uid = {model_uid} # model UID returned from launching a model ) llm( prompt="Q: where can we visit in the capital of France? A:", generate_config={"max_tokens": 1024}, ) ``` You can also use RESTful client to launch a model: ```python from xinference.client import RESTfulClient client = RESTfulClient("http://0.0.0.0:9997") model_uid = client.launch_model(model_name="vicuna-v1.3", model_size_in_billions=7, quantization="q4_0") ``` The following code block demonstrates how to use Xinference embeddings with LangChain: ```python from langchain.embeddings import XinferenceEmbeddings xinference = XinferenceEmbeddings( server_url="http://0.0.0.0:9997", model_uid = model_uid ) ``` ```python query_result = xinference.embed_query("This is a test query") ``` ```python doc_result = xinference.embed_documents(["text A", "text B"]) ``` Xinference is still under rapid development. Feel free to [join our Slack community](https://xorbitsio.slack.com/join/shared_invite/zt-1z3zsm9ep-87yI9YZ_B79HLB2ccTq4WA) to get the latest updates! - Request for review: @hwchase17, @baskaryan - Twitter handle: https://twitter.com/Xorbitsio --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	877d384bc9	Revert "PromptTemplate update documentation and expand kwargs (#8234 )" (#8395 ) fyi @eyurtsev was failing a unit test	1 year ago
Gordon Clark	e66759cc9d	Github add "Create PR" tool + Docs update (#8235 ) Added a new tool to the Github toolkit called Create Pull Request. Now we can make our own langchain contributor in langchain 😁 In order to have somewhere to pull from, I also added a new env var, "GITHUB_BASE_BRANCH." This will allow the existing env var, "GITHUB_BRANCH," to be a working branch for the bot (so that it doesn't have to always commit on the main/master). For example, if you want the bot to work in a branch called `bot_dev` and your repo base is `main`, you would set up the vars like: ``` GITHUB_BASE_BRANCH = "main" GITHUB_BRANCH = "bot_dev" ``` Maintainer responsibilities: - Agents / Tools / Toolkits: @hinthornw	1 year ago
William FH	ecd4aae818	Few Shot Chat Prompt (#8038 ) Proposal for a few shot chat message example selector --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	1 year ago
Eugene Yurtsev	6dd18eee26	PromptTemplate update documentation and expand kwargs (#8234 ) # PromptTemplate * Update documentation to highlight the classmethod for instantiating a prompt template. * Expand kwargs in the classmethod to make parameters easier to discover	1 year ago
Karan V	a003a0baf6	fix(petals) allows to run models that aren't Bloom (Support for LLama and newer models) (#8356 ) In this PR: - Removed restricted model loading logic for Petals-Bloom - Removed petals imports (DistributedBloomForCausalLM, BloomTokenizerFast) - Instead imported more generalized versions of loader (AutoDistributedModelForCausalLM, AutoTokenizer) - Updated the Petals example notebook to allow for a successful installation of Petals in Apple Silicon Macs - Tag maintainer: @hwchase17, @baskaryan --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
lars.gersmann	e758e9e7f5	fix(openapi): openapi chain will work without/empty description/summa… (#8351 ) Description: This PR will enable the Open API chain to work with valid Open API specifications missing `description` and `summary` properties for path and operation nodes in open api specs. Since both `description` and `summary` property are declared optional we cannot be sure they are defined. This PR resolves this problem by providing an empty (`''`) description as fallback. The previous behavior of the Open API chain was that the underlying LLM (OpenAI) throw ed an exception since `None` is not of type string: ``` openai.error.InvalidRequestError: None is not of type 'string' - 'functions.0.description' ``` Using this PR the Open API chain will succeed also using Open API specs lacking `description` and `summary` properties for path and operation nodes. Thanks for your amazing work ! Tag maintainer: @baskaryan --------- Co-authored-by: Lars Gersmann <lars.gersmann@cm4all.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
ljeagle	caa6caeb8a	Upgrade the AwaDB from v0.3.7 to v0.3.9 and change the default embeddings (#8281 ) 1. Upgrade the AwaDB from v0.3.7 to v0.3.9 2. Change the default embedding to AwaEmbedding --------- Co-authored-by: ljeagle <awadb.vincent@gmail.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Holt Skinner	d7e6770de8	refactor: Code refactoring & simplification for Google Cloud Enterprise Search retriever (#8369 ) Followup to https://github.com/langchain-ai/langchain/pull/7857 - Changes `_convert_search_response()` to use object attributes instead of converting to dictionary - Simplifies logic for readability	1 year ago
Taozhi Wang	594f195e54	Add embeddings for AwaEmbedding (#8353 ) - Description: Adds AwaEmbeddings class for embeddings, which provides users with a convenient way to do fine-tuning, as well as the potential need for multimodality - Tag maintainer: @baskaryan Create `Awa.ipynb`: an example notebook for AwaEmbeddings class Modify `embeddings/__init__.py`: Import the class Create `embeddings/awa.py`: The embedding class Create `embeddings/test_awa.py`: The test file. --------- Co-authored-by: taozhiwang <taozhiwa@gmail.com>	1 year ago
thehunmonkgroup	ba4e82bb47	fix missing _identifying_params() in _VertexAICommon (#8303 ) Full set of params are missing from Vertex* LLMs when `dict()` method is called. ``` >>> from langchain.chat_models.vertexai import ChatVertexAI >>> from langchain.llms.vertexai import VertexAI >>> chat_llm = ChatVertexAI() l>>> llm = VertexAI() >>> chat_llm.dict() {'_type': 'vertexai'} >>> llm.dict() {'_type': 'vertexai'} ``` This PR just uses the same mechanism used elsewhere to expose the full params. Since `_identifying_params()` is on the `_VertexAICommon` class, it should cover the chat and non-chat cases.	1 year ago
Caitlin2694	b2e4b9dca4	Fix exception caused by restrictions in OWL (#8341 ) Description: Fix exception caused by restrictions in OWL Issue: #8331 Dependencies: none Maintainer: @baskaryan	1 year ago
Nikita Pokidyshev	f499e6ea6a	Add FunctionMessage to _message_from_dict (#8374 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
emarco177	2ab13ab743	added unit tests for mrkl output_parser.py (#8321 ) - Description: added unit tests for mrkl output_parser.py, - Tag maintainer: @hinthornw - Twitter handle: EdenEmarco177	1 year ago
Harrison Chase	f5bf893035	rename to str output parser (#8373 )	1 year ago
William FH	0e9e5b5202	Retry events on any run type (#8375 )	1 year ago
William FH	ff98fad2d9	Add Retry Events (#8053 ) ![image](https://github.com/hwchase17/langchain/assets/13333726/59a5c3b4-4367-47e6-9f58-5b6557576a8a) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Nuno Campos	0eca3e7d90	Add Runnable.bind method to attach kwargs to a Runnable that will be passed to all invoke/stream/batch calls when it is run (#8368 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
Nuno Campos	1bbadde77b	Support using RunnableMap directly (#8317 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
Bagatur	944321c6ab	bump 245 (#8359 )	1 year ago
Rubén Barragán	ef6332ead6	Support loading files from Dropbox (#8271 ) ## Description This commit introduces the `DropboxLoader` class, a new document loader that allows loading files from Dropbox into the application. The loader relies on a Dropbox app, which requires creating an app on Dropbox, obtaining the necessary scope permissions, and generating an access token. Additionally, the dropbox Python package is required. The `DropboxLoader` class is designed to be used as a document loader for processing various file types, including text files, PDFs, and Dropbox Paper files. ## Dependencies `pip install dropbox` and `pip install unstructured` for PDF reading. ## Tag maintainer @rlancemartin, @eyurtsev (from Data Loaders). I'd appreciate some feedback here 🙏 . ## Social Networks https://github.com/rubenbarragan https://www.linkedin.com/in/rgbarragan/ https://twitter.com/RubenBarraganP --------- Co-authored-by: Ruben Barragan <rbarragan@Rubens-MacBook-Air.local>	1 year ago
Pranay Chandekar	41bb3a6f9b	fixed the bug #8343 (#8345 ) - Issue: #8343 Signed-off-by: Pranay Chandekar <pranayc6@gmail.com>	1 year ago
Martin Krasser	93260a9922	Fix broken `make` targets `format_diff` and `lint_diff` (#8344 ) Since the refactoring into sub-projects `libs/langchain` and `libs/experimental`, the `make` targets `format_diff` and `lint_diff` do not work anymore when running `make` from these subdirectories. Reason is that ``` PYTHON_FILES=$(shell git diff --name-only --diff-filter=d master \| grep -E '\.py$$\|\.ipynb$$') ``` generates paths from the project's root directory instead of the corresponding subdirectories. This PR fixes this by adding a `--relative` command line option. - Tag maintainer: @baskaryan	1 year ago
Harrison Chase	ae78ef7fe6	bump experimental to 005 (#8339 )	1 year ago
Vadim Gubergrits	e7e5cb9d08	Tree of Thought introducing a new ToTChain. (#5167 ) # [WIP] Tree of Thought introducing a new ToTChain. This PR adds a new chain called ToTChain that implements the ["Large Language Model Guided Tree-of-Though"](https://arxiv.org/pdf/2305.08291.pdf) paper. There's a notebook example `docs/modules/chains/examples/tot.ipynb` that shows how to use it. Implements #4975 ## Who can review? Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested: - @hwchase17 - @vowelparrot --------- Co-authored-by: Vadim Gubergrits <vgubergrits@outbox.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
William FH	9eb7e6e27f	Delete Old Evals Examples (#8252 ) Still retain: - Comparison Examples - Data + QA walkthrough - QA (but really minimize it)	1 year ago
Saurabh Misra	db9d5b213a	Optimize the cosine_similarity_top_k function performance (#8151 ) Optimizing important numerical code and making it run faster. Performance went up by 1.48x (148%). Runtime went down from 138715us to 56020us Optimization explanation: The `cosine_similarity_top_k` function is where we made the most significant optimizations. Instead of sorting the entire score_array which needs considering all elements, `np.argpartition` is utilized to find the top_k largest scores indices, this operation has a time complexity of O(n), higher performance than sorting. Remember, `np.argpartition` doesn't guarantee the order of the values. So we need to use argsort() to get the indices that would sort our top-k values after partitioning, which is much more efficient because it only sorts the top-K elements, not the entire array. Then to get the row and column indices of sorted top_k scores in the original score array, we use `np.unravel_index`. This operation is more efficient and cleaner than a list comprehension. The code has been tested for correctness by running the following snippet on both the original function and the optimized function and averaged over 5 times. ``` def test_cosine_similarity_top_k_large_matrices(): X = np.random.rand(1000, 1000) Y = np.random.rand(1000, 1000) top_k = 100 score_threshold = 0.5 gc.disable() counter = time.perf_counter_ns() return_value = cosine_similarity_top_k(X, Y, top_k, score_threshold) duration = time.perf_counter_ns() - counter gc.enable() ``` @hwaking @hwchase17 @jerwelborn Unit tests pass, I also generated more regression tests which all passed.	1 year ago
Fabrizio Ruocco	ddc353a768	Azure Cognitive Search: Custom index and scoring profile support (#6843 ) Description: Adding support for custom index and scoring profile support in Azure Cognitive Search @hwchase17 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Kacper Łukawski	c5988c1d4b	Implement async support for Cohere (#8237 ) This PR introduces async API support for Cohere, both LLM and embeddings. It requires updating `cohere` package to `^4`. Tagging @hwchase17, @baskaryan, @agola11 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Daniel Alexander Brenot	bf1357f584	Added async support to PlanAndExecute Chain (#8239 ) - Description: Adds async support to the PlanAndExecute Chain Maintainer responsibilities: - Async: @agola11 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bastin Florian	a3ac9b23eb	feat(confluence): add markdown format option (#8246 ) # Description: Add the possibility to keep text as Markdown in the ConfluenceLoader Add a bool variable that allows to keep the Markdown format of the Confluence pages. It is useful because it allows to use MarkdownHeaderTextSplitter as a DataSplitter. If this variable in set to True in the load() method, the pages are extracted using the markdownify library. # Issue: [4407](https://github.com/langchain-ai/langchain/issues/4407) # Dependencies: Add the markdownify library # Tag maintainer: @rlancemartin, @eyurtsev # Twitter handle: FloBastinHeyI - https://twitter.com/FloBastinHeyI --------- Co-authored-by: Florian Bastin <florian.bastin@octo.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Leonid Ganeline	ee6ff96e28	docstrings cleanup (#8311 ) - added missed docstrings - changed docstrings into consistent format @baskaryan	1 year ago
Rohit Gupta	e5dba8978a	Avoid re-computation of embedding in weaviate similarity search (#8284 ) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Nuno Campos	a612800ef0	Runnable single protocol (#7800 ) Objects implementing Runnable: BasePromptTemplate, LLM, ChatModel, Chain, Retriever, OutputParser - [x] Implement Runnable in base Retriever - [x] Raise TypeError in operator methods for unsupported things - [x] Implement dict which calls values in parallel and outputs dict with results - [x] Merge in `+` for prompts - [x] Confirm precedence order for operators, ideal would be `+` `\|`, https://docs.python.org/3/reference/expressions.html#operator-precedence - [x] Add support for openai functions, ie. Chat Models must return messages - [x] Implement BaseMessageChunk return type for BaseChatModel, a subclass of BaseMessage which implements __add__ to return BaseMessageChunk, concatenating all str args - [x] Update implementation of stream/astream for llm and chat models to use new `_stream`, `_astream` optional methods, with default implementation in base class `raise NotImplementedError` use https://stackoverflow.com/a/59762827 to see if it is implemented in base class - [x] Delete the IteratorCallbackHandler (leave the async one because people using) - [x] Make BaseLLMOutputParser implement Runnable, accepting either str or BaseMessage --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	1 year ago
Bharat	04a4d3e312	Fixes #8310 Fix maximum recursion depth exceeded error (#8313 ) ElasticsearchVectorStore.as_retriever() method is returning `RecursionError: maximum recursion depth exceeded` because of incorrect field reference in `embeddings()` method - Description: Fix RecursionError because of a typo - Issue: the issue #8310 - Dependencies: None, - Tag maintainer: @eyurtsev - Twitter handle: bpatel	1 year ago
Caitlin2694	b9db3dd09b	Fix "missing key op" RDFGraph OWL serialization (#8276 ) Replace this comment with: - Description: Fix "missing key op" error in RDFGraph OWL Serialization - Issue: #8263 - Dependencies: None - Tag maintainer: @baskaryan	1 year ago

... 2 3 4 5 6 ...

420 Commits (de8dfde7f75f0b6ade0629647e67f53b595230b5)