58 Commits (75ba22793fcd56bcc983811323e0490db477cd6a)

Author	SHA1	Message	Date
Erick Friis	75ba22793f	community: Vectara summarization (#14970) ... Description: Adding Summarization to Vectara, to reflect it provides not only vector-store type functionality but also can return a summary. Also added: MMR capability (in the Vectara platform side) Updated templates Updated documentation and IPYNB examples Tag maintainer: @baskaryan Twitter handle: @ofermend --------- Co-authored-by: Ofer Mendelevitch <ofermend@gmail.com>	9 months ago
Liang Zhang	6479aab74f	community[patch]: Add param "task" to Databricks LLM to work around serialization of transform_output_fn (#14933) ... **What is the reproduce code?** ```python from langchain.chains import LLMChain, load_chain from langchain.llms import Databricks from langchain.prompts import PromptTemplate def transform_output(response): # Extract the answer from the responses. return str(response["candidates"][0]["text"]) def transform_input(**request): full_prompt = f"""{request["prompt"]} Be Concise. """ request["prompt"] = full_prompt return request chat_model = Databricks( endpoint_name="llama2-13B-chat-Brambles", transform_input_fn=transform_input, transform_output_fn=transform_output, verbose=True, ) print(f"Test chat model: {chat_model('What is Apache Spark')}") # This works llm_chain = LLMChain(llm=chat_model, prompt=PromptTemplate.from_template("{chat_input}")) llm_chain("colorful socks") # this works llm_chain.save("databricks_llm_chain.yaml") # transform_input_fn and transform_output_fn are not serialized into the model yaml file loaded_chain = load_chain("databricks_llm_chain.yaml") # The Databricks LLM is recreated with transform_input_fn=None, transform_output_fn=None. loaded_chain("colorful socks") # Thus this errors. The transform_output_fn is needed to produce the correct output ``` Error: ``` File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-6c34afab-3473-421d-877f-1ef18930ef4d/lib/python3.10/site-packages/pydantic/v1/main.py", line 341, in __init__ raise validation_error pydantic.v1.error_wrappers.ValidationError: 1 validation error for Generation text str type expected (type=type_error.str) request payload: {'query': 'What is a databricks notebook?'}'} ``` **What does the error mean?** When the LLM generates an answer, represented by a Generation data object. The Generation data object takes a str field called text, e.g. Generation(text=”blah”). However, the Databricks LLM tried to put a non-str to text, e.g. Generation(text={“candidates”:[{“text”: “blah”}]}) Thus, pydantic errors. **Why the output format becomes incorrect after saving and loading the Databricks LLM?** Databrick LLM does not support serializing transform_input_fn and transform_output_fn, so they are not serialized into the model yaml file. When the Databricks LLM is loaded, it is recreated with transform_input_fn=None, transform_output_fn=None. Without transform_output_fn, the output text is not unwrapped, thus errors. Missing transform_output_fn causes this error. Missing transform_input_fn causes the additional prompt “Be Concise.” to be lost after saving and loading. <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/ If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
Bagatur	b03845e069	community[patch]: Release 0.0.5 (#14960)	9 months ago
Anush	60c70effe9	community[minor]: Qdrant sparse vector retriever (#14814) ... ## Description This PR intends to add support for Qdrant's new [sparse vector retrieval](https://qdrant.tech/articles/sparse-vectors/) by introducing a new retriever class, `QdrantSparseVectorRetriever`. Necessary usage docs and integration tests have been added for the retriever. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
mogith-pn	c53fab63a3	community[patch]: Fixed duplicate input id issue in clarifai vectorstore (#14914) ... - **Description:** This PR fixes the issue faces with duplicate input id in Clarifai vectorstore class when ingesting documents into the vectorstore more than the batch size. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
Sypherd	5642132c0c	community[patch]: Add safe lookup to OpenAI response adapter (#14765) ... ## Description Similar to https://github.com/langchain-ai/langchain/issues/5861, I've experienced `KeyError`s resulting from unsafe lookups in the `convert_dict_to_message` function in [this file](https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/adapters/openai.py). While that issue focused on `KeyError 'content'`, I've opened another issue (#14764) about how the problem still exists in the same function but with `KeyError 'role'`. The fix for #5861 only added a safe lookup to the specific line that was giving them trouble.. This PR fixes the unsafe lookup in the rest of the function but the problem still exists across the repo. ## Issues * #14764 * #5861 ## Dependencies * None ## Checklist [x] make format [x] make lint [ ] make test - Results in `make: *** No rule to make target 'test'. Stop.` ## Maintainers * @hinthornw --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
AlpinDale	b0588774f1	community[minor]: Add Aphrodite Engine support (#14759) ... This PR adds support for PygmalionAI's [Aphrodite Engine](https://github.com/PygmalionAI/aphrodite-engine), based on vLLM's attention mechanism. At the moment, this PR does not include support for the API servers, but they will be added in a later PR. The only dependency as of now is `aphrodite-engine==0.4.2`. We pin the version to prevent breakage due to changes in the aphrodite-engine library. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
Dmitry Tyumentsev	d21f44b484	community[minor]: Add YandexGPT embeddings (#14767) ... - **Description:** Introducing an ability to work with the [YandexGPT](https://cloud.yandex.com/en/services/yandexgpt) embeddings models. --------- Co-authored-by: Dmitry Tyumentsev <dmitry.tyumentsev@raftds.com>	9 months ago
Nicolas Suzor	529144649e	community[patch]: add png support for vertexai._parse_chat_history_gemini() (#14788) ... - **Description:** Modify community chat model vertexai to handle png and other image types encoded in base64 - **Dependencies:** added `import re` but no new dependencies. This addresses a problem where the vertexai method _parse_chat_history_gemini() was only recognizing image uris in jpeg format. I made a simple change to cover other extension types.	9 months ago
Liu Jun	b0c48dc983	community[patch]: make ak and sk optional in qianfan endpoint (#14835) ... - **Description:** The Qianfan SDK offers multiple authentication methods, but in the `QianfanEndpoint` of Langchain, it currently only supports authentication through AK and SK. In order to accommodate users who wish to use alternative authentication methods, this pull request makes AK and SK optional. This change should not impact existing users, while allowing users to configure other authentication methods as per the Qianfan SDK documentation. - **Issue:** / - **Dependencies:** No - **Tag maintainer:** No - **Twitter handle:**	9 months ago
Archan Ghosh	65678b3816	community[patch]: Update arxiv.py with Entry ID as a return value (#14915) ... Added Entry ID as a return value inside get_summaries_as_docs - **Description:** Added the Entry ID as a return, so it's easier to track the IDs of the papers that are being returned. With the addition return of the entry ID in functions like ArxivRetriever, it will be easier to reference the ID of the paper itself.	9 months ago
Bagatur	345acb26ac	community[patch]: Matching engine, return doc id (#14930)	9 months ago
Michael Feil	7b96de3d5d	community[patch]: update Gradient embeddings (#14846) ... - **Description:** Going forward, we have a own API `pip install gradientai`. Therefore gradually removing the self-build packages in llamaindex, haystack and langchain. - **Issue:** None. - **Dependencies:** `pip install gradientai` - **Tag maintainer:** @michaelfeil	9 months ago
Igor Dvorkin	6cc3c2452c	community[patch]: Enhance iMessage chat loader with timestamp parsing and message ownership (#14804) ... --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
Mohammad Mohtashim	e3abe12243	community[patch]: helpful error message for GitHubAPIWrapper (#14803) ... Very simple change in relation to the issue https://github.com/langchain-ai/langchain/issues/14550 @baskaryan, @eyurtsev, @hwchase17. --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
Dmitry Tyumentsev	50381abc42	community[patch]: Add retry logic to Yandex GPT API Calls (#14907) ... **Description:** Added logic for re-calling the YandexGPT API in case of an error --------- Co-authored-by: Dmitry Tyumentsev <dmitry.tyumentsev@raftds.com>	9 months ago
Sirjanpreet Singh Banga	425e5e1791	community[minor]: rename ChatGPTRouter to GPTRouter (#14913) ... **Description:**: Rename integration to GPTRouter **Tag maintainer:** @Gupta-Anubhav12 @samanyougarg @sirjan-ws-ext **Twitter handle:** [@SamanyouGarg](https://twitter.com/SamanyouGarg)	9 months ago
JaguarDB	992b04e475	community[minor]: added jaguar vector store (#14838) ... Description: A new vector store Jaguar is being added. Class, test scripts, and documentation is added. Issue: None -- This is the first PR contributing to LangChain Dependencies: This depends on "pip install -U jaguardb-http-client" client http package Tag maintainer: @baskaryan, @eyurtsev, @hwchase1 Twitter handle: @workbot --------- Co-authored-by: JY <jyjy@jaguardb> Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
Sirjanpreet Singh Banga	44cb899a93	community[minor]: Integrating GPTRouter (#14900) ... **Description:** Adding a langchain integration for [GPTRouter](https://gpt-router.writesonic.com/) 🚀 , **Tag maintainer:** @Gupta-Anubhav12 @samanyougarg @sirjan-ws-ext **Twitter handle:** [@SamanyouGarg](https://twitter.com/SamanyouGarg) Integration Tests Passing: <img width="1137" alt="Screenshot 2023-12-19 at 5 45 31 PM" src="https://github.com/Writesonic/langchain/assets/151817113/4a59df9a-ee30-47aa-9df9-b8c4eeb9dc76">	9 months ago
Leonid Ganeline	b2fd41331e	docs: docstrings `langchain_community` update (#14889) ... Addded missed docstrings. Fixed inconsistency in docstrings. **Note** CC @efriis There were PR errors on `langchain_experimental/prompt_injection_identifier/hugging_face_identifier.py` But, I didn't touch this file in this PR! Can it be some cache problems? I fixed this error.	9 months ago
abhjaw	6fbd068b3f	Update kendra.py to avoid Kendra query ValidationException (#14866) ... Fixing issue - https://github.com/langchain-ai/langchain/issues/14494 to avoid Kendra query ValidationException <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** Update kendra.py to avoid Kendra query ValidationException, - **Issue:** the issue #https://github.com/langchain-ai/langchain/issues/14494, - **Dependencies:** None, - **Tag maintainer:** , - **Twitter handle:** If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Erick Friis	5f839beab9	community: replace deprecated davinci models (#14860) ... This is technically a breaking change because it'll switch out default models from `text-davinci-003` to `gpt-3.5-turbo-instruct`, but OpenAI is shutting off those endpoints on 1/4 anyways. Feels less disruptive to switch out the default instead.	9 months ago
Bagatur	61ad0e8be9	community[patch]: Release 0.0.4 (#14864)	9 months ago
Bob Lin	5de1dc72b9	community[patch]: Update Tongyi default model_name (#14844) ... <img width="1305" alt="Screenshot 2023-12-18 at 9 54 01 PM" src="https://github.com/langchain-ai/langchain/assets/10000925/c943fd81-cd48-46eb-8dff-4680424d9ba9"> The current model is no longer available.	9 months ago
Vlad Kolesnikov	11fda490ca	community[minor]: New model parameters and dynamic batching for VertexAIEmbeddings (#13999) ... - **Description:** VertexAIEmbeddings performance improvements - **Twitter handle:** @vladkol ## Improvements - Dynamic batch size, starting from 250, lowering down to 5. Batch size varies across regions. Some regions support larger batches, and it significantly improves performance. When running large batches of texts in `us-central1`, performance gain can be up to 3.5x. The dynamic batching also makes sure every batch is below 20K token limit. - New model parameter `embeddings_type` that translates to `task_type` parameter of the API. Newer model versions support [different embeddings task types](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings#api_changes_to_models_released_on_or_after_august_2023).	9 months ago
William FH	2d91d2b978	community: Add logprobs in gen output (#14826) ... Now that it's supported again for OAI chat models . Shame this wouldn't include it in the `.invoke()` output though (it's not included in the message itself). Would need to do a follow-up for that to be the case	9 months ago
Dmitry Tyumentsev	78ae276df7	community[patch]: fix agenerate return value (#14815) ... Fixed: - `_agenerate` return value in the YandexGPT Chat Model - duplicate line in the documentation Co-authored-by: Dmitry Tyumentsev <dmitry.tyumentsev@raftds.com>	9 months ago
sujeet	f1d3f29bc4	community[patch]: support for Sybase SQL anywhere added. (#14821) ... - **Description:** support for Sybase SQL anywhere added in sql_database.py file at path langchain\libs\community\langchain_community\utilities - **Issue:** It will resolve default schema setting for Sybase SQL anywhere - **Dependencies:** No, - **Tag maintainer:** @baskaryan, @eyurtsev, @hwchase17, - **Twitter handle:** NA --------- Co-authored-by: learn360sujeet <121271779+learn360sujeet@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
Erick Friis	8a07c56313	docs: developer docs (#14776) ... Builds out a developer documentation section in the docs - Links it from contributing.md - Adds an initial guide on how to contribute an integration --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
Noah Stapp	34e6f3ff72	community[patch]: Implement similarity_score_threshold for MongoDB Vector Store (#14740) ... Adds the option for `similarity_score_threshold` when using `MongoDBAtlasVectorSearch` as a vector store retriever. Example use: ``` vector_search = MongoDBAtlasVectorSearch.from_documents(...) qa_retriever = vector_search.as_retriever( search_type="similarity_score_threshold", search_kwargs={ "score_threshold": 0.5, } ) qa = RetrievalQA.from_chain_type( llm=OpenAI(), chain_type="stuff", retriever=qa_retriever, ) docs = qa({"query": "..."}) ``` I've tested this feature locally, using a MongoDB Atlas Cluster with a vector search index.	9 months ago
Dmitry Tyumentsev	dcead816df	community[patch]: Update YandexGPT API (#14773) ... Update LLMand Chat model to use new api version --------- Co-authored-by: Dmitry Tyumentsev <dmitry.tyumentsev@raftds.com>	9 months ago
Lance Martin	42421860bc	Add image support for Ollama (#14713) ... Support [LLaVA](https://ollama.ai/library/llava): * Upgrade Ollama * `ollama pull llava` Ensure compatibility with [image prompt template](https://github.com/langchain-ai/langchain/pull/14263) --------- Co-authored-by: jacoblee93 <jacoblee93@gmail.com>	9 months ago
Harrison Chase	16399fd61d	langchain[patch]: remove unused imports (#14680) ... Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
Karim Lalani	a0064330b1	community[minor]: Add SurrealDB vectorstore (#13331) ... **Description:** Vectorstore implementation around [SurrealDB](https://www.surrealdb.com) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
William FH	4855964332	Fix OAI Tool Message (#14746) ... See format here: https://platform.openai.com/docs/guides/function-calling/parallel-function-calling It expects a "name" argument, which we aren't providing by default.  Alternative is to add the 'name' field directly to the message if people prefer.	9 months ago
William FH	93c7eb4e6b	[Tracing] String Stacktrace (#14131) ... Add full stacktrace	9 months ago
Leonid Kuligin	7f42811e14	google-genai[patch], community[patch]: Added support for new Google GenerativeAI models (#14530) ... Replace this entire comment with: - **Description:** added support for new Google GenerativeAI models - **Twitter handle:** lkuligin --------- Co-authored-by: Erick Friis <erick@langchain.dev>	9 months ago
Erick Friis	9fb26a2a71	community[patch]: fix pgvector sqlalchemy (#14726) ... Fixes #14699	9 months ago
Funkeke	ea99612caa	community[patch]: fix dashvector endpoint params error (#14484) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> Co-authored-by: fangkeke <3339698829@qq.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Bob Lin	dce3c74905	community[patch]: Correct type annotation for azure_ad_token_provider Closed: #14402 (#14432) ... Description Fix https://github.com/langchain-ai/langchain/issues/14402, Similar changes: https://github.com/langchain-ai/langchain/pull/14166 Twitter handle [lin_bob57617](https://twitter.com/lin_bob57617)	9 months ago
Fran Cirka	8a4162d15e	community[patch]: Fixed issue with importing Row from sqlalchemy (#14488) ... - **Description:** Fixed import of Row in cache.py, - **Issue:** the issue # #13464 https://creditone.us.to/langchain-ai/langchain/issues/13464, - **Dependencies:** None, - **Twitter handle:** @frankybridman Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Bagatur	4574749147	communty[patch]: Release 0.0.3 (#14673)	9 months ago
William FH	75b8891399	Update Vertex AI to include Gemini (#14670) ... h/t to @lkuligin - **Description:** added new models on VertexAI - **Twitter handle:** @lkuligin --------- Co-authored-by: Leonid Kuligin <lkuligin@yandex.ru> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Tomaz Bratanic	ea2616ae23	Fix RRF and lucene escape characters for neo4j vector store (#14646) ... * Remove Lucene special characters (fixes https://github.com/langchain-ai/langchain/issues/14232) * Fixes RRF normalization for hybrid search	9 months ago
Chengzu Ou	df95abb7e7	docs: Add Databricks Vector Search example notebook (#14158) ... This PR adds an example notebook for the Databricks Vector Search vector store. It also adds an introduction to the Databricks Vector Search product on the Databricks's provider page. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	9 months ago
葛尧	e780433f6b	Fix token_usage None issue in ChatOpenAI with local Chatglm2-6B (#14493) ... When using local Chatglm2-6B by changing OPENAI_BASE_URL to localhost, the token_usage in ChatOpenAI becomes None. This leads to an AttributeError when trying to access token_usage.items(). This commit adds a check to ensure token_usage is not None before accessing its items. This change prevents the AttributeError and allows ChatOpenAI to work seamlessly with a local Chatglm2-6B model, aligning with the way it operates with the OpenAI API. <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Massimiliano Pronesti	6080c98108	fix(embeddings): huggingface hub embeddings and TEI (#14489) ... **Description:** This PR fixes `HuggingFaceHubEmbeddings` by making the API token optional (as in the client beneath). Most models don't require one. I also updated the notebook for TEI (text-embeddings-inference) accordingly as requested here #14288. In addition, I fixed a mistake in the POST call parameters. **Tag maintainers:** @baskaryan	9 months ago
Bob Lin	a019183a01	create mypy cache dir if it doesn't exist (#14579) ... ### Description When running `make lint` multiple times, i can see the error `mkdir: .mypy_cache: File exists`. Use `mkdir -p` to solve this problem. <img width="1512" alt="Screenshot 2023-12-12 at 11 22 01 AM" src="https://github.com/langchain-ai/langchain/assets/10000925/1429383d-3283-4e22-8882-5693bc50b502">	9 months ago
dandanwei	e5bd88383f	fix a bug in RedisNum filter againt value 0 (#14587) ... - **Description:** There is a bug in RedisNum filter that filter towards value 0 will be parsed as "*". This is a fix to it. - **Issue:** NA - **Dependencies:** NA - **Tag maintainer:** NA - **Twitter handle:** NA	9 months ago
Bagatur	ca7da8f7ef	docs: fix links in readme (#14624)	9 months ago