Commit Graph

2959 Commits (f92738a6f6c37909d42cda9e0493e5ad05b9af6e)

Author SHA1 Message Date
Kirushikesh DB 47bd58dc11
docs: Added illustration of using RetryOutputParser with LLMChain (#16722)
**Description:**
Updated the retry.ipynb notebook, it contains the illustrations of
RetryOutputParser in LangChain. But the notebook lacks to explain the
compatibility of RetryOutputParser with existing chains. This changes
adds some code to illustrate the workflow of using RetryOutputParser
with the user chain.

Changes:
1. Changed RetryWithErrorOutputParser with RetryOutputParser, as the
markdown text says so.
2. Added code at the last of the notebook to define a chain which passes
the LLM completions to the retry parser, which can be customised for
user needs.

**Issue:** 
Since RetryOutputParser/RetryWithErrorOutputParser does not implement
the parse function it cannot be used with LLMChain directly like
[this](https://python.langchain.com/docs/expression_language/cookbook/prompt_llm_parser#prompttemplate-llm-outputparser).
This also raised various issues #15133 #12175 #11719 still open, instead
of adding new features/code changes its best to explain the "how to
integrate LLMChain with retry parsers" clearly with an example in the
corresponding notebook.

Inspired from:
https://github.com/langchain-ai/langchain/issues/15133#issuecomment-1868972580

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
7 months ago
Jael Gu a1aa3a657c
community[patch]: Milvus supports add & delete texts by ids (#16256)
# Description

To support [langchain
indexing](https://python.langchain.com/docs/modules/data_connection/indexing)
as requested by users, vectorstore Milvus needs to support:
- document addition by id (`add_documents` method with `ids` argument)
- delete by id (`delete` method with `ids` argument)

Example usage:

```python
from langchain.indexes import SQLRecordManager, index
from langchain.schema import Document
from langchain_community.vectorstores import Milvus
from langchain_openai import OpenAIEmbeddings

collection_name = "test_index"
embedding = OpenAIEmbeddings()
vectorstore = Milvus(embedding_function=embedding, collection_name=collection_name)

namespace = f"milvus/{collection_name}"
record_manager = SQLRecordManager(
    namespace, db_url="sqlite:///record_manager_cache.sql"
)
record_manager.create_schema()

doc1 = Document(page_content="kitty", metadata={"source": "kitty.txt"})
doc2 = Document(page_content="doggy", metadata={"source": "doggy.txt"})

index(
    [doc1, doc1, doc2],
    record_manager,
    vectorstore,
    cleanup="incremental",  # None, "incremental", or "full"
    source_id_key="source",
)
```

# Fix issues

Fix https://github.com/milvus-io/milvus/issues/30112

---------

Signed-off-by: Jael Gu <mengjia.gu@zilliz.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
7 months ago
Abhinav 8e44363ec9
langchain_community: Update documentation for installing llama-cpp-python on windows (#16666)
**Description** : This PR updates the documentation for installing
llama-cpp-python on Windows.

- Updates install command to support pyproject.toml
- Makes CPU/GPU install instructions clearer
- Adds reinstall with GPU support command

**Issue**: Existing
[documentation](https://python.langchain.com/docs/integrations/llms/llamacpp#compiling-and-installing)
lists the following commands for installing llama-cpp-python
```
python setup.py clean
python setup.py install
````
The current version of the repo does not include a `setup.py` and uses a
`pyproject.toml` instead.
This can be replaced with
```
python -m pip install -e .
```
As explained in
https://github.com/abetlen/llama-cpp-python/issues/965#issuecomment-1837268339
**Dependencies**: None
**Twitter handle**: None

---------

Co-authored-by: blacksmithop <angstycoder101@gmaii.com>
7 months ago
Benito Geordie f3fdc5c5da
community: Added integrations for ThirdAI's NeuralDB with Retriever and VectorStore frameworks (#15280)
**Description:** Adds ThirdAI NeuralDB retriever and vectorstore
integration. NeuralDB is a CPU-friendly and fine-tunable text retrieval
engine.
7 months ago
Jonathan Bennion 815896ff13
langchain: pubmed tool path update in doc (#16716)
- **Description:** The current pubmed tool documentation is referencing
the path to langchain core not the path to the tool in community. The
old tool redirects anyways, but for efficiency of using the more direct
path, just adding this documentation so it references the new path
  - **Issue:** doesn't fix an issue
  - **Dependencies:** no dependencies
  - **Twitter handle:** rooftopzen
7 months ago
Lance Martin 1bfadecdd2
Update Slack agent toolkit (#16732)
Co-authored-by: taimoOptTech <132860814+taimo3810@users.noreply.github.com>
7 months ago
Choi JaeHun ba70630829
docs: Syntax correction according to langchain version update in 'Retry Parser' tutorial example (#16699)
- **Description:** Syntax correction according to langchain version
update in 'Retry Parser' tutorial example,
- **Issue:** #16698

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
7 months ago
Bob Lin 0866a984fe
Update `n_gpu_layers`"s description (#16685)
The `n_gpu_layers` parameter in `llama.cpp` supports the use of `-1`,
which means to offload all layers to the GPU, so the document has been
updated.

Ref:
35918873b4/llama_cpp/server/settings.py (L29C22-L29C117)


35918873b4/llama_cpp/llama.py (L125)
7 months ago
Daniel Erenrich 0600998f38
community: Wikidata tool support (#16691)
- **Description:** Adds Wikidata support to langchain. Can read out
documents from Wikidata.
  - **Issue:** N/A
- **Dependencies:** Adds implicit dependencies for
`wikibase-rest-api-client` (for turning items into docs) and
`mediawikiapi` (for hitting the search endpoint)
  - **Twitter handle:** @derenrich

You can see an example of this tool used in a chain
[here](https://nbviewer.org/urls/d.erenrich.net/upload/Wikidata_Langchain.ipynb)
or
[here](https://nbviewer.org/urls/d.erenrich.net/upload/Wikidata_Lars_Kai_Hansen.ipynb)

<!-- Thank you for contributing to LangChain!


Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` from the root
of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
7 months ago
Owen Sims e451c8adc1
Community: Update Ionic Shopping Docs (#16700)
- **Description:** Update to docs as originally introduced in
https://github.com/langchain-ai/langchain/pull/16649 (reviewed by
@baskaryan),
- **Twitter handle:**
[@ioniccommerce](https://twitter.com/ioniccommerce)
7 months ago
Yelin Zhang bc7607a4e9
docs: remove iprogress warnings (#16697)
- **Description:** removes iprogress warning texts from notebooks,
resulting in a little nicer to read documentation
7 months ago
ARKA1112 3c387bc12d
docs: Error when importing packages from pydantic [docs] (#16564)
URL : https://python.langchain.com/docs/use_cases/extraction

Desc: 
<b> While the following statement executes successfully, it throws an
error which is described below when we use the imported packages</b>
 ```py 
from pydantic import BaseModel, Field, validator
```
Code: 
```python
from langchain.output_parsers import PydanticOutputParser
from langchain.prompts import (
    PromptTemplate,
)
from langchain_openai import OpenAI
from pydantic import BaseModel, Field, validator

# Define your desired data structure.
class Joke(BaseModel):
    setup: str = Field(description="question to set up a joke")
    punchline: str = Field(description="answer to resolve the joke")

    # You can add custom validation logic easily with Pydantic.
    @validator("setup")
    def question_ends_with_question_mark(cls, field):
        if field[-1] != "?":
            raise ValueError("Badly formed question!")
        return field
```

Error:
```md
PydanticUserError: The `field` and `config` parameters are not available
in Pydantic V2, please use the `info` parameter instead.

For further information visit
https://errors.pydantic.dev/2.5/u/validator-field-config-info
```

Solution:
Instead of doing:
```py
from pydantic import BaseModel, Field, validator
```
We should do:
```py
from langchain_core.pydantic_v1 import BaseModel, Field, validator
```
Thanks.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
8 months ago
Leonid Ganeline 5e73603e8a
docs: `DeepInfra` provider page update (#16665)
- added description, links
- consistent formatting
- added links to the example pages
8 months ago
Jarod Stewart 0bc397957b
docs: document Ionic Tool (#16649)
- **Description:** Documentation for the Ionic Tool. A shopping
assistant tool that effortlessly adds e-commerce capabilities to your
Agent.
8 months ago
Seungwoo Ryu 570b4f8e66
docs: Update openai_tools.ipynb (#16618)
typo
8 months ago
Callum 6a75ef74ca
docs: Fix typo in XML agent documentation (#16645)
This is a tiny PR that just replacer "moduels" with "modules" in the
documentation for XML agents.
8 months ago
baichuan-assistant 70ff54eace
community[minor]: Add Baichuan Text Embedding Model and Baichuan Inc introduction (#16568)
- **Description:** Adding Baichuan Text Embedding Model and Baichuan Inc
introduction.

Baichuan Text Embedding ranks #1 in C-MTEB leaderboard:
https://huggingface.co/spaces/mteb/leaderboard

Co-authored-by: BaiChuanHelper <wintergyc@WinterGYCs-MacBook-Pro.local>
8 months ago
Ghani e30c6662df
Langchain-community : EdenAI chat integration. (#16377)
- **Description:** This PR adds [EdenAI](https://edenai.co/) for the
chat model (already available in LLM & Embeddings). It supports all
[ChatModel] functionality: generate, async generate, stream, astream and
batch. A detailed notebook was added.

  - **Dependencies**: No dependencies are added as we call a rest API.

---------

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
8 months ago
Bagatur 61e876aad8
openai[patch]: Explicitly support embedding dimensions (#16596) 8 months ago
Bagatur 6c89507988
docs: add rag citations page (#16549) 8 months ago
Bagatur db80832e4f
docs: output parser nits (#16588) 8 months ago
Bagatur ef42d9d559
core[patch], community[patch], openai[patch]: consolidate openai tool… (#16485)
… converters

One way to convert anything to an OAI function:
convert_to_openai_function
One way to convert anything to an OAI tool: convert_to_openai_tool
Corresponding bind functions on OAI models: bind_functions, bind_tools
8 months ago
Brian Burgin 148347e858
community[minor]: Add LiteLLM Router Integration (#15588)
community:

  - **Description:**
- Add new ChatLiteLLMRouter class that allows a client to use a LiteLLM
Router as a LangChain chat model.
- Note: The existing ChatLiteLLM integration did not cover the LiteLLM
Router class.
    - Add tests and Jupyter notebook.
  - **Issue:** None
  - **Dependencies:** Relies on existing ChatLiteLLM integration
  - **Twitter handle:** @bburgin_0

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
8 months ago
Bob Lin 35e60728b7
docs: Fix broken urls (#16559) 8 months ago
Bob Lin 6023953ea7
docs: Fix github link (#16560) 8 months ago
Erick Friis adc008407e
exa: init pkg (#16553) 8 months ago
Rave Harpaz c4e9c9ca29
community[minor]: Add OCI Generative AI integration (#16548)
<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is
whichever of langchain, community, core, experimental, etc. is being
modified.

Replace this entire comment with:
- **Description:** Adding Oracle Cloud Infrastructure Generative AI
integration. Oracle Cloud Infrastructure (OCI) Generative AI is a fully
managed service that provides a set of state-of-the-art, customizable
large language models (LLMs) that cover a wide range of use cases, and
which is available through a single API. Using the OCI Generative AI
service you can access ready-to-use pretrained models, or create and
host your own fine-tuned custom models based on your own data on
dedicated AI clusters.
https://docs.oracle.com/en-us/iaas/Content/generative-ai/home.htm
  - **Issue:** None,
  - **Dependencies:** OCI Python SDK,
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` from the root
of the package you've modified to check this locally.
Passed

See contribution guidelines for more information on how to write/run
tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.

we provide unit tests. However, we cannot provide integration tests due
to Oracle policies that prohibit public sharing of api keys.
 
If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->

---------

Co-authored-by: Arthur Cheng <arthur.cheng@oracle.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
8 months ago
Leonid Ganeline f6a05e964b
docs: `Hugging Face` update (#16490)
- added missed integrations to the platform page
- updated integration examples: added links and fixed formats
8 months ago
Harel Gal a91181fe6d
community[minor]: add support for Guardrails for Amazon Bedrock (#15099)
Added support for optionally supplying 'Guardrails for Amazon Bedrock'
on both types of model invocations (batch/regular and streaming) and for
all models supported by the Amazon Bedrock service.

@baskaryan  @hwchase17

```python 
llm = Bedrock(model_id="<model_id>", client=bedrock,
                  model_kwargs={},
                  guardrails={"id": " <guardrail_id>",
                              "version": "<guardrail_version>",
                               "trace": True}, callbacks=[BedrockAsyncCallbackHandler()])

class BedrockAsyncCallbackHandler(AsyncCallbackHandler):
    """Async callback handler that can be used to handle callbacks from langchain."""

    async def on_llm_error(
            self,
            error: BaseException,
            **kwargs: Any,
    ) -> Any:
        reason = kwargs.get("reason")
        if reason == "GUARDRAIL_INTERVENED":
           # kwargs contains additional trace information sent by 'Guardrails for Bedrock' service.
            print(f"""Guardrails: {kwargs}""")


# streaming 
llm = Bedrock(model_id="<model_id>", client=bedrock,
                  model_kwargs={},
                  streaming=True,
                  guardrails={"id": "<guardrail_id>",
                              "version": "<guardrail_version>"})
```

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
8 months ago
Martin Kolb 04651f0248
community[minor]: VectorStore integration for SAP HANA Cloud Vector Engine (#16514)
- **Description:**
This PR adds a VectorStore integration for SAP HANA Cloud Vector Engine,
which is an upcoming feature in the SAP HANA Cloud database
(https://blogs.sap.com/2023/11/02/sap-hana-clouds-vector-engine-announcement/).

  - **Issue:** N/A
- **Dependencies:** [SAP HANA Python
Client](https://pypi.org/project/hdbcli/)
  - **Twitter handle:** @sapopensource

Implementation of the integration:
`libs/community/langchain_community/vectorstores/hanavector.py`

Unit tests:
`libs/community/tests/unit_tests/vectorstores/test_hanavector.py`

Integration tests:
`libs/community/tests/integration_tests/vectorstores/test_hanavector.py`

Example notebook:
`docs/docs/integrations/vectorstores/hanavector.ipynb`

Access credentials for execution of the integration tests can be
provided to the maintainers.

---------

Co-authored-by: sascha <sascha.stoll@sap.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
8 months ago
Bob Lin 54dd8e52a8
docs: Updated comments about `n_gpu_layers` in the Metal section (#16501)
Ref: https://github.com/langchain-ai/langchain/issues/16502
8 months ago
Anastasiia Manokhina ce595f0203
docs:Updated integration docs structure for chat/google_vertex_ai_palm (#16201)
Description: 

- checked that the doc chat/google_vertex_ai_palm is using new
functions: invoke, stream etc.
- added Gemini example
- fixed wrong output in Sanskrit example

Issue: https://github.com/langchain-ai/langchain/issues/15664
Dependencies: None
Twitter handle: None
8 months ago
Erick Friis 8d299645f9
docs: rm output (#16519) 8 months ago
Lance Martin 0b740ebd49
Update SQL agent toolkit docs (#16409) 8 months ago
Francisco Ingham 13cf4594f4
docs: added a few suggestions for sql docs (#16508) 8 months ago
Eugene Yurtsev 6004e9706f
Docs: Add streaming section (#16468)
Adds a streaming section to LangChain documentation, explaining
`stream`/`astream` API and `astream_events` API.
8 months ago
Tipwheal 66aafc0573
Docs: typo in tool use quick start page (#16494)
Minor typo fix
8 months ago
BeatrixCohere 2b2285dac0
docs: Update cohere rerank and comparison docs (#16198)
- **Description:** Update the cohere rerank docs to use cohere
embeddings
  - **Issue:** n/a
  - **Dependencies:** n/a
  - **Twitter handle:** n/a
8 months ago
Raunak 476bf8b763
community[patch]: Load list of files using UnstructuredFileLoader (#16216)
- **Description:** Updated `_get_elements()` function of
`UnstructuredFileLoader `class to check if the argument self.file_path
is a file or list of files. If it is a list of files then it iterates
over the list of file paths, calls the partition function for each one,
and appends the results to the elements list. If self.file_path is not a
list, it calls the partition function as before.
  
  - **Issue:** Fixed #15607,
  - **Dependencies:** NA
  - **Twitter handle:** NA

Co-authored-by: H161961 <Raunak.Raunak@Honeywell.com>
8 months ago
Xudong Sun 019b6ebe8d
community[minor]: Add iFlyTek Spark LLM chat model support (#13389)
- **Description:** This PR enables LangChain to access the iFlyTek's
Spark LLM via the chat_models wrapper.
  - **Dependencies:** websocket-client ^1.6.1
  - **Tag maintainer:** @baskaryan 

### SparkLLM chat model usage

Get SparkLLM's app_id, api_key and api_secret from [iFlyTek SparkLLM API
Console](https://console.xfyun.cn/services/bm3) (for more info, see
[iFlyTek SparkLLM Intro](https://xinghuo.xfyun.cn/sparkapi) ), then set
environment variables `IFLYTEK_SPARK_APP_ID`, `IFLYTEK_SPARK_API_KEY`
and `IFLYTEK_SPARK_API_SECRET` or pass parameters when using it like the
demo below:

```python3
from langchain.chat_models.sparkllm import ChatSparkLLM

client = ChatSparkLLM(
    spark_app_id="<app_id>",
    spark_api_key="<api_key>",
    spark_api_secret="<api_secret>"
)
```
8 months ago
Eugene Yurtsev d898d2f07b
docs: Fix version in which astream_events was released (#16481)
Fix typo in version
8 months ago
bu2kx ff3163297b
community[minor]: Add KDBAI vector store (#12797)
Addition of KDBAI vector store (https://kdb.ai).

Dependencies: `kdbai_client` v0.1.2 Python package.

Sample notebook: `docs/docs/integrations/vectorstores/kdbai.ipynb`

Tag maintainer: @bu2kx
Twitter handle: @kxsystems
8 months ago
JongRok BAEK 4ec3fe4680
docs: Updated integration docs structure for chat/anthropic (#16268)
Description: 
- Added output and environment variables
- Updated the documentation for chat/anthropic, changing references from
`langchain.schema` to `langchain_core.prompts`.

Issue: https://github.com/langchain-ai/langchain/issues/15664
Dependencies: None
Twitter handle: None

Since this is my first open-source PR, please feel free to point out any
mistakes, and I'll be eager to make corrections.
8 months ago
Shivani Modi 4e160540ff
community[minor]: Adding Konko Completion endpoint (#15570)
This PR introduces update to Konko Integration with LangChain.

1. **New Endpoint Addition**: Integration of a new endpoint to utilize
completion models hosted on Konko.

2. **Chat Model Updates for Backward Compatibility**: We have updated
the chat models to ensure backward compatibility with previous OpenAI
versions.

4. **Updated Documentation**: Comprehensive documentation has been
updated to reflect these new changes, providing clear guidance on
utilizing the new features and ensuring seamless integration.

Thank you to the LangChain team for their exceptional work and for
considering this PR. Please let me know if any additional information is
needed.

---------

Co-authored-by: Shivani Modi <shivanimodi@Shivanis-MacBook-Pro.local>
Co-authored-by: Shivani Modi <shivanimodi@Shivanis-MBP.lan>
8 months ago
Facundo Santiago 92e6a641fd
feat: adding paygo api support for Azure ML / Azure AI Studio (#14560)
- **Description:** Introducing support for LLMs and Chat models running
in Azure AI studio and Azure ML using the new deployment mode
pay-as-you-go (model as a service).
- **Issue:** NA
- **Dependencies:** None.
- **Tag maintainer:** @prakharg-msft @gdyre 
- **Twitter handle:** @santiagofacundo

Examples added:
*
[docs/docs/integrations/llms/azure_ml.ipynb](https://github.com/santiagxf/langchain/blob/santiagxf/azureml-endpoints-paygo-community/docs/docs/integrations/chat/azureml_endpoint.ipynb)
*
[docs/docs/integrations/chat/azureml_chat_endpoint.ipynb](https://github.com/santiagxf/langchain/blob/santiagxf/azureml-endpoints-paygo-community/docs/docs/integrations/chat/azureml_chat_endpoint.ipynb)

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
8 months ago
baichuan-assistant 20fcd49348
community: Fix Baichuan Chat. (#15207)
- **Description:** Baichuan Chat (with both Baichuan-Turbo and
Baichuan-Turbo-192K models) has updated their APIs. There are breaking
changes. For example, BAICHUAN_SECRET_KEY is removed in the latest API
but is still required in Langchain. Baichuan's Langchain integration
needs to be updated to the latest version.
  - **Issue:** #15206
  - **Dependencies:** None,
  - **Twitter handle:** None

@hwchase17.

Co-authored-by: BaiChuanHelper <wintergyc@WinterGYCs-MacBook-Pro.local>
8 months ago
gcheron cfc225ecb3
community: SQLStrStore/SQLDocStore provide an easy SQL alternative to `InMemoryStore` to persist data remotely in a SQL storage (#15909)
**Description:**

- Implement `SQLStrStore` and `SQLDocStore` classes that inherits from
`BaseStore` to allow to persist data remotely on a SQL server.
- SQL is widely used and sometimes we do not want to install a caching
solution like Redis.
- Multiple issues/comments complain that there is no easy remote and
persistent solution that are not in memory (users want to replace
InMemoryStore), e.g.,
https://github.com/langchain-ai/langchain/issues/14267,
https://github.com/langchain-ai/langchain/issues/15633,
https://github.com/langchain-ai/langchain/issues/14643,
https://stackoverflow.com/questions/77385587/persist-parentdocumentretriever-of-langchain
- This is particularly painful when wanting to use
`ParentDocumentRetriever `
- This implementation is particularly useful when:
     * it's expensive to construct an InMemoryDocstore/dict
     * you want to retrieve documents from remote sources
     * you just want to reuse existing objects
- This implementation integrates well with PGVector, indeed, when using
PGVector, you already have a SQL instance running. `SQLDocStore` is a
convenient way of using this instance to store documents associated to
vectors. An integration example with ParentDocumentRetriever and
PGVector is provided in docs/docs/integrations/stores/sql.ipynb or
[here](https://github.com/gcheron/langchain/blob/sql-store/docs/docs/integrations/stores/sql.ipynb).
- It persists `str` and `Document` objects but can be easily extended.

 **Issue:**

Provide an easy SQL alternative to `InMemoryStore`.

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
8 months ago
dudgeon 26b2ad6d5b
Fixed typo on quickstart.ipynb (#16482)
- **Description:** Quick typo fix: `inpect` >> `inspect`
  - **Issue:** N/A
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** @geoffdudgeon
8 months ago
Eugene Yurtsev 39d1cbfecf
Docs: Document astream_events API (#16300)
Document astream events API
8 months ago
Florian MOREL 4b7969efc5
community[minor]: New documents loader for visio files (with extension .vsdx) (#16171)
**Description** : New documents loader for visio files (with extension
.vsdx)

A [visio file](https://fr.wikipedia.org/wiki/Microsoft_Visio) (with
extension .vsdx) is associated with Microsoft Visio, a diagram creation
software. It stores information about the structure, layout, and
graphical elements of a diagram. This format facilitates the creation
and sharing of visualizations in areas such as business, engineering,
and computer science.

A Visio file can contain multiple pages. Some of them may serve as the
background for others, and this can occur across multiple layers. This
loader extracts the textual content from each page and its associated
pages, enabling the extraction of all visible text from each page,
similar to what an OCR algorithm would do.

**Dependencies** : xmltodict package
8 months ago
KhoPhi fb41b68ea1
docs: Update with LCEL examples to Ollama & ChatOllama Integration notebook (#16194)
- **Description:** Updated the Chat/Ollama docs notebook with LCEL chain
examples

- **Issue:**  #15664 I'm a new contributor 😊

- **Dependencies:** No dependencies

- **Twitter handle:** 

Comments:

- How do I truncate the output of the stream in the notebook if and or
when it goes on and on and on for even the basic of prompts?

Edit:

Looking forward to feedback @baskaryan

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
8 months ago
Michael Gorham 3b0226b2c6
docs: Update redis_chat_message_history.ipynb (#16344)
## Problem
Spent several hours trying to figure out how to pass
`RedisChatMessageHistory` as a `GetSessionHistoryCallable` with a
different REDIS hostname. This example kept connecting to
`redis://localhost:6379`, but I wanted to connect to a server not hosted
locally.

## Cause
Assumption the user knows how to implement `BaseChatMessageHistory` and
`GetSessionHistoryCallable`

## Solution
Update documentation to show how to explicitly set the REDIS hostname
using a lambda function much like the MongoDB and SQLite examples.
8 months ago
Ian c98994c3c9
docs: Improve notebook to show how to use tidb to store history messages (#16420)
After merging [PR
#16304](https://github.com/langchain-ai/langchain/pull/16304), I
realized that our notebook example for integrating TiDB with LangChain
was too basic. To make it more useful and user-friendly, I plan to
create a detailed example. This will show how to use TiDB for saving
history messages in LangChain, offering a clearer, more practical guide
for our users
8 months ago
Eugene Yurtsev c88750d54b
Docs: Agent streaming notebooks (#15858)
Update information about streaming in the agents section. Show how to
use astream_events to get token by token streaming.
8 months ago
Eugene Yurtsev e5672bc944
docs: Re-write custom agent to show to write a tools agent (#15907)
Shows how to write a tools agent rather than a functions agent.
8 months ago
Boris Feld 404abf139a
community: Add CometLLM tracing context var (#15765)
I also added LANGCHAIN_COMET_TRACING to enable the CometLLM tracing
integration similar to other tracing integrations. This is easier for
end-users to enable it rather than importing the callback and pass it
manually.

(This is the same content as
https://github.com/langchain-ai/langchain/pull/14650 but rebased and
squashed as something seems to confuse Github Action).
8 months ago
Jennifer Melot d6275e47f2
docs: Updated integration docs structure for tools/arxiv (#16091) (#16250)
- **Description:** Updated docs for tools/arxiv to use `AgentExecutor`
and `invoke`
  - **Issue:** #15664
  - **Dependencies:** None
  - **Twitter handle:** None
8 months ago
ChengZi a950fa0487
docs: add milvus multitenancy doc (#16177)
- **Description:** add milvus multitenancy doc, it is an example for
this [pr](https://github.com/langchain-ai/langchain/pull/15740) .
  - **Issue:** No,
  - **Dependencies:** No,
  - **Twitter handle:** No

Signed-off-by: ChengZi <chen.zhang@zilliz.com>
8 months ago
parkererickson-tg b26a22f307
community[minor]: add TigerGraph support (#16280)
**Description:** Add support for querying TigerGraph databases through
the InquiryAI service.
**Issue**: N/A
**Dependencies:** N/A
**Twitter handle:** @TigerGraphDB
8 months ago
Christophe Bornet 8da34118bc
docs: Add documentation for Cassandra Document Loader (#16282) 8 months ago
Jonathan Algar 774e543e1f
docs: fix formatting issue in rockset.ipynb (#16328)
**Description:** randomly discovered while working on another PR
https://github.com/quarto-dev/quarto-cli/discussions/8131#discussioncomment-8027706

@anubhav94N ICYI
8 months ago
Ian b9f5104e6c
communty[minor]: Store Message History to TiDB Database (#16304)
This pull request integrates the TiDB database into LangChain for
storing message history, marking one of several steps towards a
comprehensive integration of TiDB with LangChain.


A simple usage
```python
from datetime import datetime
from langchain_community.chat_message_histories import TiDBChatMessageHistory

history = TiDBChatMessageHistory(
    connection_string="mysql+pymysql://<host>:<PASSWORD>@<host>:4000/<db>?ssl_ca=/etc/ssl/cert.pem&ssl_verify_cert=true&ssl_verify_identity=true",
    session_id="code_gen",
    earliest_time=datetime.utcnow(),  # Optional to set earliest_time to load messages after this time point.
)

history.add_user_message("hi! How's feature going?")
history.add_ai_message("It's almot done")
```
8 months ago
Sarthak Chaure dd5b8107b1
Docs: Updated callbacks/index.mdx (#16404)
The callbacks get started demo code was updated , replacing the
chain.run() command ( which is now depricated) ,with the updated
chain.invoke() command.
Solving the following issue : #16379
Twitter/X : @Hazxhx
8 months ago
Omar-aly 873de14cd8
docs: update vectorstores/llm_rails integration doc (#16199)
Description:
- Updated the docs for the vectorstores integration module
llm_rails.ipynb

Issue:
- [Connected to Issue
#15664](https://github.com/langchain-ai/langchain/issues/15664)
 
Dependencies:
- N/A

Co-authored-by: omaraly23 <112936089+omaraly22@users.noreply.github.com>
8 months ago
Lance Martin 369e90d427
docs: Minor update to Robocorp toolkit docs (#16399) 8 months ago
Hadi a1c0cf21c9
docs: Update import library for StreamlitCallbackHandler (#16401)
- **Description:** Some code sources have been moved from `langchain` to
`langchain_community` and so the documentation is not yet up-to-date.
This is specifically true for `StreamlitCallbackHandler` which returns a
`warning` message if not loaded from `langchain_community`.,
- **Issue:** I don't see a # issue that could address this problem but
perhaps #10744,
- **Dependencies:** Since it's a documentation change no dependencies
are required
8 months ago
JaguarDB 7ecd2f22ac
community[patch]: update documentation on jaguar vector store (#16346)
- **Description:** update documentation on jaguar vector store:
Instruction for setting up jaguar server and usage of text_tag.
  - **Issue:** 
  - **Dependencies:** 
  - **Twitter handle:**

---------

Co-authored-by: JY <jyjy@jaguardb>
8 months ago
Iskren Ivov Chernev fc196cab12
community[minor]: DeepInfra support for chat models (#16380)
Add deepinfra chat models support.

This is https://github.com/langchain-ai/langchain/pull/14234 re-opened
from my branch (so maintainers can edit).
8 months ago
Bagatur eac91b60c9
docs: qa rag nit (#16400) 8 months ago
Bagatur 1dc6c1ce06
core[patch], community[patch], langchain[patch], docs: Update SQL chains/agents/docs (#16168)
Revamp SQL use cases docs. In the process update SQL chains and agents.
8 months ago
Jatin Chawda 05162928c0
Docs: Fixed Urls of AsyncHtmlLoader, AsyncChromiumLoader and HTML2Text links in Web scraping Docs (#16365)
Fixing links in documentation.
8 months ago
Christophe Bornet f9be877ed7
Docs: Add self-querying retriever and store to AstraDB provider doc (#16362)
Add self-querying retriever and store to AstraDB provider doc
8 months ago
Mateusz Szewczyk 076dbb1a8f
docs: IBM watsonx.ai Use `invoke` instead of `__call__` (#16371)
- **Description:** Updating documentation of IBM
[watsonx.ai](https://www.ibm.com/products/watsonx-ai) LLM with using
`invoke` instead of `__call__`
- **Dependencies:**
[ibm-watsonx-ai](https://pypi.org/project/ibm-watsonx-ai/),
  - **Tag maintainer:** : 

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally. 

The following warning information show when i use `run` and `__call__`
method:
```
LangChainDeprecationWarning: The function `__call__` was deprecated in LangChain 0.1.7 and will be removed in 0.2.0. Use invoke instead.
  warn_deprecated(
```

We need to update documentation for using `invoke` method
8 months ago
Bob Lin c6bd7778b0
Use `invoke` instead of `__call__` (#16369)
The following warning information will be displayed when i use
`llm(PROMPT)`:

```python
/Users/169/llama.cpp/venv/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The function `__call__` was deprecated in LangChain 0.1.7 and will be removed in 0.2.0. Use invoke instead.
  warn_deprecated(
```

So I changed to standard usage.
8 months ago
Virat Singh c2a614eddc
community: Add PolygonLastQuote Tool and Toolkit (#15990)
**Description:** 
In this PR, I am adding a `PolygonLastQuote` Tool, which can be used to
get the latest price quote for a given ticker / stock.

Additionally, I've added a Polygon Toolkit, which we can use to
encapsulate future tools that we build for Polygon.

**Twitter handle:** [@virattt](https://twitter.com/virattt)

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
8 months ago
Bagatur 1e29b676d5
core[patch]: simple fallback streaming (#16055) 8 months ago
Hamza Kyamanywa 39b3c6d94c
langchain[patch]: Add konlpy based text splitting for Korean (#16003)
- **Description:** Adds a text splitter based on
[Konlpy](https://konlpy.org/en/latest/#start) which is a Python package
for natural language processing (NLP) of the Korean language. (It is
like Spacy or NLTK for Korean)
- **Dependencies:** Konlpy would have to be installed before this
splitter is used,
  - **Twitter handle:** @untilhamza
8 months ago
Hongyu Lin 9b0a531aa2
doc: Fix small typo in quickstart (#16164)
- **Description:** fix small typo in quickstart

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
8 months ago
Lance Martin 881d1c3ec5
Update MultiON toolkit docs (#16286) 8 months ago
Bagatur 6f7a414955
docs: fix links (#16284) 8 months ago
Lance Martin f63906a9c2
Test and update MultiON agent toolkit docs (#16235) 8 months ago
Ashley Xu 0f99646ca6
docs: add the enrollment form for`BigQueryVectorSearch` (#16240)
This PR adds the enrollment form for BigQueryVectorSearch.
8 months ago
Eugene Yurtsev 177af65dc4
core[minor]: RFC Add astream_events to Runnables (#16172)
This PR adds `astream_events` method to Runnables to make it easier to
stream data from arbitrary chains.

* Streaming only works properly in async right now
* One should use `astream()` with if mixing in imperative code as might
be done with tool implementations
* Astream_log has been modified with minimal additive changes, so no
breaking changes are expected
* Underlying callback code / tracing code should be refactored at some
point to handle things more consistently (OK for now)

- ~~[ ] verify event for on_retry~~ does not work until we implement
streaming for retry
- ~~[ ] Any rrenaming? Should we rename "event" to "hook"?~~
- [ ] Any other feedback from community?
- [x] throw NotImplementedError for `RunnableEach` for now

## Example

See this [Example
Notebook](dbbc7fa0d6/docs/docs/modules/agents/how_to/streaming_events.ipynb)
for an example with streaming in the context of an Agent

## Event Hooks Reference

Here is a reference table that shows some events that might be emitted
by the various Runnable objects.
Definitions for some of the Runnable are included after the table.


| event | name | chunk | input | output |

|----------------------|------------------|---------------------------------|-----------------------------------------------|-------------------------------------------------|
| on_chat_model_start | [model name] | | {"messages": [[SystemMessage,
HumanMessage]]} | |
| on_chat_model_stream | [model name] | AIMessageChunk(content="hello")
| | |
| on_chat_model_end | [model name] | | {"messages": [[SystemMessage,
HumanMessage]]} | {"generations": [...], "llm_output": None, ...} |
| on_llm_start | [model name] | | {'input': 'hello'} | |
| on_llm_stream | [model name] | 'Hello' | | |
| on_llm_end | [model name] | | 'Hello human!' |
| on_chain_start | format_docs | | | |
| on_chain_stream | format_docs | "hello world!, goodbye world!" | | |
| on_chain_end | format_docs | | [Document(...)] | "hello world!,
goodbye world!" |
| on_tool_start | some_tool | | {"x": 1, "y": "2"} | |
| on_tool_stream | some_tool | {"x": 1, "y": "2"} | | |
| on_tool_end | some_tool | | | {"x": 1, "y": "2"} |
| on_retriever_start | [retriever name] | | {"query": "hello"} | |
| on_retriever_chunk | [retriever name] | {documents: [...]} | | |
| on_retriever_end | [retriever name] | | {"query": "hello"} |
{documents: [...]} |
| on_prompt_start | [template_name] | | {"question": "hello"} | |
| on_prompt_end | [template_name] | | {"question": "hello"} |
ChatPromptValue(messages: [SystemMessage, ...]) |


Here are declarations associated with the events shown above:

`format_docs`:

```python
def format_docs(docs: List[Document]) -> str:
    '''Format the docs.'''
    return ", ".join([doc.page_content for doc in docs])

format_docs = RunnableLambda(format_docs)
```

`some_tool`:

```python
@tool
def some_tool(x: int, y: str) -> dict:
    '''Some_tool.'''
    return {"x": x, "y": y}
```

`prompt`:

```python
template = ChatPromptTemplate.from_messages(
    [("system", "You are Cat Agent 007"), ("human", "{question}")]
).with_config({"run_name": "my_template", "tags": ["my_template"]})
```
8 months ago
Erick Friis aa35b43bcd
docs, google-vertex[patch]: function docs (#16231) 8 months ago
Erick Friis f2b2d59e82
docs: transport and client options docs (#16226)
<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is
whichever of langchain, community, core, experimental, etc. is being
modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` from the root
of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
8 months ago
Rajesh Thallam 6bc6d64a12
langchain_google_vertexai[patch]: Add support for SystemMessage for Gemini chat model (#15933)
- **Description:** In Google Vertex AI, Gemini Chat models currently
doesn't have a support for SystemMessage. This PR adds support for it
only if a user provides additional convert_system_message_to_human flag
during model initialization (in this case, SystemMessage would be
prepended to the first HumanMessage). **NOTE:** The implementation is
similar to #14824


- **Twitter handle:** rajesh_thallam

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
8 months ago
jzaldi ed118950fe
docs: Updated integration docs structure for llm/google_vertex_ai_palm (#16091)
- **Description**: Updated doc for llm/google_vertex_ai_palm with new
functions: `invoke`, `stream`... Changed structure of the document to
match the required one.
- **Issue**: #15664 
- **Dependencies**: None
- **Twitter handle**: None

---------

Co-authored-by: Jorge Zaldívar <jzaldivar@google.com>
8 months ago
Bagatur aa2e642ce3
docs: tool use nits (#16211) 8 months ago
Eugene Zapolsky 6b9e3ed9e9
google-vertexai[minor]: added safety_settings property to gemini wrapper (#15344)
**Description:** Gemini model has quite annoying default safety_settings
settings. In addition, current VertexAI class doesn't provide a property
to override such settings.
So, this PR aims to 
 - add safety_settings property to VertexAI
- fix issue with incorrect LLM output parsing when LLM responds with
appropriate 'blocked' response
- fix issue with incorrect parsing LLM output when Gemini API blocks
prompt itself as inappropriate
- add safety_settings related tests

I'm not enough familiar with langchain code base and guidelines. So, any
comments and/or suggestions are very welcome.
 
**Issue:** it will likely fix #14841

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
8 months ago
Bagatur 27ad65cc68
docs: add tool use diagrams (#16207) 8 months ago
Bagatur 27ed2673da
docs: model io order (#16163) 8 months ago
Bagatur 2af813c7eb
docs: bump sphinx>=5 (#16162) 8 months ago
David DeCaprio ec9642d667
docs: Updated MongoDB Chat history example notebook to use LCEL format. (#15750)
- **Description:** Updated the MongoDB example integration notebook to
latest standards
- **Issue:**
[15664](https://github.com/langchain-ai/langchain/issues/15664)
  - **Dependencies:** None
  - **Twitter handle:** @davedecaprio

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
8 months ago
Bagatur e7ddec1f2c
docs: change parallel doc name (#16152) 8 months ago
Joshua Carroll bc0cb1148a
docs: Fix StreamlitChatMessageHistory docs to latest API (#16072)
- **Description:** Update [this
page](https://python.langchain.com/docs/integrations/memory/streamlit_chat_message_history)
to use the latest API
  - **Issue:** https://github.com/langchain-ai/langchain/issues/13995
  - **Dependencies:** None
  - **Twitter handle:** @OhSynap
8 months ago
David DeCaprio 9c2f1f07a0
docs: Updated SQLite example to use LCEL and SQLChatMessageHistory (#16094)
- **Description:** Updated the SQLite example integration notebook to
latest standards
- **Issue:**
[15664](https://github.com/langchain-ai/langchain/issues/15664)
  - **Dependencies:** None
  - **Twitter handle:** @davedecaprio
8 months ago
Abhinav da96c511d1
docs: Replace azure_cosmos_db_vector_search with azure_cosmos_db in Cosmos DB Documentation (#16122)
**Description**: This PR fixes an error in the documentation for Azure
Cosmos DB Integration.
**Issue**: The correct way to import `AzureCosmosDBVectorSearch` is
```python
from langchain_community.vectorstores.azure_cosmos_db import (
    AzureCosmosDBVectorSearch,
)
```
While the
[documentation](https://python.langchain.com/docs/integrations/vectorstores/azure_cosmos_db)
states it to be
```python
from langchain_community.vectorstores.azure_cosmos_db_vector_search import (
    AzureCosmosDBVectorSearch,
    CosmosDBSimilarityType,
)
```
As you can see in
[azure_cosmos_db.py](c323742f4f/libs/langchain/langchain/vectorstores/azure_cosmos_db.py (L1C45-L2))
**Dependencies:**: None
**Twitter handle**: None
8 months ago
purificant 3606c5d5e9
infra: update poetry 1.6.1 -> 1.7.1 (#15027) 8 months ago
Ikko Eltociear Ashimine a35e5f19a8
docs: Update gradient.ipynb (#16149)
Enviroment -> Environment
8 months ago
David c323742f4f
mistralai[minor]: Add embeddings (#15282)
- **Description:** Adds MistralAIEmbeddings class for embeddings, using
the new official API.
- **Dependencies:** mistralai
- **Tag maintainer**: @efriis, @hwchase17
- **Twitter handle:** @LMS_David_RS

Create `integrations/text_embedding/mistralai.ipynb`: an example
notebook for MistralAIEmbeddings class
Modify `embeddings/__init__.py`: Import the class
Create `embeddings/mistralai.py`: The embedding class
Create `integration_tests/embeddings/test_mistralai.py`: The test file.

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
8 months ago