Commit Graph

7544 Commits

Author SHA1 Message Date
Kirushikesh DB
47bd58dc11
docs: Added illustration of using RetryOutputParser with LLMChain (#16722)
**Description:**
Updated the retry.ipynb notebook, it contains the illustrations of
RetryOutputParser in LangChain. But the notebook lacks to explain the
compatibility of RetryOutputParser with existing chains. This changes
adds some code to illustrate the workflow of using RetryOutputParser
with the user chain.

Changes:
1. Changed RetryWithErrorOutputParser with RetryOutputParser, as the
markdown text says so.
2. Added code at the last of the notebook to define a chain which passes
the LLM completions to the retry parser, which can be customised for
user needs.

**Issue:** 
Since RetryOutputParser/RetryWithErrorOutputParser does not implement
the parse function it cannot be used with LLMChain directly like
[this](https://python.langchain.com/docs/expression_language/cookbook/prompt_llm_parser#prompttemplate-llm-outputparser).
This also raised various issues #15133 #12175 #11719 still open, instead
of adding new features/code changes its best to explain the "how to
integrate LLMChain with retry parsers" clearly with an example in the
corresponding notebook.

Inspired from:
https://github.com/langchain-ai/langchain/issues/15133#issuecomment-1868972580

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-29 11:24:52 -08:00
Jael Gu
a1aa3a657c
community[patch]: Milvus supports add & delete texts by ids (#16256)
# Description

To support [langchain
indexing](https://python.langchain.com/docs/modules/data_connection/indexing)
as requested by users, vectorstore Milvus needs to support:
- document addition by id (`add_documents` method with `ids` argument)
- delete by id (`delete` method with `ids` argument)

Example usage:

```python
from langchain.indexes import SQLRecordManager, index
from langchain.schema import Document
from langchain_community.vectorstores import Milvus
from langchain_openai import OpenAIEmbeddings

collection_name = "test_index"
embedding = OpenAIEmbeddings()
vectorstore = Milvus(embedding_function=embedding, collection_name=collection_name)

namespace = f"milvus/{collection_name}"
record_manager = SQLRecordManager(
    namespace, db_url="sqlite:///record_manager_cache.sql"
)
record_manager.create_schema()

doc1 = Document(page_content="kitty", metadata={"source": "kitty.txt"})
doc2 = Document(page_content="doggy", metadata={"source": "doggy.txt"})

index(
    [doc1, doc1, doc2],
    record_manager,
    vectorstore,
    cleanup="incremental",  # None, "incremental", or "full"
    source_id_key="source",
)
```

# Fix issues

Fix https://github.com/milvus-io/milvus/issues/30112

---------

Signed-off-by: Jael Gu <mengjia.gu@zilliz.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2024-01-29 11:19:50 -08:00
Michard Hugo
e9d3527b79
community[patch]: Add missing async similarity_distance_threshold handling in RedisVectorStoreRetriever (#16359)
Add missing async similarity_distance_threshold handling in
RedisVectorStoreRetriever

- **Description:** added method `_aget_relevant_documents` to
`RedisVectorStoreRetriever` that overrides parent method to add support
of `similarity_distance_threshold` in async mode (as for sync mode)
  - **Issue:** #16099
  - **Dependencies:** N/A
  - **Twitter handle:** N/A
2024-01-29 11:19:30 -08:00
Jarod Stewart
7c6a2a8384
templates: Ionic Shopping Assistant (#16648)
- **Description:** This is a template for creating shopping assistant
chat bots
- **Issue:** Example for creating a shopping assistant with OpenAI Tools
Agent
- **Dependencies:** Ionic
https://github.com/ioniccommerce/ionic_langchain
  - **Twitter handle:** @ioniccommerce

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
2024-01-29 11:08:24 -08:00
Bagatur
7237dc67d4
core[patch]: Release 0.1.17 (#16737) 2024-01-29 11:02:29 -08:00
Anthony Bernabeu
2db79ab111
community[patch]: Implement TTL for DynamoDBChatMessageHistory (#15478)
- **Description:** Implement TTL for DynamoDBChatMessageHistory, 
  - **Issue:** see #15477,
  - **Dependencies:** N/A,

---------

Co-authored-by: Piyush Jain <piyushjain@duck.com>
2024-01-29 10:22:46 -08:00
Massimiliano Pronesti
1bc8d9a943
experimental[patch]: missing resolution strategy in anonymization (#16653)
- **Description:** Presidio-based anonymizers are not working because
`_remove_conflicts_and_get_text_manipulation_data` was being called
without a conflict resolution strategy. This PR fixes this issue. In
addition, it removes some mutable default arguments (antipattern).
 
To reproduce the issue, just run the very first cell of this
[notebook](https://python.langchain.com/docs/guides/privacy/2/) from
langchain's documentation.

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is
whichever of langchain, community, core, experimental, etc. is being
modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` from the root
of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
2024-01-29 09:56:16 -08:00
Abhinav
8e44363ec9
langchain_community: Update documentation for installing llama-cpp-python on windows (#16666)
**Description** : This PR updates the documentation for installing
llama-cpp-python on Windows.

- Updates install command to support pyproject.toml
- Makes CPU/GPU install instructions clearer
- Adds reinstall with GPU support command

**Issue**: Existing
[documentation](https://python.langchain.com/docs/integrations/llms/llamacpp#compiling-and-installing)
lists the following commands for installing llama-cpp-python
```
python setup.py clean
python setup.py install
````
The current version of the repo does not include a `setup.py` and uses a
`pyproject.toml` instead.
This can be replaced with
```
python -m pip install -e .
```
As explained in
https://github.com/abetlen/llama-cpp-python/issues/965#issuecomment-1837268339
**Dependencies**: None
**Twitter handle**: None

---------

Co-authored-by: blacksmithop <angstycoder101@gmaii.com>
2024-01-29 08:41:29 -08:00
taimo
d3d9244fee
langchain-community: fix unicode escaping issue with SlackToolkit (#16616)
- **Description:** fix unicode escaping issue with SlackToolkit
  - **Issue:**  #16610
2024-01-29 08:38:12 -08:00
Benito Geordie
f3fdc5c5da
community: Added integrations for ThirdAI's NeuralDB with Retriever and VectorStore frameworks (#15280)
**Description:** Adds ThirdAI NeuralDB retriever and vectorstore
integration. NeuralDB is a CPU-friendly and fine-tunable text retrieval
engine.
2024-01-29 08:35:42 -08:00
Jonathan Bennion
815896ff13
langchain: pubmed tool path update in doc (#16716)
- **Description:** The current pubmed tool documentation is referencing
the path to langchain core not the path to the tool in community. The
old tool redirects anyways, but for efficiency of using the more direct
path, just adding this documentation so it references the new path
  - **Issue:** doesn't fix an issue
  - **Dependencies:** no dependencies
  - **Twitter handle:** rooftopzen
2024-01-29 08:25:29 -08:00
Lance Martin
1bfadecdd2
Update Slack agent toolkit (#16732)
Co-authored-by: taimoOptTech <132860814+taimo3810@users.noreply.github.com>
2024-01-29 08:03:44 -08:00
Pashva Mehta
22d90800c8
community: Fixed schema discrepancy in from_texts function for weaviate vectorstore (#16693)
* Description: Fixed schema discrepancy in **from_texts** function for
weaviate vectorstore which created a redundant property "key" inside a
class.
* Issue: Fixed: https://github.com/langchain-ai/langchain/issues/16692
* Twitter handle: @pashvamehta1
2024-01-28 16:53:31 -08:00
Choi JaeHun
ba70630829
docs: Syntax correction according to langchain version update in 'Retry Parser' tutorial example (#16699)
- **Description:** Syntax correction according to langchain version
update in 'Retry Parser' tutorial example,
- **Issue:** #16698

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-28 16:53:04 -08:00
ccurme
ec0ae23645
core: expand docstring for RunnableGenerator (#16672)
- **Description:** expand docstring for RunnableGenerator
  - **Issue:** https://github.com/langchain-ai/langchain/issues/16631
2024-01-28 16:47:08 -08:00
Bob Lin
0866a984fe
Update n_gpu_layers"s description (#16685)
The `n_gpu_layers` parameter in `llama.cpp` supports the use of `-1`,
which means to offload all layers to the GPU, so the document has been
updated.

Ref:
35918873b4/llama_cpp/server/settings.py (L29C22-L29C117)


35918873b4/llama_cpp/llama.py (L125)
2024-01-28 16:46:50 -08:00
Daniel Erenrich
0600998f38
community: Wikidata tool support (#16691)
- **Description:** Adds Wikidata support to langchain. Can read out
documents from Wikidata.
  - **Issue:** N/A
- **Dependencies:** Adds implicit dependencies for
`wikibase-rest-api-client` (for turning items into docs) and
`mediawikiapi` (for hitting the search endpoint)
  - **Twitter handle:** @derenrich

You can see an example of this tool used in a chain
[here](https://nbviewer.org/urls/d.erenrich.net/upload/Wikidata_Langchain.ipynb)
or
[here](https://nbviewer.org/urls/d.erenrich.net/upload/Wikidata_Lars_Kai_Hansen.ipynb)

<!-- Thank you for contributing to LangChain!


Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` from the root
of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
2024-01-28 16:45:21 -08:00
Tze Min
6ef718c5f4
Core: fix Anthropic json issue in streaming (#16670)
**Description:** fix ChatAnthropic json issue in streaming 
**Issue:** https://github.com/langchain-ai/langchain/issues/16423
**Dependencies:** n/a

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-28 16:41:17 -08:00
Owen Sims
e451c8adc1
Community: Update Ionic Shopping Docs (#16700)
- **Description:** Update to docs as originally introduced in
https://github.com/langchain-ai/langchain/pull/16649 (reviewed by
@baskaryan),
- **Twitter handle:**
[@ioniccommerce](https://twitter.com/ioniccommerce)
2024-01-28 16:39:49 -08:00
Christophe Bornet
2e3af04080
Use Postponed Evaluation of Annotations in Astra and Cassandra doc loaders (#16694)
Minor/cosmetic change
2024-01-28 16:39:27 -08:00
Yelin Zhang
bc7607a4e9
docs: remove iprogress warnings (#16697)
- **Description:** removes iprogress warning texts from notebooks,
resulting in a little nicer to read documentation
2024-01-28 16:38:14 -08:00
Erick Friis
0255c5808b
infra: move release workflow back (#16707) 2024-01-28 12:11:23 -07:00
Erick Friis
88e3129587
robocorp: release 0.0.2 (#16706) 2024-01-28 11:28:58 -07:00
Christophe Bornet
36e432672a
community[minor]: Add async methods to AstraDBLoader (#16652) 2024-01-27 17:05:41 -08:00
William FH
38425c99d2
core[minor]: Image prompt template (#14263)
Builds on Bagatur's (#13227). See unit test for example usage (below)

```python
def test_chat_tmpl_from_messages_multipart_image() -> None:
    base64_image = "abcd123"
    other_base64_image = "abcd123"
    template = ChatPromptTemplate.from_messages(
        [
            ("system", "You are an AI assistant named {name}."),
            (
                "human",
                [
                    {"type": "text", "text": "What's in this image?"},
                    # OAI supports all these structures today
                    {
                        "type": "image_url",
                        "image_url": "data:image/jpeg;base64,{my_image}",
                    },
                    {
                        "type": "image_url",
                        "image_url": {"url": "data:image/jpeg;base64,{my_image}"},
                    },
                    {"type": "image_url", "image_url": "{my_other_image}"},
                    {
                        "type": "image_url",
                        "image_url": {"url": "{my_other_image}", "detail": "medium"},
                    },
                    {
                        "type": "image_url",
                        "image_url": {"url": "https://www.langchain.com/image.png"},
                    },
                    {
                        "type": "image_url",
                        "image_url": {"url": "data:image/jpeg;base64,foobar"},
                    },
                ],
            ),
        ]
    )
    messages = template.format_messages(
        name="R2D2", my_image=base64_image, my_other_image=other_base64_image
    )
    expected = [
        SystemMessage(content="You are an AI assistant named R2D2."),
        HumanMessage(
            content=[
                {"type": "text", "text": "What's in this image?"},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"},
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{other_base64_image}"
                    },
                },
                {
                    "type": "image_url",
                    "image_url": {"url": f"{other_base64_image}"},
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"{other_base64_image}",
                        "detail": "medium",
                    },
                },
                {
                    "type": "image_url",
                    "image_url": {"url": "https://www.langchain.com/image.png"},
                },
                {
                    "type": "image_url",
                    "image_url": {"url": "data:image/jpeg;base64,foobar"},
                },
            ]
        ),
    ]
    assert messages == expected
```

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Brace Sproul <braceasproul@gmail.com>
2024-01-27 17:04:29 -08:00
ARKA1112
3c387bc12d
docs: Error when importing packages from pydantic [docs] (#16564)
URL : https://python.langchain.com/docs/use_cases/extraction

Desc: 
<b> While the following statement executes successfully, it throws an
error which is described below when we use the imported packages</b>
 ```py 
from pydantic import BaseModel, Field, validator
```
Code: 
```python
from langchain.output_parsers import PydanticOutputParser
from langchain.prompts import (
    PromptTemplate,
)
from langchain_openai import OpenAI
from pydantic import BaseModel, Field, validator

# Define your desired data structure.
class Joke(BaseModel):
    setup: str = Field(description="question to set up a joke")
    punchline: str = Field(description="answer to resolve the joke")

    # You can add custom validation logic easily with Pydantic.
    @validator("setup")
    def question_ends_with_question_mark(cls, field):
        if field[-1] != "?":
            raise ValueError("Badly formed question!")
        return field
```

Error:
```md
PydanticUserError: The `field` and `config` parameters are not available
in Pydantic V2, please use the `info` parameter instead.

For further information visit
https://errors.pydantic.dev/2.5/u/validator-field-config-info
```

Solution:
Instead of doing:
```py
from pydantic import BaseModel, Field, validator
```
We should do:
```py
from langchain_core.pydantic_v1 import BaseModel, Field, validator
```
Thanks.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2024-01-27 16:46:48 -08:00
Rashedul Hasan Rijul
481493dbce
community[patch]: apply embedding functions during query if defined (#16646)
**Description:** This update ensures that the user-defined embedding
function specified during vector store creation is applied during
queries. Previously, even if a custom embedding function was defined at
the time of store creation, Bagel DB would default to using the standard
embedding function during query execution. This pull request addresses
this issue by consistently using the user-defined embedding function for
queries if one has been specified earlier.
2024-01-27 16:46:33 -08:00
Serena Ruan
f01fb47597
community[patch]: MLflowCallbackHandler -- Move textstat and spacy as optional dependency (#16657)
Signed-off-by: Serena Ruan <serena.rxy@gmail.com>
2024-01-27 16:15:07 -08:00
Zhuoyun(John) Xu
508bde7f40
community[patch]: Ollama - Pass headers to post request in async method (#16660)
# Description
A previous PR (https://github.com/langchain-ai/langchain/pull/15881)
added option to pass headers to ollama endpoint, but headers are not
pass to the async method.
2024-01-27 16:11:32 -08:00
Leonid Ganeline
5e73603e8a
docs: DeepInfra provider page update (#16665)
- added description, links
- consistent formatting
- added links to the example pages
2024-01-27 16:05:29 -08:00
João Carlos Ferra de Almeida
3e87b67a3c
community[patch]: Add Cookie Support to Fetch Method (#16673)
- **Description:** This change allows the `_fetch` method in the
`WebBaseLoader` class to utilize cookies from an existing
`requests.Session`. It ensures that when the `fetch` method is used, any
cookies in the provided session are included in the request. This
enhancement maintains compatibility with existing functionality while
extending the utility of the `fetch` method for scenarios where cookie
persistence is necessary.
- **Issue:** Not applicable (new feature),
- **Dependencies:** Requires `aiohttp` and `requests` libraries (no new
dependencies introduced),
- **Twitter handle:** N/A

Co-authored-by: Joao Almeida <joao.almeida@mercedes-benz.io>
2024-01-27 16:03:53 -08:00
Daniel Erenrich
c314137f5b
docs: Fix broken link in CONTRIBUTING.md (#16681)
- **Description:** link in CONTRIBUTING.md is broken
  - **Issue:** N/A
  - **Dependencies:** N/A
  - **Twitter handle:** @derenrich
2024-01-27 15:43:44 -08:00
Harrison Chase
27665e3546
[community] fix anthropic streaming (#16682) 2024-01-27 15:16:22 -08:00
Bagatur
5975bf39ec
infra: delete old CI workflows (#16680) 2024-01-27 14:14:53 -08:00
Christophe Bornet
4915c3cd86
[Fix] Fix Cassandra Document loader default page content mapper (#16273)
We can't use `json.dumps` by default as many types returned by the
cassandra driver are not serializable. It's safer to use `str` and let
users define their own custom `page_content_mapper` if needed.
2024-01-27 11:23:02 -08:00
Nuno Campos
e86fd946c8
In stream_event and stream_log handle closed streams (#16661)
if eg. the stream iterator is interrupted then adding more events to the
send_stream will raise an exception that we should catch (and handle
where appropriate)

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is
whichever of langchain, community, core, experimental, etc. is being
modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` from the root
of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
2024-01-27 08:09:29 -08:00
Jarod Stewart
0bc397957b
docs: document Ionic Tool (#16649)
- **Description:** Documentation for the Ionic Tool. A shopping
assistant tool that effortlessly adds e-commerce capabilities to your
Agent.
2024-01-26 16:02:07 -08:00
Nuno Campos
52ccae3fb1
Accept message-like things in Chat models, LLMs and MessagesPlaceholder (#16418) 2024-01-26 15:44:28 -08:00
Seungwoo Ryu
570b4f8e66
docs: Update openai_tools.ipynb (#16618)
typo
2024-01-26 15:26:27 -08:00
Pasha
4e189cd89a
community[patch]: youtube loader transcript format (#16625)
- **Description**: YoutubeLoader right now returns one document that
contains the entire transcript. I think it would be useful to add an
option to return multiple documents, where each document would contain
one line of transcript with the start time and duration in the metadata.
For example,
[AssemblyAIAudioTranscriptLoader](https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/document_loaders/assemblyai.py)
is implemented in a similar way, it allows you to choose between the
format to use for the document loader.
2024-01-26 15:26:09 -08:00
yin1991
a936472512
docs: Update documentation to use 'model_id' rather than 'model_name' to match actual API (#16615)
- **Description:** Replace 'model_name' with 'model_id' for accuracy 
- **Issue:**
[link-to-issue](https://github.com/langchain-ai/langchain/issues/16577)
  - **Dependencies:** 
  - **Twitter handle:**
2024-01-26 15:01:12 -08:00
Micah Parker
6543e585a5
community[patch]: Added support for Ollama's num_predict option in ChatOllama (#16633)
Just a simple default addition to the options payload for a ollama
generate call to support a max_new_tokens parameter.

Should fix issue: https://github.com/langchain-ai/langchain/issues/14715
2024-01-26 15:00:19 -08:00
Callum
6a75ef74ca
docs: Fix typo in XML agent documentation (#16645)
This is a tiny PR that just replacer "moduels" with "modules" in the
documentation for XML agents.
2024-01-26 14:59:46 -08:00
baichuan-assistant
70ff54eace
community[minor]: Add Baichuan Text Embedding Model and Baichuan Inc introduction (#16568)
- **Description:** Adding Baichuan Text Embedding Model and Baichuan Inc
introduction.

Baichuan Text Embedding ranks #1 in C-MTEB leaderboard:
https://huggingface.co/spaces/mteb/leaderboard

Co-authored-by: BaiChuanHelper <wintergyc@WinterGYCs-MacBook-Pro.local>
2024-01-26 12:57:26 -08:00
Bagatur
5b5115c408
google-vertexai[patch]: streaming bug (#16603)
Fixes errors seen here
https://github.com/langchain-ai/langchain/actions/runs/7661680517/job/20881556592#step:9:229
2024-01-26 09:45:34 -08:00
ccurme
a989f82027
core: expand docstring for RunnableParallel (#16600)
- **Description:** expand docstring for RunnableParallel
  - **Issue:** https://github.com/langchain-ai/langchain/issues/16462

Feel free to modify this or let me know how it can be improved!
2024-01-26 10:03:32 -05:00
Ghani
e30c6662df
Langchain-community : EdenAI chat integration. (#16377)
- **Description:** This PR adds [EdenAI](https://edenai.co/) for the
chat model (already available in LLM & Embeddings). It supports all
[ChatModel] functionality: generate, async generate, stream, astream and
batch. A detailed notebook was added.

  - **Dependencies**: No dependencies are added as we call a rest API.

---------

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2024-01-26 09:56:43 -05:00
Antonio Lanza
08d3fd7f2e
langchain[patch]: inconsistent results with RecursiveCharacterTextSplitter's add_start_index=True (#16583)
This PR fixes issue #16579
2024-01-25 15:50:06 -08:00
Eugene Yurtsev
42db96477f
docs: Update in code documentation for runnable with message history (#16585)
Update the in code documentation for Runnable With Message History
2024-01-25 15:26:34 -08:00
Jatin Chawda
a79345f199
community[patch]: Fixed tool names snake_case (#16397)
#16396
Fixed
1. golden_query
2. google_lens
3. memorize
4. merriam_webster
5. open_weather_map
6. pub_med
7. stack_exchange
8. generate_image
9. wikipedia
2024-01-25 15:24:19 -08:00