Commit Graph

6729 Commits

Author SHA1 Message Date
Eugene Yurtsev
bf0b3cc0b5
core[patch]: Further restrict recursive URL loader (#15559)
Includes code from this PR:  https://github.com/langchain-ai/langchain/compare/HEAD...m0kr4n3:security/fix_ssrf 
with additional fixes 

Unit tests cover new test cases
2024-01-04 16:33:57 -05:00
Bagatur
817b84de9e
core[patch]: Release 0.1.6 (#15547) 2024-01-04 11:02:04 -05:00
Bagatur
b2f15738dd
core[patch], langchain[patch], community[patch]: Revert #15326 (#15546) 2024-01-04 10:39:37 -05:00
Harrison Chase
7a93356cbc
add new chain howtos (#15430) 2024-01-03 21:19:58 -08:00
Erick Friis
81886ad345
docs: fix broken link (#15509) 2024-01-03 16:00:18 -08:00
Erick Friis
02f9c76791
docs: broken link in contributor docs (#15436) 2024-01-03 13:45:33 -08:00
Erick Friis
1437872df9
infra: fail check_diffs if too many files changed (#15423)
Jobs like
https://github.com/langchain-ai/langchain/actions/runs/7389187843/job/20101494206
only receive the first 300 changed files. Because of the opportunity to
miss packages, better to auto-fail and manually run.

Checking that it does what I expect in #15424
2024-01-03 13:30:16 -08:00
Erick Friis
69a8a26683
templates: fix deps (#15439) 2024-01-03 13:28:05 -08:00
Erick Friis
70beb2e40d
docs: contributor faq (#15502) 2024-01-03 13:19:39 -08:00
Bagatur
6e90b7a91b
langchain[patch]: bump community >=0.0.8,<0.1 (#15492) 2024-01-03 13:31:48 -05:00
Bagatur
8b7d6531a5
langchain[patch]: Release 0.0.354 (#15482) 2024-01-03 12:51:55 -05:00
Bagatur
0b579dc623
infra: update community test min reqs (#15490) 2024-01-03 12:13:29 -05:00
Bagatur
266db0efc8
community[patch]: bump core version >=0.1.5,<0.2 (#15488) 2024-01-03 12:03:31 -05:00
Bagatur
63e0cae2b1
infra: fix min deps test (#15486) 2024-01-03 11:34:46 -05:00
Bagatur
a2324ee533
community[patch]: Release 0.0.8 (#15481) 2024-01-03 11:28:50 -05:00
Bagatur
54b58c03db
infra: add minimum deps pre release check (#15485) 2024-01-03 11:28:35 -05:00
Bagatur
b317ad2472
core[patch]: Release 0.1.5 (#15480) 2024-01-03 10:26:27 -05:00
Bagatur
baeac236b6
langchain[patch], experimental[patch]: update utilities imports (#15438) 2024-01-03 02:18:15 -05:00
Harutaka Kawamura
73da8f863c
Remove unused Params (#14385)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->

Removes unused `Params` in `libs/langchain/langchain/llms/mlflow.py`.

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-02 22:45:18 -08:00
chyroc
b65e57971e
Patch: improve type hint (#15451) 2024-01-02 22:39:27 -08:00
Harutaka Kawamura
8ebf55ebbf
Fix llms.Mlflow example (#14386)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->

The example code for `llms.Mlflow` is outdated.

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-02 22:35:13 -08:00
Nolan
6c4b5a4eff
Add option to preserve headers in MarkdownHeaderTextSplitter (#14433)
- **Description:** `MarkdownHeaderTextSplitter` currently strips header
lines from chunked content. Many applications require these header lines
are preserved. This adds an optional parameter to preserve those headers
in the chunked content.
  - **Issue:** #2836 (relevant)
  - **Dependencies:** -
  - **Tag maintainer:** @baskaryan
  - **Twitter handle:** @finnless

Unit tests and new examples in notebook included.

cc @rlancemartin

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-02 22:34:52 -08:00
Xin Liu
0a7d360ba4
feat: new integration wasm_chat (#14787)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->

Adds `WasmChat` integration. `WasmChat` runs GGUF models locally or via
chat service in lightweight and secure WebAssembly containers. In this
PR, `WasmChatService` is introduced as the first step of the
integration. `WasmChatService` is driven by
[llama-api-server](https://github.com/second-state/llama-utils) and
[WasmEdge Runtime](https://wasmedge.org/).

---------

Signed-off-by: Xin Liu <sam@secondstate.io>
2024-01-02 22:33:14 -08:00
Harrison Chase
51dcb89a72
cleanup getting started (#15450) 2024-01-02 22:26:35 -08:00
Leonid Ganeline
2bbee894bb
fixed a dependency duplicate (#15444)
BaseModel is derived twice. Left only one.
2024-01-02 21:40:04 -08:00
William FH
65afc13b8b
[Improvement] Evals: Add git info (#15446) 2024-01-02 20:08:50 -08:00
Anush
58cc7878e9
refactor: Qdrant async improvements (#14492)
Follow up on https://github.com/langchain-ai/langchain/pull/13048.
This PR intends to simplify the Qdrant async implementation by replacing
the internal GRPC methods with the `QdrantAsyncClient` methods.
This is a backward compatible change with no additional steps required
after merge.

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-02 20:07:48 -08:00
Li-Lun Lin
cda68d717c
core[patch]: update LanguageModelInput from List to Sequence (#14405)
Co-authored-by: Erick Friis <erick@langchain.dev>
2024-01-02 18:49:01 -08:00
JuR-0
4dab37741a
Fix Bedrock broad error catching (#14398)
Fixes #14347 

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
- **Description:** Added the traceback of the previous error to keep the
initial error type,
  - **Issue:** #14347 ,
  - **Dependencies:** None,
  - **Tag maintainer:** 

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->

---------

Co-authored-by: Julien Raffy <julien.raffy@emeria.eu>
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-02 17:25:48 -08:00
amaleki2
413a56b8f1
adding vectorstore_kwarg attribute to search_similarity function (#14604)
- **Description:** the ability to add all extra parameter of vectorstore
and using them SemanticSimilarityExampleSelector.
  - **Issue:** #14583
  - **Dependencies:** no dependensies
  - **Tag maintainer:** 
  - **Twitter handle:** @AmirMalekiz

---------

Co-authored-by: Amir Maleki <amaleki@fb.com>
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-02 17:18:33 -08:00
Bob Lin
e93be14c11
Improvement: Allow passing parameters to the underlying es_client. Closes: #14403 (#14435)
### Description

In https://github.com/langchain-ai/langchain/issues/14403, the user
mentioned that he hopes not to verify ssl and needs to pass more
parameters

I found that the `Elasticsearch` class [has very many
parameters](98f2af2134/elasticsearch/_sync/client/__init__.py (L131-L191)
):

<img width="1097" alt="Screenshot 2023-12-08 at 4 24 39 PM"
src="https://github.com/langchain-ai/langchain/assets/10000925/f2201554-b41a-4388-a8e8-c14a2d0466d4">

In order to adapt to more situations, I want to add the kwargs parameter
so that users can enter more `Elasticsearch` parameters. Like
[redis](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/redis/base.py#L253),
[tair](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/tair.py#L32),
[myscale](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/myscale.py#L112)
and so on.
2024-01-02 16:48:17 -08:00
codehound42
8aa921d3a4
Support score_threshold in SupabaseVectorStore similarity search (#14439)
Description: Add support for setting the `score_threshold` for
similarity search in SupabaseVectoreStore.

This pull request addresses issue #14438

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-02 16:47:05 -08:00
Antonio Pisani
d4a98e4e04
core: update json output parser (#15079)
- **Description:** changed json.py to handle additional cases of partial
json string to be parsed, basically by dropping the last character in
the string until a valid json string is found or the string is empty.
Also added additional test cases.
  
- **Issue:** function parse_partial_json could not parse cases where the
key is present but the value is not.

---------

Co-authored-by: Nuno Campos <nuno@langchain.dev>
2024-01-02 16:34:43 -08:00
YISH
eecfa81918
Add the collection_description parameter to Milvus (#14524)
Because Milvus' collection_name doesn't support UFT8 characters in other
languages, I want the `collection_descriotion`.


<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
2024-01-02 16:28:01 -08:00
Evgenii Molov
b4ec340fb3
Fix failing serpapi response processing for Google Maps API (#14817)
**Description:** Fix for processing for serpapi response for Google Maps
API
**Issue:** Due to the fact corresponding
[api](https://serpapi.com/google-maps-api) returns 'local_results' as
list, and old version requested `res["local_results"].keys()` of the
list. As the result we got exception: ```AttributeError: 'list' object
has no attribute 'keys'```.

Way to reproduce wrong behaviour:
```
    params = {
        "engine": "google_maps",
        "type": "search",
        "google_domain": "google.de",
        "ll": "@51.1917,10.525,14z",
        "hl": "de",
        "gl": "de",
    }
    search = SerpAPIWrapper(params=params)
    results = search.run("cafe")
```

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
Co-authored-by: Ran <rccalman@gmail.com>
2024-01-02 16:17:21 -08:00
YISH
da0f750a0b
Milvus allows to store metadata as json field (#14636)
Because Milvus doesn't support nullable fields, but document metadata is
very rich, so it makes more sense to store it as json.


https://github.com/milvus-io/pymilvus/issues/1705#issuecomment-1731112372

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-02 16:12:00 -08:00
Erick Friis
620168e459
docs: together ai updates (#15435) 2024-01-02 16:05:53 -08:00
Bagatur
93e924ec96
langchain[patch], docs: update agent toolkit imports (#15434) 2024-01-02 18:58:50 -05:00
Ashley Xu
0ce7858529
feat: add Google BigQueryVectorSearch in vectorstore (#14829)
BigQuery vector search lets you use GoogleSQL to do semantic search,
using vector indexes for fast but approximate results, or using brute
force for exact results.

This PR integrates LangChain vectorstore with BigQuery Vector Search.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->

---------

Co-authored-by: Vlad Kolesnikov <vladkol@google.com>
2024-01-02 15:57:14 -08:00
JaguarDB
02f59c2035
Use args option in jaguar so it takes more options in similarity search (#15080)
- **Description:** replace score_threshold with args
  - **Issue:** needs a way to pass more options to similarity search
  - **Dependencies:** None
  - **Twitter handle:** @workbot

---------

Co-authored-by: JY <jyjy@jaguardb>
2024-01-02 15:53:06 -08:00
chyroc
37ad6ec248
Refactor: use SecretStr for tongyi chat-model (#15102) 2024-01-02 15:45:23 -08:00
Shaurya Rohatgi
e1c2cd7a28
community: Semanticscholar tool to search 200M+ scientific articles (#15151)
- **Description:** Tool now supports querying over 200 million
scientific articles, vastly expanding its reach beyond the 2 million
articles accessible through Arxiv. This update significantly broadens
access to the entire scope of scientific literature.
- **Dependencies:** semantischolar
https://github.com/danielnsilva/semanticscholar
  - **Twitter handle:** @shauryr

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2024-01-02 15:36:03 -08:00
aqibamir
073e4107cd
Fixed minor type in self_query.ipynb (#15196)
<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is
whichever of langchain, community, core, experimental, etc. is being
modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` from the root
of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
2024-01-02 15:34:09 -08:00
dudub12
7e6b0056b8
SQLDatabase drop the column names in the result. (#15361)
Fix for the following bug:
https://github.com/langchain-ai/langchain/issues/15360

---------

Co-authored-by: dudu butbul <100126964+dudu-upstream@users.noreply.github.com>
2024-01-02 15:29:25 -08:00
chyroc
07d294b5ec
Fix: fix Bing Search empty result exception, fix #15384 (#15387)
fix https://github.com/langchain-ai/langchain/issues/15384
2024-01-02 15:25:00 -08:00
Bagatur
1678d6ca17
langchain[patch], experimental[patch], docs: update tools imports (#15433) 2024-01-02 18:23:34 -05:00
Bob Lin
e57e50b213
Remove unused _get_python_repl (#15389)
This part of the code can also be safely cleaned up.
2024-01-02 15:21:00 -08:00
Dariusz Kajtoch
15b6c049d4
core:adds tests for partial_variables (#15427)
**Description:** Added small tests to test partial_variables in
PromptTemplate. It was missing.
2024-01-02 15:00:06 -08:00
suhas-kotaki
73a628de9a
added fix for key error: doc_id (#15428)
<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is
whichever of langchain, community, core, experimental, etc. is being
modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` from the root
of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
2024-01-02 14:59:53 -08:00
Leonid Ganeline
1e6519edc2
docs Microsoft platform page update (#15420)
Added two new document_loader references. Improved the format
consistency of the example pages
2024-01-02 14:59:40 -08:00