Commit Graph

10845 Commits (c3ced4c6cec165ba1fb9b9a760140ad3df4862ec)
 

Author SHA1 Message Date
blueoom c3ced4c6ce
core[patch]: use time.monotonic() instead time.time() in InMemoryRateLimiter
**Description:**

The get time point method in the _consume() method of
core.rate_limiters.InMemoryRateLimiter uses time.time(), which can be
affected by system time backwards. Therefore, it is recommended to use
the monotonically increasing monotonic() to obtain the time

```python
        with self._consume_lock:
            now = time.time()  # time.time() -> time.monotonic()

            # initialize on first call to avoid a burst
            if self.last is None:
                self.last = now

            elapsed = now - self.last  # when use time.time(), elapsed may be negative when system time backwards

```
1 month ago
Eugene Yurtsev bd6c31617e
community[patch]: Remove more @allow_reuse=True validators (#25236)
Remove some additional allow_reuse=True usage in @root_validators.
1 month ago
Eugene Yurtsev 6e57aa7c36
community[patch]: Remove usage of @root_validator(allow_reuse=True) (#25235)
Remove usage of @root_validator(allow_reuse=True)
1 month ago
thiswillbeyourgithub a2b4c33bd6
community[patch]: FAISS: ValueError mentions normalize_score_fn isntead of relevance_score_fn (#25225)
Thank you for contributing to LangChain!

- [X] **PR title**: "community: fix valueerror mentions wrong argument
missing"
- Where "package" is whichever of langchain, community, core,
experimental, etc. is being modified. Use "docs: ..." for purely docs
changes, "templates: ..." for template changes, "infra: ..." for CI
changes.
  - Example: "community: add foobar LLM"


- [X] **PR message**: ***Delete this entire checklist*** and replace
with
- **Description:** when faiss.py has a None relevance_score_fn it raises
a ValueError that says a normalize_fn_score argument is needed.

Co-authored-by: ccurme <chester.curme@gmail.com>
1 month ago
ccurme 4825dc0d76
langchain[patch]: add deprecations (#24792) 1 month ago
ccurme 02300471be
langchain[patch]: extended-tests: drop logprobs from OAI expected config (#25234)
Following https://github.com/langchain-ai/langchain/pull/25229
1 month ago
Shivendra Soni 66b7206ab6
community: Add llm-extraction option to FireCrawl Document Loader (#25231)
**Description:** This minor PR aims to add `llm_extraction` to Firecrawl
loader. This feature is supported on API and PythonSDK, but the
langchain loader omits adding this to the response.
**Twitter handle:** [scalable_pizza](https://x.com/scalablepizza)

---------

Co-authored-by: Chester Curme <chester.curme@gmail.com>
1 month ago
blaufink c81c77b465
partners: fix of issue #24880 (#25229)
- Description: As described in the related issue: There is an error
occuring when using langchain-openai>=0.1.17 which can be attributed to
the following PR: #23691
Here, the parameter logprobs is added to requests per default.
However, AzureOpenAI takes issue with this parameter as stated here:
https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chatgpt?tabs=python-new&pivots=programming-language-chat-completions
-> "If you set any of these parameters, you get an error."
Therefore, this PR changes the default value of logprobs parameter to
None instead of False. This results in it being filtered before the
request is sent.
- Issue: #24880
- Dependencies: /

Co-authored-by: blaufink <sebastian.brueckner@outlook.de>
1 month ago
ccurme 3b7437d184
docs: update integration api refs (#25195)
- [x] toolkits
- [x] retrievers (in this repo)
1 month ago
Bagatur 91ea4b7449
infra: avoid orjson 3.10.7 in vercel build (#25212) 1 month ago
Isaac Francisco 652b3fa4a4
[docs]: playwright fix (#25163) 1 month ago
Bagatur 7040013140
core[patch]: fix deprecation pydantic bug (#25204)
#25004 is incompatible with pydantic < 1.10.17. Introduces fix for this.
1 month ago
Isaac Francisco dc7423e88f
[docs]: standardizing document loader integration pages (#25002) 1 month ago
Casey Clements 25f2e25be1
partners[patch]: Mongodb Retrievers - CI final touches. (#25202)
## Description

Contains 2 updates to for integration tests to run on langchain's CI.
Addendum to #25057 to get release github action to succeed.
1 month ago
Bagatur 786ef021a3
docs: redirect toolkits (#25190) 1 month ago
Eugene Yurtsev 429a0ee7fd
core[minor]: Add factory for looking up secrets from the env (#25198)
Add factory method for looking secrets from the env.
1 month ago
Erick Friis da9281feb2
cli: release 0.0.29 (#25196) 1 month ago
Erick Friis c6ece6a96d
core: autodetect more ls params (#25044)
Co-authored-by: ccurme <chester.curme@gmail.com>
1 month ago
Eugene Yurtsev 86355640c3
experimental[patch]: Use get_fields adapter (#25193)
Change all usages of __fields__ with get_fields adapter merged into
langchain_core.

Code mod generated using the following grit pattern:

```
engine marzano(0.1)
language python


`$X.__fields__` => `get_fields($X)` where {
    add_import(source="langchain_core.utils.pydantic", name="get_fields")
}
```
1 month ago
Eugene Yurtsev b9f65e5038
experimental[patch]: Migrate pydantic extra to literals (#25194)
Migrate pydantic extra to literals

Upgrade to using a literal for specifying the extra which is the
recommended approach in pydantic 2.

This works correctly also in pydantic v1.

```python
from pydantic.v1 import BaseModel

class Foo(BaseModel, extra="forbid"):
    x: int

Foo(x=5, y=1)
```

And 


```python
from pydantic.v1 import BaseModel

class Foo(BaseModel):
    x: int

    class Config:
      extra = "forbid"

Foo(x=5, y=1)
```


## Enum -> literal using grit pattern:

```
engine marzano(0.1)
language python
or {
    `extra=Extra.allow` => `extra="allow"`,
    `extra=Extra.forbid` => `extra="forbid"`,
    `extra=Extra.ignore` => `extra="ignore"`
}
```

Resorted attributes in config and removed doc-string in case we will
need to deal with going back and forth between pydantic v1 and v2 during
the 0.3 release. (This will reduce merge conflicts.)


## Sort attributes in Config:

```
engine marzano(0.1)
language python


function sort($values) js {
    return $values.text.split(',').sort().join("\n");
}


class_definition($name, $body) as $C where {
    $name <: `Config`,
    $body <: block($statements),
    $values = [],
    $statements <: some bubble($values) assignment() as $A where {
        $values += $A
    },
    $body => sort($values),
}

```
1 month ago
Eugene Yurtsev 30fb345342
core[minor]: Add from_env utility (#25189)
Add a utility that can be used as a default factory

The goal will be to start migrating from of the pydantic models to use
`from_env` as a default factory if possible.

```python

from pydantic import Field, BaseModel
from langchain_core.utils import from_env

class Foo(BaseModel):
   name: str = Field(default_factory=from_env('HELLO'))
```
1 month ago
Eugene Yurtsev 98779797fe
community[patch]: Use get_fields adapter for pydantic (#25191)
Change all usages of __fields__ with get_fields adapter merged into
langchain_core.

Code mod generated using the following grit pattern:

```
engine marzano(0.1)
language python


`$X.__fields__` => `get_fields($X)` where {
    add_import(source="langchain_core.utils.pydantic", name="get_fields")
}
```
1 month ago
Rajendra Kadam 663638d6a8
community[minor]: [SharePointLoader] Load extended metadata for the root folder (#24872)
- **Title:** [SharePointLoader] Load extended metadata for the root
folder
- **Description:** 
    - Ensure extended metadata loads correctly for the root folder.
- Cleanup: Refactor SharePointLoader to remove unused fields(`file_id` &
`site_id`).
- **Dependencies:** NA
- **Add tests and docs:** NA
1 month ago
Eugene Yurtsev 2f209d84fa
core[patch]: Add pydantic get_fields adapter (#25187)
Add adapter to get fields
1 month ago
Eugene Yurtsev c72e522e96
langchain[patch]: Upgrade pydantic extra (#25186)
Upgrade to using a literal for specifying the extra which is the
recommended approach in pydantic 2.

This works correctly also in pydantic v1.

```python
from pydantic.v1 import BaseModel

class Foo(BaseModel, extra="forbid"):
    x: int

Foo(x=5, y=1)
```

And 


```python
from pydantic.v1 import BaseModel

class Foo(BaseModel):
    x: int

    class Config:
      extra = "forbid"

Foo(x=5, y=1)
```


## Enum -> literal using grit pattern:

```
engine marzano(0.1)
language python
or {
    `extra=Extra.allow` => `extra="allow"`,
    `extra=Extra.forbid` => `extra="forbid"`,
    `extra=Extra.ignore` => `extra="ignore"`
}
```

Resorted attributes in config and removed doc-string in case we will
need to deal with going back and forth between pydantic v1 and v2 during
the 0.3 release. (This will reduce merge conflicts.)


## Sort attributes in Config:

```
engine marzano(0.1)
language python


function sort($values) js {
    return $values.text.split(',').sort().join("\n");
}


class_definition($name, $body) as $C where {
    $name <: `Config`,
    $body <: block($statements),
    $values = [],
    $statements <: some bubble($values) assignment() as $A where {
        $values += $A
    },
    $body => sort($values),
}

```
1 month ago
Eugene Yurtsev bf5193bb99
community[patch]: Upgrade pydantic extra (#25185)
Upgrade to using a literal for specifying the extra which is the
recommended approach in pydantic 2.

This works correctly also in pydantic v1.

```python
from pydantic.v1 import BaseModel

class Foo(BaseModel, extra="forbid"):
    x: int

Foo(x=5, y=1)
```

And 


```python
from pydantic.v1 import BaseModel

class Foo(BaseModel):
    x: int

    class Config:
      extra = "forbid"

Foo(x=5, y=1)
```


## Enum -> literal using grit pattern:

```
engine marzano(0.1)
language python
or {
    `extra=Extra.allow` => `extra="allow"`,
    `extra=Extra.forbid` => `extra="forbid"`,
    `extra=Extra.ignore` => `extra="ignore"`
}
```

Resorted attributes in config and removed doc-string in case we will
need to deal with going back and forth between pydantic v1 and v2 during
the 0.3 release. (This will reduce merge conflicts.)


## Sort attributes in Config:

```
engine marzano(0.1)
language python


function sort($values) js {
    return $values.text.split(',').sort().join("\n");
}


class_definition($name, $body) as $C where {
    $name <: `Config`,
    $body <: block($statements),
    $values = [],
    $statements <: some bubble($values) assignment() as $A where {
        $values += $A
    },
    $body => sort($values),
}

```
1 month ago
Isaac Francisco 11adc09e02
[docs]: change rag reference in vector store pages (#25125) 1 month ago
Anush 6b32810b68
qdrant: Update doc with usage snippets (#25179)
## Description

This PR adds back snippets demonstrating sparse and hybrid retrieval in
the Qdrant notebook.

Without the snippets, it's hard to grok the usage.
1 month ago
Eugene Yurtsev 3da2713172
docs: Update pydantic compatibility (#25145)
Update pydantic compatibility
1 month ago
Eugene Yurtsev 425f6ffa5b
core[patch]: Fix aindex API (#25155)
A previous PR accidentally broke the aindex API by renaming a positional
argument vectorstore into vector_store. This PR reverts this change.
1 month ago
Isaac Francisco 15a36dd0a2
[docs]: combine tools and toolkits (#25158) 1 month ago
ololand 249945a572
Update polygon.py for business subscription (#25085)
For business subscription the status is STOCKSBUSINESS not OK

Thank you for contributing to LangChain!

- [ ] **PR title**: "package: description"
- Where "package" is whichever of langchain, community, core,
experimental, etc. is being modified. Use "docs: ..." for purely docs
changes, "templates: ..." for template changes, "infra: ..." for CI
changes.
  - Example: "community: add foobar LLM"


- [ ] **PR message**: ***Delete this entire checklist*** and replace
with
    - **Description:** a description of the change
    - **Issue:** the issue # it fixes, if applicable
    - **Dependencies:** any dependencies required for this change
- **Twitter handle:** if your PR gets announced, and you'd like a
mention, we'll gladly shout you out!


- [ ] **Add tests and docs**: If you're adding a new integration, please
include
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.


- [ ] **Lint and test**: Run `make format`, `make lint` and `make test`
from the root of the package(s) you've modified. See contribution
guidelines for more: https://python.langchain.com/docs/contributing/

Additional guidelines:
- Make sure optional dependencies are imported within a function.
- Please do not add dependencies to pyproject.toml files (even optional
ones) unless they are required for unit tests.
- Most PRs should not touch more than one package.
- Changes should be backwards compatible.
- If you are adding something to community, do not re-import it in
langchain.

If no one reviews your PR within a few days, please @-mention one of
baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.
1 month ago
ccurme 59b8850909
groq[patch]: update rate limit in integration tests (#25177)
Divide by ~2 to account for testing python 3.8 and 3.12 in parallel.
1 month ago
Chad Juliano 4828c441a7
docs: Update notebook name for Kinetica (#25149)
**Description:** Change notebook description in documentation.
**Issue:** N/A
**Dependencies:** N/A
1 month ago
Francisco Kurucz 725e4912ae
docs: Fix reference to SQL QA migration (#25157)
**Description:** I found that the link to the notebook in the Migration
notes is broken, i found that it was linked to this file
https://github.com/langchain-ai/langchain/blob/v0.0.250/docs/extras/use_cases/tabular/sql_query.ipynb
and i think now this tutorial
https://github.com/JuanFKurucz/langchain/blob/master/docs/docs/tutorials/sql_qa.ipynb
is the best fit for this reference

**Twitter handle:** @juanfkurucz

---------

Co-authored-by: Chester Curme <chester.curme@gmail.com>
1 month ago
ogawa d895db11d6
community[patch]: gpt-4o-2024-08-06 costs (#25164)
- **Description:** updated OpenAI cost definitions according to the
following:
  - https://openai.com/api/pricing/
- **Twitter handle:** `@ogawa65a`

---------

Co-authored-by: Chester Curme <chester.curme@gmail.com>
1 month ago
Brace Sproul d77c7c4236
docs: Fix misspelling of instantiate in docs (#25107) 1 month ago
Eugene Yurtsev 7b1a132aff
core[patch]: Add unit tests for Serializable (#25152)
Add a few test cases for serializable (many other test cases already
covered
throguh runnable tests).
1 month ago
Bagatur df99b832a7
core[patch]: support Field deprecation (#25004)
![Screenshot 2024-08-02 at 4 23 17
PM](https://github.com/user-attachments/assets/c757e093-877e-4af6-9dcd-984195454158)
1 month ago
ccurme 803eba3163
core[patch]: check for model_fields attribute (#25108)
`__fields__` raises a warning in pydantic v2
1 month ago
Casey Clements 6e9a8b188f
mongodb: Add Hybrid and Full-Text Search Retrievers, release 0.2.0 (#25057)
## Description

This pull-request extends the existing vector search strategies of
MongoDBAtlasVectorSearch to include Hybrid (Reciprocal Rank Fusion) and
Full-text via new Retrievers.

There is a small breaking change in the form of the `prefilter` kwarg to
search. For this, and because we have now added a great deal of
features, including programmatic Index creation/deletion since 0.1.0, we
plan to bump the version to 0.2.0.

### Checklist
* Unit tests have been extended
* formatting has been applied
* One mypy error remains which will either go away in CI or be
simplified.

---------

Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Co-authored-by: Erick Friis <erick@langchain.dev>
1 month ago
Isaac Francisco f337408b0f
[docs]: add sidebar for different tool categories (#25065) 1 month ago
Bagatur 0b4608f71e
infra: temp skip oai embeddings test (#25148) 1 month ago
Bagatur a4086119f8
openai[patch]: Release 0.1.21rc2 (#25146) 1 month ago
Bagatur b4c12346cc
core[patch]: Release 0.2.29 (#25126) 1 month ago
Erick Friis dff83cce66
core[patch]: base language model disable_streaming (#25070)
Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>
1 month ago
eric-langenberg 130e80b60f
docs: rag.ipynb - fixing typo (#25142)
Just changing gpt-3.5 to gpt-4o-mini . That's what's used in the code
examples now. It just didn't get updated in the main text.
1 month ago
Bagatur 09fbce13c5
openai[patch]: ChatOpenAI.with_structured_output json_schema support (#25123) 1 month ago
maang-h 0ba125c3cd
docs: Standardize QianfanLLMEndpoint LLM (#25139)
- **Description:** Standardize QianfanLLMEndpoint LLM,include:
  - docs, the issue #24803 
  - model init arg names, the issue #20085
1 month ago
Eugene Yurtsev 28e0958ff4
core[patch]: Relax rate limit unit tests in terms of timing (#25140)
Relax rate limit unit tests
1 month ago