Commit Graph

5877 Commits

Author SHA1 Message Date
Predrag Gruevski
ac7e88fbbe
Update rag-timescale-conversation to dependencies without CVEs. (#13364)
Just `poetry lock` and moving `langchain` to the latest version, in case
folks copy this template.

This resolves some vulnerable dependency alerts GitHub code scanning was
flagging.
2023-11-14 19:05:12 -08:00
Leonid Ganeline
342ed5c77a
Yi model from 01.ai , example (#13375)
Added an example with new soa `Yi` model to `HuggingFace-hub` notebook
2023-11-14 17:10:53 -08:00
Bagatur
38180ad25f
bump openai support (#13262) 2023-11-14 16:50:23 -08:00
Erick Friis
9545f0666d
fix cli release (#13373)
My thought is that the ==version would prevent pip from finding the
package on regular [pypi.org](http://pypi.org/), so it would look at
[test.pypi.org](http://test.pypi.org/) for that. Otherwise it'll pull
package from [pypi.org](http://pypi.org/) (e.g. sub deps)

Right now, the cli release is failing because it's going to
test.pypi.org by default, so it finds this incorrect FASTAPI package
instead of the real one: https://test.pypi.org/project/FASTAPI/
2023-11-14 15:08:35 -08:00
Erick Friis
7c3066f9ec
more cli interactivity, bugfix (#13360) 2023-11-14 14:49:43 -08:00
Bagatur
3596be5210
DOCS: format notebooks (#13371) 2023-11-14 14:17:44 -08:00
Predrag Gruevski
d63d4994c0
Bump all libraries to the latest ruff version. (#13350)
This version of `ruff` is the one we'll be using to lint the docs and
cookbooks (#12677), so I'm making it used everywhere else too.
2023-11-14 16:00:21 -05:00
Predrag Gruevski
2ebd167dba
Lint Python notebooks with ruff. (#12677)
The new ruff version fixed the blocking bugs, and I was able to fairly
easily us to a passing state: ruff fixed some issues on its own, I fixed
a handful by hand, and I added a list of narrowly-targeted exclusions
for files that are currently failing ruff rules that we probably should
look into eventually.

I went pretty lenient on the docs / cookbooks rules, allowing dead code
and such things. Perhaps in the future we may want to tighten the rules
further, but this is already a good set of checks that found real issues
and will prevent them going forward.
2023-11-14 15:58:22 -05:00
Massimiliano Pronesti
344cab0739
IMPROVEMENT: support Openai API v1 for Azure OpenAI completions (#13231)
Hi,
this PR adds support for OpenAI API v1 for Azure OpenAI completion API.
@baskaryan @hwchase17

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-11-14 12:10:18 -08:00
dependabot[bot]
fc886cc303
Bump pyarrow from 13.0.0 to 14.0.1 in /libs/langchain (#13363)
Bumps [pyarrow](https://github.com/apache/arrow) from 13.0.0 to 14.0.1.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ba53748361"><code>ba53748</code></a>
MINOR: [Release] Update versions for 14.0.1</li>
<li><a
href="529f3768fa"><code>529f376</code></a>
MINOR: [Release] Update .deb/.rpm changelogs for 14.0.1</li>
<li><a
href="b84bbcac64"><code>b84bbca</code></a>
MINOR: [Release] Update CHANGELOG.md for 14.0.1</li>
<li><a
href="f141709763"><code>f141709</code></a>
<a
href="https://redirect.github.com/apache/arrow/issues/38607">GH-38607</a>:
[Python] Disable PyExtensionType autoload (<a
href="https://redirect.github.com/apache/arrow/issues/38608">#38608</a>)</li>
<li><a
href="5a37e74198"><code>5a37e74</code></a>
<a
href="https://redirect.github.com/apache/arrow/issues/38431">GH-38431</a>:
[Python][CI] Update fs.type_name checks for s3fs tests (<a
href="https://redirect.github.com/apache/arrow/issues/38455">#38455</a>)</li>
<li><a
href="2dcee3f82c"><code>2dcee3f</code></a>
MINOR: [Release] Update versions for 14.0.0</li>
<li><a
href="297428cbf2"><code>297428c</code></a>
MINOR: [Release] Update .deb/.rpm changelogs for 14.0.0</li>
<li><a
href="3e9734f883"><code>3e9734f</code></a>
MINOR: [Release] Update CHANGELOG.md for 14.0.0</li>
<li><a
href="9f90995c8c"><code>9f90995</code></a>
<a
href="https://redirect.github.com/apache/arrow/issues/38332">GH-38332</a>:
[CI][Release] Resolve symlinks in RAT lint (<a
href="https://redirect.github.com/apache/arrow/issues/38337">#38337</a>)</li>
<li><a
href="bd61239a32"><code>bd61239</code></a>
<a
href="https://redirect.github.com/apache/arrow/issues/35531">GH-35531</a>:
[Python] C Data Interface PyCapsule Protocol (<a
href="https://redirect.github.com/apache/arrow/issues/37797">#37797</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/apache/arrow/compare/go/v13.0.0...go/v14.0.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pyarrow&package-manager=pip&previous-version=13.0.0&new-version=14.0.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/langchain-ai/langchain/network/alerts).

</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Predrag Gruevski <2348618+obi1kenobi@users.noreply.github.com>
2023-11-14 14:23:52 -05:00
Leonid Ganeline
f5bf3bdf14
added Cookbooks link (#13078)
It is a temporary solution before major documents refactoring.
Related to #13070 (not solving it)
2023-11-14 10:52:47 -08:00
Erick Friis
c0e6045c0b
cli 0.0.17 (#13359) 2023-11-14 09:56:18 -08:00
Erick Friis
927824b7cb
CLI interactivity (#13148)
Will implement more later
2023-11-14 09:53:29 -08:00
billytrend-cohere
2f6fe6ddf3
Fix latest message index (#13355)
There is a bug which caused the earliest message rather than the latest
message being sent
2023-11-14 09:23:25 -08:00
Manuel Soria
58f5a4d30a
Pgvector template (#13267)
Including pvector template, adapting what is covered in the
[cookbook](https://github.com/langchain-ai/langchain/blob/master/cookbook/retrieval_in_sql.ipynb).

---------

Co-authored-by: Lance Martin <lance@langchain.dev>
Co-authored-by: Erick Friis <erick@langchain.dev>
2023-11-14 07:47:48 -08:00
Harrison Chase
be854225c7
add more reasonable arxiv retriever (#13327) 2023-11-13 20:54:14 -08:00
Harrison Chase
4b7a85887e
arxiv retrieval agent improvement (#13329) 2023-11-13 20:54:03 -08:00
Krish Dholakia
5a920e14c0
fix litellm openai imports (#13307) 2023-11-13 17:55:10 -08:00
Bagatur
1c67db4c18
Move OAI assistants to langchain and add callbacks (#13236) 2023-11-13 17:42:07 -08:00
Bagatur
8006919e52
DOCS: cleanup docs directory (#13301) 2023-11-13 17:38:45 -08:00
Bagatur
c3f94f4c12
Update main readme (#13298) 2023-11-13 17:37:54 -08:00
Harrison Chase
5f60439221
add retrieval agent (#13317) 2023-11-13 17:22:39 -08:00
Harrison Chase
2ff30b50f2
FEATURE gpt researcher template (#13062)
Co-authored-by: Erick Friis <erick@langchain.dev>
2023-11-13 15:52:25 -08:00
Erick Friis
280ecfd8eb
IMPROVEMENT redirect root to docs in langserve app template (#13303) 2023-11-13 15:51:41 -08:00
wemysschen
a591cdb67d
add cookbook for RAG with baidu QIANFAN and elasticsearch (#13287)
**Description:** 
Add cookbook for RAG with baidu QIANFAN and elasticsearch.

Co-authored-by: wemysschen <root@icoding-cwx.bcc-szzj.baidu.com>
2023-11-13 14:45:24 -08:00
mertkayhan
9b4974871d
IMPROVEMENT Increase flexibility of ElasticVectorSearch (#6863)
Hey @rlancemartin, @eyurtsev ,

I did some minimal changes to the `ElasticVectorSearch` client so that
it plays better with existing ES indices.

Main changes are as follows:

1. You can pass the dense vector field name into `_default_script_query`
2. You can pass a custom script query implementation and the respective
parameters to `similarity_search_with_score`
3. You can pass functions for building page content and metadata for the
resulting `Document`

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
  4. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @dev2049
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @dev2049
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @vowelparrot
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the
same people again.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
2023-11-13 14:36:03 -08:00
Lance Martin
39852dffd2
Cookbook for multi-modal RAG eval (#13272) 2023-11-13 14:26:02 -08:00
Erick Friis
50a5c919f0
IMPROVEMENT self-query template (#13305)
- [ ]
https://github.com/langchain-ai/langchain/pull/12694#discussion_r1391334719
-> keep date
- [x]
https://github.com/langchain-ai/langchain/pull/12694#discussion_r1391336586
2023-11-13 14:03:15 -08:00
Yasin
b46f88d364
IMPROVEMENT add license file to subproject (#8403)
<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the
same people again.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->

hi!
This is pretty straight-forward: The sdist package does not contain the
license file (which is needed by e.g. conda) because the package is
built from the subdir and can't see the license.
I _copied_ the license but since I'm unfamiliar with the projects
direction, I'm not sure that's correct.
thanks!

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
2023-11-13 11:48:21 -08:00
Rui Ramos
ff19a62afc
Fix Pinecone cosine relevance score (#8920)
Fixes: #8207

Description:
Pinecone returns scores (not distances) with cosine similarity. The
values according to the docs are [-1, 1], although I could never
reproduce negative values.

This PR ensures that the score returned from Pinecone is preserved,
rather than inverted, so the most relevant documents can be filtered (eg
when using similarity thresholds)

I'll leave this as a draft PR as I couldn't run the tests (my pinecone
account might not be enough - some errors were being thrown around
namespaces) so hopefully someone who _can_ will pick this up.

Maintainers:
@rlancemartin, @eyurtsev

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
2023-11-13 11:47:38 -08:00
Bagatur
2e42ed5de6
Self-query template (#12694)
Co-authored-by: Erick Friis <erick@langchain.dev>
2023-11-13 11:44:19 -08:00
Konstantin Spieß
1e43025bf5
Fix serialization issue in Matching Engine Vector Store (#13266)
- **Description:** Fixed a serialization issue in the add_texts method
of the Matching Engine Vector Store caused by a typo, leading to an
attempt to serialize the json module itself.
  - **Issue:** #12154 
  - **Dependencies:** ./.
  - **Tag maintainer:**
2023-11-13 11:04:11 -08:00
William FH
9169d77cf6
Update error message in evaluation runner (#13296) 2023-11-13 11:03:20 -08:00
Leonie
32c493e3df
Refine Weaviate docs and add RAG example (#13057)
- **Description:** Refine Weaviate tutorial and add an example for
Retrieval-Augmented Generation (RAG)
  - **Issue:** (not applicable),
  - **Dependencies:** none
  - **Tag maintainer:** @baskaryan <!--
If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
  - **Twitter handle:** @helloiamleonie

Co-authored-by: Leonie <leonie@Leonies-MBP-2.fritz.box>
2023-11-13 10:59:19 -08:00
takatost
f22f273f93
FIX: 'from_texts' method in Weaviate with non-existent kwargs param (#11604)
Due to the possibility of external inputs including UUIDs, there may be
additional values in **kwargs, while Weaviate's `__init__` method does
not support passing extra **kwarg parameters.

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
2023-11-13 10:32:20 -08:00
Frank995
971d2b2e34
Add missing filter to max_marginal_relevance_search inner call to max_marginal_relevance_search_by_vector (#13260)
When calling max_marginal_relevance_search from PGVector the filter
param is not carried over to max_marginal_relevance_search_by_vector

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-11-13 10:31:34 -08:00
chevalmuscle
3ad78e48e2
Use endpoint_url if provided with boto3 session for dynamodb (#11622)
- **Description:** Uses `endpoint_url` if provided with a boto3 session.
When running dynamodb locally, credentials are required even if invalid.
With this change, it will be possible to pass a boto3 session with
credentials and specify an endpoint_url

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Erick Friis <erick@langchain.dev>
2023-11-13 10:31:16 -08:00
Erick Friis
18acc22f29
Ollama pass kwargs as options instead of top (#13280)
Noticed params are really in `options` instead while reviewing #12895
2023-11-13 10:28:47 -08:00
刘 方瑞
46af56dc4f
Add MyScaleWithoutJSON which allows user to wrap columns into Document's Metadata (#13164)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
Replace this entire comment with:
- **Description:** Add MyScaleWithoutJSON which allows user to wrap
columns into Document's Metadata
  - **Tag maintainer:** @baskaryan
2023-11-13 10:10:36 -08:00
Michael Landis
2aa13f1e10
chore: bump momento dependency version and refactor search hit usage (#13111)
**Description**

Bumps the Momento dependency to the latest version and refactors the
usage of `SearchHit` in the Momento Vector Index (MVI) vector store
integration. This change is a one liner where we use the preferred
attribute `score` to read the query-document similarity instead of
`distance`. The latest versions of Momento clients will use this
attribute going forward.

**Dependencies**

Updated the Momento dependency to latest version.

**Tests**

💚 I re-ran the existing MVI integration tests
(`tests/integration_tests/vectorstores/test_momento_vector_index.py`)
and they pass.

**Review**
cc @baskaryan @eyurtsev
2023-11-13 09:12:21 -08:00
Junlin Zhou
4da2faba41
docs: align custom_tool document headers (#13252)
On the [Defining Custom
Tools](https://python.langchain.com/docs/modules/agents/tools/custom_tools)
page, there's a 'Subclassing the BaseTool class' paragraph under the
'Completely New Tools - String Input and Output' header. Also there's
another 'Subclassing the BaseTool' paragraph under no header, which I
think may belong to the 'Custom Structured Tools' header.

Another thing is, there's a 'Using the tool decorator' and a 'Using the
decorator' paragraph, I think should belong to 'Completely New Tools -
String Input and Output' and 'Custom Structured Tools' separately.

This PR moves those paragraphs to corresponding headers.
2023-11-13 09:03:56 -08:00
Ikko Eltociear Ashimine
700293cae9
Fix typo in timescalevector.ipynb (#13239)
enviornment -> environment
2023-11-13 09:03:07 -08:00
kYLe
cc55d2fcee
Add OpenAI API v1 support for ChatAnyscale and fixed a bug with openai_api_key (#13237)
1. Add OpenAI API v1 support
2. Fixed a bug to call `get_secret_value` on a str value
(values["openai_api_key"])
2023-11-13 09:01:54 -08:00
juan-calvo-datatonic
545b76b0fd
Add rag google vertex ai search template (#13294)
- **Description:** This is a template demonstrating how to utilize
Google Vertex AI Search in conjunction with ChatVertexAI()
2023-11-13 08:45:36 -08:00
Govind.S.B
9024593468
added system prompt and template fields to ollama (#13022)
**Description**
the ollama api now supports passing system prompt and template directly
instead of modifying the model file , but the ollama integration in
langchain did not have this change updated . The update just adds these
two parameters to it ( there are 2 more parameters that are pending to
be updated, I was not sure about their utility wrt to langchain )
Refer :
8713ac23a8

**Issue** : None Applicable

**Dependencies** : None Changed

**Twitter handle** : https://twitter.com/violetto96

---------

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2023-11-13 08:45:11 -08:00
langchain-infra
f55f67055f
Add dockerfile template (#13240) 2023-11-13 10:33:01 -05:00
Shaurya Rohatgi
f70aa82c84
Update README.md - Added notebook for extraction_openai_tools (#13205)
added Parallel Function Calling for Structured Data Extraction notebook

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
2023-11-13 00:12:46 -08:00
Guillem Orellana Trullols
0f31cd8b49
Remove _get_kwarg_value function (#13184)
`_get_kwarg_value` function is useless, one can rely on python builtin
functionalities to do the exact same thing.

- **Description:** Removed `_get_kwarg_value`. Helps with code
readability.
  - **Issue:** the issue # it fixes (if applicable),
  - **Twitter handle:** @Guillem_96
2023-11-13 00:09:54 -08:00
SuperDa Fu
e1c020dfe1
dalle add model parameter (#13201)
- **Description:** dalle_image_generator adding a new model parameter,
  - **Issue:** N/A,
  - **Dependencies:** 
  - **Tag maintainer: @hwchase17
  - **Twitter handle:**

---------

Co-authored-by: dafu <xiangbingze@wenru.wang>
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
Co-authored-by: Erick Friis <erickfriis@gmail.com>
2023-11-13 00:09:20 -08:00
Mario Angst
96b56a4d4f
Typo fix to quickstart.mdx (#13178)
- **Description:** I fixed a very small typo in the quickstart docs
(BaeMessage -> BaseMessage)
2023-11-13 00:02:18 -08:00