Commit Graph

4883 Commits

Author SHA1 Message Date
Bagatur
24cb5cd379
bump 298 (#10892) 2023-09-21 08:26:11 -07:00
Bagatur
c1f9cc0bc5
recursive loader add status check (#10891) 2023-09-21 08:25:43 -07:00
Matvey Arye
6e02c45ca4
Add integration for Timescale Vector(Postgres) (#10650)
**Description:**
This commit adds a vector store for the Postgres-based vector database
(`TimescaleVector`).

Timescale Vector(https://www.timescale.com/ai) is PostgreSQL++ for AI
applications. It enables you to efficiently store and query billions of
vector embeddings in `PostgreSQL`:
- Enhances `pgvector` with faster and more accurate similarity search on
1B+ vectors via DiskANN inspired indexing algorithm.
- Enables fast time-based vector search via automatic time-based
partitioning and indexing.
- Provides a familiar SQL interface for querying vector embeddings and
relational data.

Timescale Vector scales with you from POC to production:
- Simplifies operations by enabling you to store relational metadata,
vector embeddings, and time-series data in a single database.
- Benefits from rock-solid PostgreSQL foundation with enterprise-grade
feature liked streaming backups and replication, high-availability and
row-level security.
- Enables a worry-free experience with enterprise-grade security and
compliance.

Timescale Vector is available on Timescale, the cloud PostgreSQL
platform. (There is no self-hosted version at this time.) LangChain
users get a 90-day free trial for Timescale Vector.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Avthar Sewrathan <avthar@timescale.com>
2023-09-21 07:33:37 -07:00
Michael Feil
55570e54e1
gradient.ai LLM intregration (#10800)
- **Description:** This PR implements a new LLM API to
https://gradient.ai
- **Issue:** Feature request for LLM #10745 
- **Dependencies**: No additional dependencies are introduced. 
- **Tag maintainer:** I am opening this PR for visibility, once ready
for review I'll tag.

- ```make format && make lint && make test``` is running.
- added a `integration` and `mock unit` test.


Co-authored-by: michaelfeil <me@michaelfeil.eu>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-09-21 07:29:16 -07:00
Bagatur
5097007407
cleanup recursive url session (#10863) 2023-09-21 07:22:13 -07:00
Harrison Chase
777b33b873
fix experimental imports (#10875) 2023-09-20 23:44:17 -07:00
Harrison Chase
808caca607
beef up agent docs (#10866) 2023-09-20 23:09:58 -07:00
Bagatur
4b558c9e17
update guide imports (#10865) 2023-09-20 17:02:46 -07:00
Sharath Rajasekar
96023f94d9
Add Javelin integration (#10275)
We are introducing the py integration to Javelin AI Gateway
www.getjavelin.io. Javelin is an enterprise-scale fast llm router &
gateway. Could you please review and let us know if there is anything
missing.

Javelin AI Gateway wraps Embedding, Chat and Completion LLMs. Uses
javelin_sdk under the covers (pip install javelin_sdk).

Author: Sharath Rajasekar, Twitter: @sharathr, @javelinai

Thanks!!
2023-09-20 16:36:39 -07:00
Bagatur
957956ba6d
bump 297 (#10861) 2023-09-20 14:45:49 -07:00
Harrison Chase
1bc3244db9
fix loading of sql chain (#10860)
Closing #6889
2023-09-20 14:37:49 -07:00
Harrison Chase
4074ea4c41
fix databricks docs (#10858) 2023-09-20 14:36:54 -07:00
Bagatur
405ba44d37
more redirects (#10859) 2023-09-20 14:26:51 -07:00
Bagatur
716c925a85
redirect platform to provider (#10857) 2023-09-20 14:17:36 -07:00
Bagatur
b05a74b106
fix recursive loader (#10856) 2023-09-20 13:55:47 -07:00
Bagatur
de0a02f507
fix extract sublink bug (#10855) 2023-09-20 13:30:42 -07:00
Harrison Chase
7dec2d399b
format intermediate steps (#10794)
Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>
2023-09-20 13:02:55 -07:00
Harrison Chase
386ef1e654
add agent output parsers (#10790) 2023-09-20 12:10:09 -07:00
Mukit Momin
67c5950df3
Amazon Bedrock Support Streaming (#10393)
### Description

- Add support for streaming with `Bedrock` LLM and `BedrockChat` Chat
Model.
- Bedrock as of now supports streaming for the `anthropic.claude-*` and
`amazon.titan-*` models only, hence support for those have been built.
- Also increased the default `max_token_to_sample` for Bedrock
`anthropic` model provider to `256` from `50` to keep in line with the
`Anthropic` defaults.
- Added examples for streaming responses to the bedrock example
notebooks.

**_NOTE:_**: This PR fixes the issues mentioned in #9897 and makes that
PR redundant.
2023-09-20 11:55:38 -07:00
Bagatur
0749a642f5
Stream refac and vertex streaming (#10470)
---------

Co-authored-by: Terry Cruz Melo <tcruz@vozy.co>
Co-authored-by: Terry Cruz Melo <33166112+TerryCM@users.noreply.github.com>
2023-09-20 11:49:16 -07:00
William FH
f421af8b80
Criteria Parser Improvements (#10824) 2023-09-20 11:18:33 -07:00
Bagatur
095f300bf6
add lcel how to index (#10850) 2023-09-20 10:19:43 -07:00
Bagatur
46aa90062b
bump exp 19 (#10851) 2023-09-20 10:17:52 -07:00
Bagatur
775f3edffd
bump 296 (#10842) 2023-09-20 08:31:14 -07:00
Bagatur
96a9c27116
fix recursive loader (#10752)
maintain same base url throughout recursion, yield initial page, fixing
recursion depth tracking
2023-09-20 08:16:54 -07:00
Nuno Campos
276125a33b
Use shallow copy on runnable locals (#10825)
- deep copy prevents storing complex objects in locals
2023-09-20 08:13:06 -07:00
DanielZzz
ebe08412ad
fix: chat_models Qianfan not compatiable with SystemMessage (#10642)
- **Description:** QianfanEndpoint bugs for SystemMessages. When the
`SystemMessage` is input as the messages to
`chat_models.QianfanEndpoint`. A `TypeError` will be raised.
  - **Issue:** #10643
  - **Dependencies:** 
  - **Tag maintainer:** @baskaryan
  - **Twitter handle:** no
2023-09-19 22:35:51 -07:00
Massimiliano Pronesti
f0198354d9
fix(embeddings): number of texts in Azure OpenAIEmbeddings batch (#10707)
This PR addresses the limitation of Azure OpenAI embeddings, which can
handle at maximum 16 texts in a batch. This can be solved setting
`chunk_size=16`. However, I'd love to have this automated, not to force
the user to figure where the issue comes from and how to solve it.

Closes #4575. 

@baskaryan

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-09-19 21:50:39 -07:00
Aashish Saini
7395c28455
corrected spelling (#62) (#10816) 2023-09-19 21:41:49 -07:00
zhanghexian
0abe996409
add clustered vearch in langchain (#10771)
---------

Co-authored-by: zhanghexian1 <zhanghexian1@jd.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-09-19 21:22:23 -07:00
HeTaoPKU
f505320a73
Add Minimax chat model (#10776)
resolve the merging issues for
https://github.com/langchain-ai/langchain/pull/6757

---------

Co-authored-by: 何涛 <taohe@bytedance.com>
2023-09-19 20:43:49 -07:00
Anar
c656a6b966
LLMRails (#10796)
### LLMRails Integration
This PR provides integration with LLMRails. Implemented here are:

langchain/vectorstore/llm_rails.py
tests/integration_tests/vectorstores/test_llm_rails.py
docs/extras/integrations/vectorstores/llm-rails.ipynb

---------

Co-authored-by: Anar Aliyev <aaliyev@mgmt.cloudnet.services>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-09-19 20:33:33 -07:00
mateai
900dbd1cbe
Substring support for similarity_search_with_score (#10746)
**Description:** Possible to filter with substrings in
similarity_search_with_score, for example: filter={'user_id':
{'substring': 'user'}}

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-09-19 20:32:44 -07:00
Ansil M B
740eafe41d
Updated return parameter of YouTubeSearchTool (#10743)
**Description:** 
changed return parameter of YouTubeSearchTool
 

1. changed the returning links of youtube videos by adding prefix
"https://www.youtube.com", now this will return the exact links to the
videos
2. updated the returning type from 'string' to 'list', which will be
more suited for further processings

 **Issue:** 
Fixes #10742

 **Dependencies:** 
None


<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** changed return parameter of YouTubeSearchTool
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** None
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-09-19 17:04:06 -07:00
Harrison Chase
1dae3c383e
Harrison/add submodule to docs (#10803) 2023-09-19 17:03:32 -07:00
Henry (Hezheng) Yin
c15bbaac31
misc: add gpt-3.5-turbo-instruct to model_token_mapping (#10808)
A one-line fix to get`max_tokens=-1` working `OpenAI` class for
`gpt-3.5-turbo-instruct` model.

Closes https://github.com/langchain-ai/langchain/issues/10806
2023-09-19 17:03:16 -07:00
Harrison Chase
5d0493f652
improve notebook (#10804) 2023-09-19 16:51:39 -07:00
Harrison Chase
d2bee34d4c
Harrison/add vald (#10807)
Co-authored-by: datelier <57349093+datelier@users.noreply.github.com>
2023-09-19 16:42:52 -07:00
Jacob Lee
bbc3fe259b
Start RunnableBranch callback tags with 1 instead of 0 (#10755)
Changes to match `RunnableSequences`

@eyurtsev
2023-09-19 16:38:08 -07:00
Ziyang Liu
931b292126
Add support for HTTP PUT in the open api agent prompt (#10763)
**Description:** This PR adds HTTP PUT support for the langchain openapi
agent toolkit by leveraging existing structure and HTTP put request
wrapper. The PUT method is almost identical to HTTP POST but should be
idempotent and therefore tighter than POST which is not idempotent. Some
APIs may consider to use PUT instead of POST which is unfortunately not
supported with the current toolkit yet.
2023-09-19 16:37:20 -07:00
Mateusz Wosinski
a29cd89923
Synthetic data generation (#9759)
### Description

Implements synthetic data generation with the fields and preferences
given by the user. Adds showcase notebook.
Corresponding prompt was proposed for langchain-hub.

### Example

```
output = chain({"fields": {"colors": ["blue", "yellow"]}, "preferences": {"style": "Make it in a style of a weather forecast."}})
print(output)

# {'fields': {'colors': ['blue', 'yellow']},
 'preferences': {'style': 'Make it in a style of a weather forecast.'},
 'text': "Good morning! Today's weather forecast brings a beautiful combination of colors to the sky, with hues of blue and yellow gently blending together like a mesmerizing painting."}
```

### Twitter handle 

@deepsense_ai @matt_wosinski

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-09-19 16:29:50 -07:00
Bagatur
c4a6de3fc9
Revert "Add ChatGLM for llm and chat_model by using ChatGLM API (#9797)" (#10805)
@etveritas reverting for now until this is resolved
https://github.com/langchain-ai/langchain/pull/9797/files#r1330795585,
apologies for merging too eagerly!
2023-09-19 16:23:42 -07:00
Mickaël
c86a1a6710
chore: allow using dataclasses_json dependency v0.6.0 (#10775)
**Description:** upgrade the `dataclasses_json` dependency to its latest
version ([no real breaking
change](https://github.com/lidatong/dataclasses-json/releases/tag/v0.6.0)
if used correctly), while allowing previous version to not break other
users' setup
**Issue:** I need to use the latest version of that dependency in my
project, but `langchain` prevents it.

Note: it looks like running `poetry lock --no-update` did some changes
to the lockfiles as it was the first time it was with the
`macosx_11_0_arm64` architecture 🤷

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-09-19 16:22:35 -07:00
Bagatur
76dd7480e6
Add batch_size param to Weaviate vector store (#9890)
cc @mcantillon21 @hsm207 @cs0lar
2023-09-19 16:20:23 -07:00
Mateusz Wosinski
720f6dbaac
Add XMLOutputParser (#10051)
**Description**
Adds new output parser, this time enabling the output of LLM to be of an
XML format. Seems to be particularly useful together with Claude model.
Addresses [issue
9820](https://github.com/langchain-ai/langchain/issues/9820).

**Twitter handle**
@deepsense_ai @matt_wosinski
2023-09-19 16:17:33 -07:00
etVERITAS
d6df288380
Add ChatGLM for llm and chat_model by using ChatGLM API (#9797)
using sample:
```
endpoint_url = API URL
ChatGLM_llm = ChatGLM(
    endpoint_url=endpoint_url,
    api_key=Your API Key by ChatGLM
)
print(ChatGLM_llm("hello"))
```

```
model = ChatChatGLM(
    chatglm_api_key="api_key",
    chatglm_api_base="api_base_url",
    model_name="model_name"
)
chain = LLMChain(llm=model)
```
Description: The call of ChatGLM has been adapted.
Issue: The call of ChatGLM has been adapted.
Dependencies: Need python package `zhipuai` and `aiostream`
Tag maintainer: @baskaryan
Twitter handle: None

I remove the compatibility test for pydantic version 2, because pydantic
v2 can't not pickle classmethod,but BaseModel use @root_validator is a
classmethod decorator.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-09-19 16:17:07 -07:00
Harrison Chase
d60145229b
make agent action serializable (#10797)
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2023-09-19 16:16:14 -07:00
Maxime Bourliatoux
21b236e5e4
Fixing _InactiveRpcError in MatchingEngine vectorstore (#10056)
- Description: There was an issue with the MatchingEngine VectorStore,
preventing from using it with a public endpoint. In the Google Cloud
library there are two similar methods for private or public endpoints :
`match()` and `find_neighbors()`.
  - Issue: Fixes #8378 
- This uses the `google.cloud.aiplatform` library :
https://github.com/googleapis/python-aiplatform/blob/main/google/cloud/aiplatform/matching_engine/matching_engine_index_endpoint.py
2023-09-19 16:16:04 -07:00
Sam Chou
4f19ba3065
Azure Search: Remove select field restrictions and expand metadata to other fields, also expose kwargs to searches (#9894)
Description: 
If metadata field returned in results, previous behavior unchanged. If
metadata field does not exist in results, expand metadata to any fields
returned outside of content field.

There's precedence for this as well, see the retriever:
https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/retrievers/azure_cognitive_search.py#L96C46-L96C46

Issue: 
#9765 - Ameliorates hard-coding in case you already indexed to cognitive
search without a metadata field but rather placed metadata in separate
fields.

@hwchase17
2023-09-19 16:10:29 -07:00
Piyush Jain
94cf71ecfa
Updated Neptune graph to use boto (#10121)
## Description
This PR updates the `NeptuneGraph` class to start using the boto API for
connecting to the Neptune service. With boto integration, the graph
class now supports authenticating requests using Sigv4; this is
encapsulated with the boto API, and users only have to ensure they have
the correct AWS credentials setup in their workspace to work with the
graph class.

This PR also introduces a conditional prompt that uses a simpler prompt
when using the `Anthropic` model provider. A simpler prompt have seemed
to work better for generating cypher queries in our testing.

**Note**: This version will require boto3 version 1.28.38 or greater to
work.
2023-09-19 16:03:08 -07:00