Commit Graph

37 Commits

Author SHA1 Message Date
chyroc
a4ae4bc361
feat: mask api_key for konko (#14010)
for https://github.com/langchain-ai/langchain/issues/12165
2024-01-01 13:42:49 -08:00
NuODaniel
7773943a51
community:qianfan endpoint support init params & remove useless params definietion (#15381)
- **Description:**
- support custom kwargs in object initialization. For instantance, QPS
differs from multiple object(chat/completion/embedding with diverse
models), for which global env is not a good choice for configuration.
  - **Issue:** no
  - **Dependencies:** no
  - **Twitter handle:** no

@baskaryan PTAL
2024-01-01 13:12:31 -08:00
Ankush Gola
7eec8f2487
Delete V1 tracer and refactor tracer tests to core (#15326) 2023-12-29 15:55:56 -08:00
Shuai Liu
4b53440e70
Upgrades the Tongyi LLM and ChatTongyi Model (#14793)
- **Description:** fixes and upgrades for the Tongyi LLM and ChatTongyi
Model
      - Fixed typos; it should be `Tongyi`, not `OpenAI`.
- Fixed a bug in `stream_generate_with_retry`; it's a real stream
generator now.
- Fixed a bug in `validate_environment`; the `dashscope_api_key` should
be properly handled when set by environment variables or initialization
parameters.
- Changed the `dashscope` response to incremental output by setting the
parameter `incremental_output`, which eliminates the need for the
prefix-removal trick.
      - Removed some unused parameters, like `n`, `prefix_messages`.
      - Added `_stream` method.
- Added async methods support, such as `_astream`, `_agenerate`,
`_abatch`.
  - **Dependencies:** No new dependencies.
  - **Tag maintainer:** @hwchase17 

> PS: Some may be confused about the terms `dashscope`, `tongyi`, and
`Qwen`:
> - `dashscope`: A platform to deploy LLMs and provide APIs to invoke
the LLM.
> - `tongyi`: A brand name or overall term about Alibaba Cloud's LLM/AI.
> - `Qwen`: An LLM that is open-sourced and deployed in `dashscope`.
> 
> We use the `dashscope` SDK to interact with the `tongyi`-`Qwen` LLM.

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-12-29 12:06:12 -08:00
Diego Rani Mazine
ec72225265
refactor: enable connection pool usage in PGVector (#11514)
- **Description:** `PGVector` refactored to use connection pool.
  - **Issue:** #11433,
  - **Tag maintainer:** @hwchase17 @eyurtsev,

---------

Co-authored-by: Diego Rani Mazine <diego.mazine@mercadolivre.com>
Co-authored-by: Nuno Campos <nuno@langchain.dev>
2023-12-28 15:07:16 -08:00
chyroc
1abcf441ae
Refactor: use SecretStr for Predibase llms (#15119) 2023-12-26 13:01:42 -08:00
chyroc
0a9a73a9c9
Refactor: use SecretStr for PipelineAI llms (#15120) 2023-12-26 13:00:58 -08:00
chyroc
d63ceb65b3
Refactor: use SecretStr for StochasticAI llms (#15118) 2023-12-26 12:59:51 -08:00
chyroc
674fde87d2
Refactor: use SecretStr for VolcEngineMaas llms (#15117) 2023-12-26 12:59:08 -08:00
chyroc
3cc1da2b38
Refactor: use SecretStr for Petals llms (#15121) 2023-12-26 12:57:37 -08:00
ccurme
f2782f4c86
community: add args_schema to GmailSendMessage (#14973)
- **Description:** `tools.gmail.send_message` implements a
`SendMessageSchema` that is not used anywhere. `GmailSendMessage` also
does not have an `args_schema` attribute (this led to issues when
invoking the tool with an OpenAI functions agent, at least for me). Here
we add the missing attribute and a minimal test for the tool.
  - **Issue:** N/A
  - **Dependencies:** N/A
  - **Twitter handle:** N/A

---------

Co-authored-by: Chester Curme <chestercurme@microsoft.com>
2023-12-22 13:07:44 -08:00
Philip Kiely - Baseten
6342da333a
community: refactor Baseten integration with new API endpoints & docs (#15017)
- **Description:** In response to user feedback, this PR refactors the
Baseten integration with updated model endpoints, as well as updates
relevant documentation. This PR has been tested by end users in
production and works as expected.
  - **Issue:** N/A
- **Dependencies:** This PR actually removes the dependency on the
`baseten` package!
  - **Twitter handle:** https://twitter.com/basetenco
2023-12-22 12:46:24 -08:00
QIAN Zifei
2460f977c5
community[minor]: Azure DocumentIntelligenceLoader/Parser support update with latest SDK (#14389)
- **Description:**
Add DocumentIntelligenceLoader & DocumentIntelligenceParser
implementation using the latest Azure Document Intelligence SDK with
markdown support.
The core logic resides in DocumentIntelligenceParser and
DocumentIntelligenceLoader is a mere wrapper of the parser.
The parser will takes api_endpoint and api_key and creates
DocumentIntelligenceClient for the user. 4 parsing modes are supported:
1. Markdown (default)
2. Single
3. Page 
4. Object

UT and notebook are also updated accordingly.

- **Dependencies:** Azure Document Intelligence SDK:
azure-ai-documentintelligence
[azure-sdk-for-python/sdk/documentintelligence/azure-ai-documentintelligence
at 7c42462ac662522a6fd21b17d2a20f4cd40d0356 · Azure/azure-sdk-for-python
(github.com)](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-python%2Ftree%2F7c42462ac662522a6fd21b17d2a20f4cd40d0356%2Fsdk%2Fdocumentintelligence%2Fazure-ai-documentintelligence&data=05%7C01%7CZifei.Qian%40microsoft.com%7C298225aa3e31468a863108dbf07374ff%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638368150928704292%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=oE0Sl4HERnMKdbkV9KgBV46Z2xytcQAShdTWf7ZNl%2Bs%3D&reserved=0).

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
2023-12-21 16:40:27 -08:00
Ran
129a929d69
infra: Fix test filesystem paths incompatible with windows (#14388)
- **Description:** This PR fixes test failures on Windows caused by path
handling differences and unescaped special characters in regex. The
failing tests are:
```
FAILED tests/unit_tests/storage/test_filesystem.py::test_yield_keys - AssertionError: assert ['key1', 'subdir\\key2'] == ['key1', 'subdir/key2']
FAILED tests/unit_tests/test_imports.py::test_importable_all - ModuleNotFoundError: No module named 'langchain_community.langchain_community\\adapters'
FAILED tests/unit_tests/tools/file_management/test_utils.py::test_get_validated_relative_path_errs_on_absolute - re.error: incomplete escape \U at position 53
FAILED tests/unit_tests/tools/file_management/test_utils.py::test_get_validated_relative_path_errs_on_parent_dir - re.error: incomplete escape \U at position 69
FAILED tests/unit_tests/tools/file_management/test_utils.py::test_get_validated_relative_path_errs_for_symlink_outside_root - re.error: incomplete escape \U at position 64
```

- **Issue:** fixes
https://github.com/langchain-ai/langchain/issues/11775 (partially)
- **Dependencies:** none
2023-12-21 13:45:42 -08:00
Jacob Lee
1b01ee0e3c
community[minor]: add hf chat wrapper (#14736)
Builds on #14040 with community refactor merged and notebook updated.

Note that with this refactor, models will be imported from
`langchain_community.chat_models.huggingface` rather than the main
`langchain` repo.

---------

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>
Signed-off-by: ugm2 <unaigaraymaestre@gmail.com>
Signed-off-by: Yuchen Liang <yuchenl3@andrew.cmu.edu>
Co-authored-by: Andrew Reed <andrew.reed.r@gmail.com>
Co-authored-by: Andrew Reed <areed1242@gmail.com>
Co-authored-by: A-Roucher <aymeric.roucher@gmail.com>
Co-authored-by: Aymeric Roucher <69208727+A-Roucher@users.noreply.github.com>
2023-12-21 12:28:30 -05:00
Leonid Kuligin
b99274c9d8
community[patch]: changed default for VertexAIEmbeddings (#14614)
Replace this entire comment with:
- **Description:** @kurtisvg has raised a point that it's a good idea to
have a fixed version for embeddings (since otherwise a user might run a
query with one version vs a vectorstore where another version was used).
In order to avoid breaking changes, I'd suggest to give users a warning,
and make a `model_name` a required argument in 1.5 months.
2023-12-21 12:15:19 -05:00
Michael Landis
1c934fff0e
community[patch]: support momento vector index filter expressions (#14978)
**Description**

For the Momento Vector Index (MVI) vector store implementation, pass
through `filter_expression` kwarg to the MVI client, if specified. This
change will enable the MVI self query implementation in a future PR.

Also fixes some integration tests.
2023-12-20 19:11:43 -08:00
MING KANG
ed5e0cfe57
community: add OCI Endpoint (#14250)
- **Description:** 
- [OCI Data
Science](https://docs.oracle.com/en-us/iaas/data-science/using/home.htm)
is a fully managed and serverless platform for data science teams to
build, train, and manage machine learning models in the Oracle Cloud
Infrastructure. This PR add integration for using LangChain with an LLM
hosted on a [OCI Data Science Model
Deployment](https://docs.oracle.com/en-us/iaas/data-science/using/model-dep-about.htm).
To authenticate,
[oracle-ads](https://accelerated-data-science.readthedocs.io/en/latest/user_guide/cli/authentication.html)
has been used to automatically load credentials for invoking endpoint.
- **Issue:** None
- **Dependencies:** `oracle-ads`
- **Tag maintainer:** @baskaryan
- **Twitter handle:** None

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
2023-12-20 11:52:20 -08:00
Erick Friis
75ba22793f
community: Vectara summarization (#14970)
Description: Adding Summarization to Vectara, to reflect it provides not
only vector-store type functionality but also can return a summary.
Also added:
MMR capability (in the Vectara platform side)

Updated templates

Updated documentation and IPYNB examples

Tag maintainer: @baskaryan
Twitter handle: @ofermend

---------

Co-authored-by: Ofer Mendelevitch <ofermend@gmail.com>
2023-12-20 11:51:33 -08:00
Anush
60c70effe9
community[minor]: Qdrant sparse vector retriever (#14814)
## Description

This PR intends to add support for Qdrant's new [sparse vector
retrieval](https://qdrant.tech/articles/sparse-vectors/) by introducing
a new retriever class, `QdrantSparseVectorRetriever`.

Necessary usage docs and integration tests have been added for the
retriever.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-12-20 02:22:19 -05:00
AlpinDale
b0588774f1
community[minor]: Add Aphrodite Engine support (#14759)
This PR adds support for PygmalionAI's [Aphrodite
Engine](https://github.com/PygmalionAI/aphrodite-engine), based on
vLLM's attention mechanism. At the moment, this PR does not include
support for the API servers, but they will be added in a later PR.

The only dependency as of now is `aphrodite-engine==0.4.2`. We pin the
version to prevent breakage due to changes in the aphrodite-engine
library.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-12-20 01:16:57 -05:00
Michael Feil
7b96de3d5d
community[patch]: update Gradient embeddings (#14846)
- **Description:** Going forward, we have a own API `pip install
gradientai`. Therefore gradually removing the self-build packages in
llamaindex, haystack and langchain.
  - **Issue:** None.
  - **Dependencies:** `pip install gradientai`
  - **Tag maintainer:** @michaelfeil
2023-12-19 11:46:33 -05:00
Igor Dvorkin
6cc3c2452c
community[patch]: Enhance iMessage chat loader with timestamp parsing and message ownership (#14804)
---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-12-19 11:09:01 -05:00
Sirjanpreet Singh Banga
425e5e1791
community[minor]: rename ChatGPTRouter to GPTRouter (#14913)
**Description:**: Rename integration to GPTRouter 
**Tag maintainer:** @Gupta-Anubhav12 @samanyougarg @sirjan-ws-ext  
**Twitter handle:** [@SamanyouGarg](https://twitter.com/SamanyouGarg)
2023-12-19 10:48:52 -05:00
JaguarDB
992b04e475
community[minor]: added jaguar vector store (#14838)
Description: A new vector store Jaguar is being added. Class, test
scripts, and documentation is added.
Issue: None -- This is the first PR contributing to LangChain
Dependencies: This depends on "pip install -U jaguardb-http-client"
client http package
Tag maintainer: @baskaryan, @eyurtsev, @hwchase1
Twitter handle: @workbot

---------

Co-authored-by: JY <jyjy@jaguardb>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-12-19 10:40:18 -05:00
Sirjanpreet Singh Banga
44cb899a93
community[minor]: Integrating GPTRouter (#14900)
**Description:** Adding a langchain integration for
[GPTRouter](https://gpt-router.writesonic.com/) 🚀 ,
 **Tag maintainer:** @Gupta-Anubhav12 @samanyougarg @sirjan-ws-ext  
 **Twitter handle:** [@SamanyouGarg](https://twitter.com/SamanyouGarg)
 
Integration Tests Passing:
<img width="1137" alt="Screenshot 2023-12-19 at 5 45 31 PM"
src="https://github.com/Writesonic/langchain/assets/151817113/4a59df9a-ee30-47aa-9df9-b8c4eeb9dc76">
2023-12-19 10:08:36 -05:00
Erick Friis
5f839beab9
community: replace deprecated davinci models (#14860)
This is technically a breaking change because it'll switch out default
models from `text-davinci-003` to `gpt-3.5-turbo-instruct`, but OpenAI
is shutting off those endpoints on 1/4 anyways.

Feels less disruptive to switch out the default instead.
2023-12-18 13:49:46 -08:00
Vlad Kolesnikov
11fda490ca
community[minor]: New model parameters and dynamic batching for VertexAIEmbeddings (#13999)
- **Description:** VertexAIEmbeddings performance improvements
  - **Twitter handle:** @vladkol

## Improvements

- Dynamic batch size, starting from 250, lowering down to 5. Batch size
varies across regions.
Some regions support larger batches, and it significantly improves
performance.
When running large batches of texts in `us-central1`, performance gain
can be up to 3.5x.
The dynamic batching also makes sure every batch is below 20K token
limit.
- New model parameter `embeddings_type` that translates to `task_type`
parameter of the API. Newer model versions support [different embeddings
task
types](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings#api_changes_to_models_released_on_or_after_august_2023).
2023-12-17 22:24:22 -05:00
Harrison Chase
16399fd61d
langchain[patch]: remove unused imports (#14680)
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-12-15 14:12:02 -08:00
Karim Lalani
a0064330b1
community[minor]: Add SurrealDB vectorstore (#13331)
**Description:** Vectorstore implementation around
[SurrealDB](https://www.surrealdb.com)

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-12-15 13:34:51 -08:00
William FH
93c7eb4e6b
[Tracing] String Stacktrace (#14131)
Add full stacktrace
2023-12-14 22:15:07 -08:00
Leonid Kuligin
7f42811e14
google-genai[patch], community[patch]: Added support for new Google GenerativeAI models (#14530)
Replace this entire comment with:
  - **Description:** added support for new Google GenerativeAI models
  - **Twitter handle:** lkuligin

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
2023-12-14 20:56:46 -08:00
William FH
75b8891399
Update Vertex AI to include Gemini (#14670)
h/t to @lkuligin 
-  **Description:** added new models on VertexAI
  - **Twitter handle:** @lkuligin

---------

Co-authored-by: Leonid Kuligin <lkuligin@yandex.ru>
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-12-13 10:45:02 -08:00
Tomaz Bratanic
ea2616ae23
Fix RRF and lucene escape characters for neo4j vector store (#14646)
* Remove Lucene special characters (fixes
https://github.com/langchain-ai/langchain/issues/14232)
* Fixes RRF normalization for hybrid search
2023-12-13 09:09:50 -08:00
Bagatur
d388863a3b
community[patch]: Release 0.0.2 (#14610) 2023-12-12 09:58:04 -08:00
Bagatur
a844b495c4
community[patch]: Fix agenttoolkits imports (#14559) 2023-12-11 14:19:25 -08:00
Bagatur
ed58eeb9c5
community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)
Moved the following modules to new package langchain-community in a backwards compatible fashion:

```
mv langchain/langchain/adapters community/langchain_community
mv langchain/langchain/callbacks community/langchain_community/callbacks
mv langchain/langchain/chat_loaders community/langchain_community
mv langchain/langchain/chat_models community/langchain_community
mv langchain/langchain/document_loaders community/langchain_community
mv langchain/langchain/docstore community/langchain_community
mv langchain/langchain/document_transformers community/langchain_community
mv langchain/langchain/embeddings community/langchain_community
mv langchain/langchain/graphs community/langchain_community
mv langchain/langchain/llms community/langchain_community
mv langchain/langchain/memory/chat_message_histories community/langchain_community
mv langchain/langchain/retrievers community/langchain_community
mv langchain/langchain/storage community/langchain_community
mv langchain/langchain/tools community/langchain_community
mv langchain/langchain/utilities community/langchain_community
mv langchain/langchain/vectorstores community/langchain_community
mv langchain/langchain/agents/agent_toolkits community/langchain_community
mv langchain/langchain/cache.py community/langchain_community
mv langchain/langchain/adapters community/langchain_community
mv langchain/langchain/callbacks community/langchain_community/callbacks
mv langchain/langchain/chat_loaders community/langchain_community
mv langchain/langchain/chat_models community/langchain_community
mv langchain/langchain/document_loaders community/langchain_community
mv langchain/langchain/docstore community/langchain_community
mv langchain/langchain/document_transformers community/langchain_community
mv langchain/langchain/embeddings community/langchain_community
mv langchain/langchain/graphs community/langchain_community
mv langchain/langchain/llms community/langchain_community
mv langchain/langchain/memory/chat_message_histories community/langchain_community
mv langchain/langchain/retrievers community/langchain_community
mv langchain/langchain/storage community/langchain_community
mv langchain/langchain/tools community/langchain_community
mv langchain/langchain/utilities community/langchain_community
mv langchain/langchain/vectorstores community/langchain_community
mv langchain/langchain/agents/agent_toolkits community/langchain_community
mv langchain/langchain/cache.py community/langchain_community
```

Moved the following to core
```
mv langchain/langchain/utils/json_schema.py core/langchain_core/utils
mv langchain/langchain/utils/html.py core/langchain_core/utils
mv langchain/langchain/utils/strings.py core/langchain_core/utils
cat langchain/langchain/utils/env.py >> core/langchain_core/utils/env.py
rm langchain/langchain/utils/env.py
```

See .scripts/community_split/script_integrations.sh for all changes
2023-12-11 13:53:30 -08:00