Commit Graph

6512 Commits (65a9193db240a45a5cbf1633f713875090511375)
 

Author SHA1 Message Date
Leonid Ganeline 65a9193db2
docs: `alibaba cloud` (#14772)
The [provider
page](https://python.langchain.com/docs/integrations/providers/alibabacloud_opensearch)
holds the vector store information. The [Chat
example](https://python.langchain.com/docs/integrations/chat/pai_eas_chat_endpoint)
was incorrectly sorted in the navbar because of the wrong file name.
- Recreated a provide page
- Added missed links and descriptions
- Compound information about vector store from two pages into one
- Fixed file name
7 months ago
Bagatur 99f839d6f3
infra: pr template update (#14963) 7 months ago
MING KANG ed5e0cfe57
community: add OCI Endpoint (#14250)
- **Description:** 
- [OCI Data
Science](https://docs.oracle.com/en-us/iaas/data-science/using/home.htm)
is a fully managed and serverless platform for data science teams to
build, train, and manage machine learning models in the Oracle Cloud
Infrastructure. This PR add integration for using LangChain with an LLM
hosted on a [OCI Data Science Model
Deployment](https://docs.oracle.com/en-us/iaas/data-science/using/model-dep-about.htm).
To authenticate,
[oracle-ads](https://accelerated-data-science.readthedocs.io/en/latest/user_guide/cli/authentication.html)
has been used to automatically load credentials for invoking endpoint.
- **Issue:** None
- **Dependencies:** `oracle-ads`
- **Tag maintainer:** @baskaryan
- **Twitter handle:** None

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
7 months ago
Erick Friis 75ba22793f
community: Vectara summarization (#14970)
Description: Adding Summarization to Vectara, to reflect it provides not
only vector-store type functionality but also can return a summary.
Also added:
MMR capability (in the Vectara platform side)

Updated templates

Updated documentation and IPYNB examples

Tag maintainer: @baskaryan
Twitter handle: @ofermend

---------

Co-authored-by: Ofer Mendelevitch <ofermend@gmail.com>
7 months ago
Erick Friis cf6951a0c9
docs: links (#14940) 7 months ago
Liang Zhang 6479aab74f
community[patch]: Add param "task" to Databricks LLM to work around serialization of transform_output_fn (#14933)
**What is the reproduce code?**

```python
from langchain.chains import LLMChain, load_chain
from langchain.llms import Databricks
from langchain.prompts import PromptTemplate

def transform_output(response):
    # Extract the answer from the responses.
    return str(response["candidates"][0]["text"])

def transform_input(**request):
    full_prompt = f"""{request["prompt"]}
    Be Concise.
    """
    request["prompt"] = full_prompt
    return request

chat_model = Databricks(
    endpoint_name="llama2-13B-chat-Brambles",
    transform_input_fn=transform_input,
    transform_output_fn=transform_output,
    verbose=True,
)
print(f"Test chat model: {chat_model('What is Apache Spark')}") # This works

llm_chain = LLMChain(llm=chat_model, prompt=PromptTemplate.from_template("{chat_input}"))
llm_chain("colorful socks") # this works
llm_chain.save("databricks_llm_chain.yaml") # transform_input_fn and transform_output_fn are not serialized into the model yaml file
loaded_chain = load_chain("databricks_llm_chain.yaml") # The Databricks LLM is recreated with transform_input_fn=None, transform_output_fn=None.
loaded_chain("colorful socks") # Thus this errors. The transform_output_fn is needed to produce the correct output
```


Error:
```
 File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-6c34afab-3473-421d-877f-1ef18930ef4d/lib/python3.10/site-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 1 validation error for Generation
text
  str type expected (type=type_error.str)
 request payload: {'query': 'What is a databricks notebook?'}'}
```

**What does the error mean?**

When the LLM generates an answer, represented by a Generation data
object. The Generation data object takes a str field called text, e.g.
Generation(text=”blah”). However, the Databricks LLM tried to put a
non-str to text, e.g. Generation(text={“candidates”:[{“text”: “blah”}]})
Thus, pydantic errors.

**Why the output format becomes incorrect after saving and loading the
Databricks LLM?**

Databrick LLM does not support serializing transform_input_fn and
transform_output_fn, so they are not serialized into the model yaml
file. When the Databricks LLM is loaded, it is recreated with
transform_input_fn=None, transform_output_fn=None. Without
transform_output_fn, the output text is not unwrapped, thus errors.

Missing transform_output_fn causes this error.
Missing transform_input_fn causes the additional prompt “Be Concise.” to
be lost after saving and loading.
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
7 months ago
Bagatur 1ea6d83188
langchain[patch]: Release 0.0.352 (#14961) 7 months ago
Bagatur b03845e069
community[patch]: Release 0.0.5 (#14960) 7 months ago
Bagatur a841f62791
core[patch]: 0.1.2 (#14959) 7 months ago
Anush 60c70effe9
community[minor]: Qdrant sparse vector retriever (#14814)
## Description

This PR intends to add support for Qdrant's new [sparse vector
retrieval](https://qdrant.tech/articles/sparse-vectors/) by introducing
a new retriever class, `QdrantSparseVectorRetriever`.

Necessary usage docs and integration tests have been added for the
retriever.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
7 months ago
mogith-pn c53fab63a3
community[patch]: Fixed duplicate input id issue in clarifai vectorstore (#14914)
- **Description:** 
This PR fixes the issue faces with duplicate input id in Clarifai
vectorstore class when ingesting documents into the vectorstore more
than the batch size.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
7 months ago
Sypherd 5642132c0c
community[patch]: Add safe lookup to OpenAI response adapter (#14765)
## Description
Similar to https://github.com/langchain-ai/langchain/issues/5861, I've
experienced `KeyError`s resulting from unsafe lookups in the
`convert_dict_to_message` function in [this
file](https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/adapters/openai.py).
While that issue focused on `KeyError 'content'`, I've opened another
issue (#14764) about how the problem still exists in the same function
but with `KeyError 'role'`. The fix for #5861 only added a safe lookup
to the specific line that was giving them trouble.. This PR fixes the
unsafe lookup in the rest of the function but the problem still exists
across the repo.

## Issues
* #14764
* #5861 

## Dependencies
* None

## Checklist
[x] make format
[x] make lint
[ ] make test - Results in `make: *** No rule to make target 'test'.
Stop.`

## Maintainers
* @hinthornw

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
7 months ago
AlpinDale b0588774f1
community[minor]: Add Aphrodite Engine support (#14759)
This PR adds support for PygmalionAI's [Aphrodite
Engine](https://github.com/PygmalionAI/aphrodite-engine), based on
vLLM's attention mechanism. At the moment, this PR does not include
support for the API servers, but they will be added in a later PR.

The only dependency as of now is `aphrodite-engine==0.4.2`. We pin the
version to prevent breakage due to changes in the aphrodite-engine
library.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
7 months ago
Dmitry Tyumentsev d21f44b484
community[minor]: Add YandexGPT embeddings (#14767)
- **Description:** Introducing an ability to work with the
[YandexGPT](https://cloud.yandex.com/en/services/yandexgpt) embeddings
models.
---------

Co-authored-by: Dmitry Tyumentsev <dmitry.tyumentsev@raftds.com>
7 months ago
Nicolas Suzor 529144649e
community[patch]: add png support for vertexai._parse_chat_history_gemini() (#14788)
- **Description:** Modify community chat model vertexai to handle png
and other image types encoded in base64
  - **Dependencies:** added `import re` but no new dependencies.

This addresses a problem where the vertexai method
_parse_chat_history_gemini() was only recognizing image uris in jpeg
format. I made a simple change to cover other extension types.
7 months ago
Dr. Christoph Mittendorf f348ad4ba8
docs: typo LLaMA2_sql_chat.ipynb (#14798)
"language" (right) vs "langugae" (wrong)
7 months ago
Liu Jun b0c48dc983
community[patch]: make ak and sk optional in qianfan endpoint (#14835)
- **Description:** The Qianfan SDK offers multiple authentication
methods, but in the `QianfanEndpoint` of Langchain, it currently only
supports authentication through AK and SK. In order to accommodate users
who wish to use alternative authentication methods, this pull request
makes AK and SK optional. This change should not impact existing users,
while allowing users to configure other authentication methods as per
the Qianfan SDK documentation.
  - **Issue:** /
  - **Dependencies:** No
  - **Tag maintainer:** No
  - **Twitter handle:**
7 months ago
Archan Ghosh 65678b3816
community[patch]: Update arxiv.py with Entry ID as a return value (#14915)
Added Entry ID as a return value inside get_summaries_as_docs

- **Description:** Added the Entry ID as a return, so it's easier to
track the IDs of the papers that are being returned.


With the addition return of the entry ID in functions like
ArxivRetriever, it will be easier to reference the ID of the paper
itself.
7 months ago
thehunmonkgroup dc20766513
docs: readme for langchain-mistralai (#14917)
- **Description:** Add README doc for MistralAI partner package.
  - **Tag maintainer:** @baskaryan
7 months ago
Elena Mata Yandiola b66659fc28
docs: Clarification google_cloud_storage_directory.ipynb (#14922)
- Description: Just a minor add to the documentation to clarify how to
load all files from a folder. I assumed and try to do it specifying it
in the bucket (BUCKET/FOLDER), instead of using the prefix.
7 months ago
Ari Roffe 8bcadfd446
docs: nit embedding_distance.ipynb (#14929)
**Description:** Fix the docs about embedding distance evaluations
guide.
7 months ago
Yacine 20eacd4b5e
docs: update notebook documentation for custom tool (#14942)
- **Description:** Documentation update. The custom tool notebook
documentation is updated to revome the warning caused by directly
instantiating of the LLMMathChain with an llm which is is deprecated.
The from_llm class method is used instead. LLM output results gets
updated as well.
  - **Issue:** no applicable
  - **Dependencies:** No dependencies
  - **Tag maintainer:** @baskaryan
  - **Twitter handle:** @ybouakkaz

Co-authored-by: Yacine Bouakkaz <Yacine.Bouakkaz@evokegroup.com>
7 months ago
Bagatur 345acb26ac
community[patch]: Matching engine, return doc id (#14930) 7 months ago
Erick Friis 8a3360edf6
anthropic: beta messages integration (#14928) 7 months ago
Erick Friis 795cf2ddda
together: package and embedding model (#14936) 7 months ago
Erick Friis c21379438c
docs: remove unused contributor steps (#14938) 7 months ago
William FH 758bcd4671
Add langsmith and benchmark repo links (#14931)
Think we could link to these in more places
7 months ago
João Galego d306d89a9b
template: Add Bedrock JCVD template (#14480)
This PR adds a simple LangChain template that uses [Anthropic's Claude
on Amazon Bedrock ⛰️](https://aws.amazon.com/bedrock/claude/) to behave
like JCVD.

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
7 months ago
Erick Friis 8b29b31554
cli: test_integration group (#14924) 7 months ago
Erick Friis 4d48aedea3
cli: 0.0.20 (#14920) 7 months ago
Erick Friis bbb20804bd
templates: fix sql-research-assistant (#14921) 7 months ago
Erick Friis 9ef2feb674
cli[patch]: add embedding to integration template (#14881) 7 months ago
Michael Feil 7b96de3d5d
community[patch]: update Gradient embeddings (#14846)
- **Description:** Going forward, we have a own API `pip install
gradientai`. Therefore gradually removing the self-build packages in
llamaindex, haystack and langchain.
  - **Issue:** None.
  - **Dependencies:** `pip install gradientai`
  - **Tag maintainer:** @michaelfeil
7 months ago
Igor Dvorkin 6cc3c2452c
community[patch]: Enhance iMessage chat loader with timestamp parsing and message ownership (#14804)
---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
7 months ago
Mohammad Mohtashim e3abe12243
community[patch]: helpful error message for GitHubAPIWrapper (#14803)
Very simple change in relation to the issue
https://github.com/langchain-ai/langchain/issues/14550

@baskaryan, @eyurtsev, @hwchase17.

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
7 months ago
Leonid Ganeline 922693caba
docs: `chunkviz` reference (#14802)
Added a reference to the `Chunkviz` utility.
7 months ago
Dmitry Tyumentsev 50381abc42
community[patch]: Add retry logic to Yandex GPT API Calls (#14907)
**Description:** Added logic for re-calling the YandexGPT API in case of
an error

---------

Co-authored-by: Dmitry Tyumentsev <dmitry.tyumentsev@raftds.com>
7 months ago
Sirjanpreet Singh Banga 425e5e1791
community[minor]: rename ChatGPTRouter to GPTRouter (#14913)
**Description:**: Rename integration to GPTRouter 
**Tag maintainer:** @Gupta-Anubhav12 @samanyougarg @sirjan-ws-ext  
**Twitter handle:** [@SamanyouGarg](https://twitter.com/SamanyouGarg)
7 months ago
JaguarDB 992b04e475
community[minor]: added jaguar vector store (#14838)
Description: A new vector store Jaguar is being added. Class, test
scripts, and documentation is added.
Issue: None -- This is the first PR contributing to LangChain
Dependencies: This depends on "pip install -U jaguardb-http-client"
client http package
Tag maintainer: @baskaryan, @eyurtsev, @hwchase1
Twitter handle: @workbot

---------

Co-authored-by: JY <jyjy@jaguardb>
Co-authored-by: Bagatur <baskaryan@gmail.com>
7 months ago
Bagatur a5be9f9475
mistralai: Add langchain-mistralai partner package (#14783)
Co-authored-by: Chad Phillips <chad@apartmentlines.com>
7 months ago
Sirjanpreet Singh Banga 44cb899a93
community[minor]: Integrating GPTRouter (#14900)
**Description:** Adding a langchain integration for
[GPTRouter](https://gpt-router.writesonic.com/) 🚀 ,
 **Tag maintainer:** @Gupta-Anubhav12 @samanyougarg @sirjan-ws-ext  
 **Twitter handle:** [@SamanyouGarg](https://twitter.com/SamanyouGarg)
 
Integration Tests Passing:
<img width="1137" alt="Screenshot 2023-12-19 at 5 45 31 PM"
src="https://github.com/Writesonic/langchain/assets/151817113/4a59df9a-ee30-47aa-9df9-b8c4eeb9dc76">
7 months ago
Bagatur 1069a93d18
langchain[patch]: export sagemaker LLMContentHandler (#14906)
Resolves #14904
7 months ago
Kostas Botsas 4f4b078bf3
docs: add reference for XataVectorStore constructor (#14903)
Adds doc reference to the XataVectorStore constructor for use with
existing Xata table contents.

@tsg @philkra
7 months ago
Leonid Ganeline b2fd41331e
docs: docstrings `langchain_community` update (#14889)
Addded missed docstrings. Fixed inconsistency in docstrings.

**Note** CC @efriis 
There were PR errors on
`langchain_experimental/prompt_injection_identifier/hugging_face_identifier.py`
But, I didn't touch this file in this PR! Can it be some cache problems?
I fixed this error.
7 months ago
William FH 583696732c
[Partner] NVIDIA TRT Package (#14733)
Simplify #13976 and add as a separate package.

- [] Add README
- [X] Add doc notebook
- [X] Add simple LLM integration

---------

Co-authored-by: Jeremy Dyer <jdye64@gmail.com>
7 months ago
William FH 0d4cbbcc85
[Partner] Update google integration test (#14883)
Gemini has decided that pickle rick is unsafe:
https://github.com/langchain-ai/langchain/actions/runs/7256642294/job/19769249444#step:8:189


![image](https://github.com/langchain-ai/langchain/assets/13333726/cfbf4312-53b6-4290-84ee-6ce0742e739e)
7 months ago
William FH f88af1f1cd
[Partner] Google GenAi new release (#14882)
to support the system message merging

Also fix integration tests that weren't passing
7 months ago
Leonid Kuligin 2d0f1cae8c
added history and support for system_message as param (#14824)
- **Description:** added support for chat_history for Google
GenerativeAI (to actually use the `chat` API) plus since Gemini
currently doesn't have a support for SystemMessage, added support for it
only if a user provides additional `convert_system_message_to_human`
flag during model initialization (in this case, SystemMessage would be
prepanded to the first HumanMessage)
  - **Issue:** #14710 
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
  - **Twitter handle:** lkuligin

---------

Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>
7 months ago
Leonid Ganeline 2861766d0d
Docs `tencent` pages update (#14879)
- updated `Tencent` provider page: added a chat model and document
loader references; company description
- updated Chat model and Document loader pages with descriptions, links
- renamed files to consistent formats; redirected file names
Note:
I was getting this linting error on code that **was not changed in my
PR**!

> Error:
docs/docs/guides/safety/hugging_face_prompt_injection.ipynb:1:1: I001
Import block is un-sorted or un-formatted
> make: *** [Makefile:47: lint_package] Error 1

I've fixed this error in the notebook
7 months ago
Timothy Ji c5a685b10b
OPENAI_PROXY not working (#14833)
Replace this entire comment with:
- **Description:** OPENAI_PROXY is not working for openai==1.3.9, The
`proxies` argument is deprecated. The `http_client` argument should be
passed instead,
  - **Issue:** OPENAI_PROXY is not working,
  - **Dependencies:** None,
  - **Tag maintainer:** @hwchase17 ,
  - **Twitter handle:** timothy66666
7 months ago