Commit Graph

3440 Commits (f1313339ac6f45bcf0afcf572d372c7e3660c0af)

Author SHA1 Message Date
glaze f7ad14acfa
Add etherscan document loader (#7943)
@rlancemartin 
The modification includes:
* etherscanLoader
* test_etherscan
* document ipynb

I have run the test, lint, format, and spell check. I do encounter a
linting error on ipynb, I am not sure how to address that.
```
docs/extras/modules/data_connection/document_loaders/integrations/Etherscan.ipynb:55: error: Name "null" is not defined  [name-defined]
docs/extras/modules/data_connection/document_loaders/integrations/Etherscan.ipynb:76: error: Name "null" is not defined  [name-defined]
Found 2 errors in 1 file (checked 1 source file)
```
- Description: The Etherscan loader uses etherscan api to load
transaction histories under specific accounts on Ethereum Mainnet.
- No dependency is introduced by this PR.
- Twitter handle: glazecl

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Julien Salinas 73d5cba308
Allow user to modify the GPU and language settings when using NLP Cloud (#7985)
---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Liu Ming 24f889f2bc
Change with_history option to False for ChatGLM by default (#8076)
ChatGLM LLM integration will by default accumulate conversation
history(with_history=True) to ChatGLM backend api, which is not expected
in most cases. This PR set with_history=False by default, user should
explicitly set llm.with_history=True to turn this feature on. Related
PR: #8048 #7774

---------

Co-authored-by: mlot <limpo2000@gmail.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Mahip Soni 1f055775f8
Fixing issue with MSSQL connection (#8040)
My team recently faced an issue while using MSSQL and passing a schema
name.

We noticed that "SET search_path TO {self.schema}" is being called for
us, which is not a valid ms-sql query, and is specific to postgresql
dialect.

We were able to run it locally after this fix.


---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Anthony Mahanna 76102971c0
ArangoDB/AQL support for Graph QA Chain (#7880)
**Description**: Serves as an introduction to LangChain's support for
[ArangoDB](https://github.com/arangodb/arangodb), similar to
https://github.com/hwchase17/langchain/pull/7165 and
https://github.com/hwchase17/langchain/pull/4881

**Issue**: No issue has been created for this feature

**Dependencies**: `python-arango` has been added as an optional
dependency via the `CONTRIBUTING.md` guidelines
 
**Twitter handle**: [at]arangodb

- Integration test has been added
- Notebook has been added:
[graph_arangodb_qa.ipynb](https://github.com/amahanna/langchain/blob/master/docs/extras/modules/chains/additional/graph_arangodb_qa.ipynb)

[![Open In
Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/amahanna/langchain/blob/master/docs/extras/modules/chains/additional/graph_arangodb_qa.ipynb)

```
docker run -p 8529:8529 -e ARANGO_ROOT_PASSWORD= arangodb/arangodb
```

```
pip install git+https://github.com/amahanna/langchain.git
```

```python
from arango import ArangoClient

from langchain.chat_models import ChatOpenAI
from langchain.graphs import ArangoGraph
from langchain.chains import ArangoGraphQAChain

db = ArangoClient(hosts="localhost:8529").db(name="_system", username="root", password="", verify=True)

graph = ArangoGraph(db)

chain = ArangoGraphQAChain.from_llm(ChatOpenAI(temperature=0), graph=graph)

chain.run("Is Ned Stark alive?")
```

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Adilkhan Sarsen 3e7d2a1b64
SelfQuery support for deeplake (#7888)
Added support SelfQuery for Deeplake
1 year ago
Leonid Ganeline c580c81cca
docstrings `experimental` (#7969)
- added/changed docstring for `experimental`
- added/changed docstrings for different artifacts
- 
@baskaryan
1 year ago
Leonid Ganeline 3eb4112a1f
Refactored `example_generator` (#8099)
Refactored `example_generator.py`. The same as #7961 
`example_generator.py` is in the root code folder. This creates the
`langchain.example_generator: Example Generator ` group on the API
Reference navigation ToC, on the same level as `Chains` and `Agents`
which is not correct.

Refactoring:
- moved `example_generator.py` content into
`chains/example_generator.py` (not in `utils` because the
`example_generator` has dependencies on other LangChain classes. It also
doesn't work for moving into `utilities/`)
- added the backwards compatibility ref in the original
`example_generator.py`

@hwchase17
1 year ago
Leonid Ganeline 7cbe28ba9b
Refactored `input` (#8202)
Refactored `input.py`. The same as
https://github.com/langchain-ai/langchain/pull/7961 #8098 #8099
input.py is in the root code folder. This creates the `langchain.input:
Input` group on the API Reference navigation ToC, on the same level as
Chains and Agents which is incorrect.

Refactoring:

- copied input.py file into utils/input.py
- I added the backwards compatibility ref in the original input.py. 
- changed several imports to a new ref

@hwchase17, @baskaryan
1 year ago
Monty Evans 72eb4fa4e8
Change WebBaseLoader metadata parsing to set missing metadata to descriptive string instead of `None` (#8175)
Solves #8174 & #3542

Co-authored-by: mevans <mevans@palantir.com>
1 year ago
Bagatur 1a7d8667c8
Bagatur/gateway chat (#8198)
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>
1 year ago
Ettore Di Giacinto ae28568e2a
Add embeddings for LocalAI (#8134)
Description:

This PR adds embeddings for LocalAI (
https://github.com/go-skynet/LocalAI ), a self-hosted OpenAI drop-in
replacement. As LocalAI can re-use OpenAI clients it is mostly following
the lines of the OpenAI embeddings, however when embedding documents, it
just uses string instead of sending tokens as sending tokens is
best-effort depending on the model being used in LocalAI. Sending tokens
is also tricky as token id's can mismatch with the model - so it's safer
to just send strings in this case.

Partly related to: https://github.com/hwchase17/langchain/issues/5256

Dependencies: No new dependencies

Twitter: @mudler_it
---------

Signed-off-by: mudler <mudler@localai.io>
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Mike Nitsenko d983046f90
Extend Cube Semantic Loader functionality (#8186)
**PR Description:**

This pull request introduces several enhancements and new features to
the `CubeSemanticLoader`. The changes include the following:

1. Added imports for the `json` and `time` modules.
2. Added new constructor parameters: `load_dimension_values`,
`dimension_values_limit`, `dimension_values_max_retries`, and
`dimension_values_retry_delay`.
3. Updated the class documentation with descriptions for the new
constructor parameters.
4. Added a new private method `_get_dimension_values()` to retrieve
dimension values from Cube's REST API.
5. Modified the `load()` method to load dimension values for string
dimensions if `load_dimension_values` is set to `True`.
6. Updated the API endpoint in the `load()` method from the base URL to
the metadata endpoint.
7. Refactored the code to retrieve metadata from the response JSON.
8. Added the `column_member_type` field to the metadata dictionary to
indicate if a column is a measure or a dimension.
9. Added the `column_values` field to the metadata dictionary to store
the dimension values retrieved from Cube's API.
10. Modified the `page_content` construction to include the column title
and description instead of the table name, column name, data type,
title, and description.

These changes improve the functionality and flexibility of the
`CubeSemanticLoader` class by allowing the loading of dimension values
and providing more detailed metadata for each document.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Bagatur 82b8d8596c
bump lc241 exp3 (#8193) 1 year ago
Leonid Ganeline 848454d1e7
Refactored `formatting` (#8191)
Refactored `formatting.py`. The same as
https://github.com/langchain-ai/langchain/pull/7961 #8098 #8099
formatting.py is in the root code folder. This creates the
`langchain.formatting: Formatting` group on the API Reference navigation
ToC, on the same level as Chains and Agents which is incorrect.

Refactoring:

- moved formatting.py content into utils/formatting.py
- I did not add the backwards compatibility ref in the original
formatting.py. It seems unnecessary.
---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Bagatur 4928f7a9f5
undo bump (#8192) 1 year ago
Leonid Ganeline 120cdf813d
docstrings `memory` (#8018)
docstrings `memory`:
- added module summary
- added missed docstrings
- updated docstrings into consistent format
- 
@baskaryan
1 year ago
Bagatur d5689d58ab
Bagatur/bump 241 (#8182) 1 year ago
Harrison Chase 3caccf304c
Harrison/hugginggpt (#8162)
Co-authored-by: Yongliang Shen <withsyl@163.com>
1 year ago
rajib f3908627ed
changed to mlflow-ai-gateway in llms/__init__.py (#8114)
- Description: In the llms/__init__.py, the key name is wrong for
mlflowaigateway. It should be mlflow-ai-gateway
  - Issue: NA
  - Dependencies: NA
  - Tag maintainer: @hwchase17, @baskaryan
  - Twitter handle: na

Without this fix, when we run the code for mlflowaigateway, we will get
error as below

ValueError: Loading mlflow-ai-gateway LLM not supported

---------

Co-authored-by: rajib76 <rajib76@yahoo.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Gordon Clark 80b3ec5869
GitHub toolkit improvements (#8121)
Fixes an issue with the github tool where the API returned special
objects but the tool was expecting dictionaries.

Also added proper docstrings to the GitHubAPIWraper methods and a (very
basic) integration test.

Maintainer responsibilities:
  - Agents / Tools / Toolkits: @hinthornw

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
1 year ago
shibuiwilliam 8f5000146c
add faiss test for score threshold (#8143)
# What
- Add faiss vector search test for score threshold
- Fix failing faiss vector search test; filtering with list value is
wrong.

<!-- Thank you for contributing to LangChain!

Replace this comment with:
- Description: Add faiss vector search test for score threshold; Fix
failing faiss vector search test; filtering with list value is wrong.
  - Issue: None
  - Dependencies: None
  - Tag maintainer: @rlancemartin, @eyurtsev
  - Twitter handle: @MlopsJ

Please make sure you're PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the
same people again.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
1 year ago
Nolan 7686dabd36
Unbreak devcontainer (#8154)
Codespaces and devcontainer was broken by the [repo
restructure](https://github.com/langchain-ai/langchain/discussions/8043).



- Description: Add libs/langchain to container so it can be built
without error.
  - Issue: -
  - Dependencies: -
  - Tag maintainer: @hwchase17 @baskaryan 
  - Twitter handle: @finnless

The failed build log says:
```
#10 [langchain-dev-dependencies 2/2] RUN poetry install --no-interaction --no-ansi --with dev,test,docs
#10 sha256:e850ee99fc966158bfd2d85e82b7c57244f47ecbb1462e75bd83b981a56a1929
2023-07-23 23:30:33.692Z: #10 0.827 
#10 0.827 Directory libs/langchain does not exist
2023-07-23 23:30:33.738Z: #10 ERROR: executor failed running [/bin/sh -c poetry install --no-interaction --no-ansi --with dev,test,docs]: exit code: 1
```

The new pyproject.toml imports from libs/langchain:

77bf75c236/pyproject.toml (L14-L16)

But libs/langchain is never added to the dev.Dockerfile:


77bf75c236/libs/langchain/dev.Dockerfile (L37-L39)
1 year ago
Harrison Chase 9205919ad2
actually use input key (#8136) 1 year ago
Leonid Ganeline 670304a8b3
simplified nmspace (#8152)
recreated #7894 (it is easy to recreate than resolve conflicts)
A small refactoring to improve the API Reference Agents table
 @baskaryan
1 year ago
William FH c5b50be225
Function calling logging fixup (#8153)
Fix bad overwriting of "functions" arg in invocation params.
Cleanup precedence in the dict
Clean up some inappropriate types (mapping should be dict)


Example:
https://dev.smith.langchain.com/public/9a7a6817-1679-49d8-8775-c13916975aae/r


![image](https://github.com/langchain-ai/langchain/assets/13333726/94cd0775-b6ef-40c3-9e5a-3ab65e466ab9)
1 year ago
SlapDrone 961a0e200f
Implement AgentExecutorIterator (#6929)
- Description: Implements a `.iter()` method for the `AgentExecutor`
class. This allows hooking into and intercepting intermediate agent
steps.
  - Issue: #6925 
  - Dependencies: None
  - Tag maintainer: @vowelparrot @agola11 
  - Twitter handle: @SlapDron3 @lacicocodes

---------

Co-authored-by: Lacico <Lacicocodes@gmail.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
1 year ago
Harrison Chase 77bf75c236
bump experimental to 002 (#8150) 1 year ago
Harrison Chase e46126eac6
add llamaapi (#8140) 1 year ago
Harrison Chase cbf2fc8af8
prompt ergonomics (#7799) 1 year ago
Harrison Chase 9f3073d418
bump versions (#8129) 1 year ago
Harrison Chase 86946a47a8
Harrison/add back in experimental (#8128) 1 year ago
Karthik Raja A 8b08687fc4
MultiOn client toolkit (#8110)
Addition of MultiOn Client Agent Toolkit
Dependencies: multion pip package
This PR consists of the following:
- MultiOn utility,tools and integration with agent
- sample jupyter notebook.
Request @hwchase17 , @hinthornw

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
1 year ago
Harrison Chase aa0e69bc98
Harrison/official pre release (#8106) 1 year ago
Philip Kiely - Baseten 95bcf68802
add kwargs support for Baseten models (#8091)
This bugfix PR adds kwargs support to Baseten model invocations so that
e.g. the following script works properly:

```python
chatgpt_chain = LLMChain(
    llm=Baseten(model="MODEL_ID"),
    prompt=prompt,
    verbose=False,
    memory=ConversationBufferWindowMemory(k=2),
    llm_kwargs={"max_length": 4096}
)
```
1 year ago
Harrison Chase 8dcabd9205
bump releases rc0 (#8097) 1 year ago
Harrison Chase d353d668e4
remove CVEs (#8092)
This PR aims to move all code with CVEs into `langchain.experimental`.
Note that we are NOT yet removing from the core `langchain` package - we
will give people a week to migrate here.

See MIGRATE.md for how to migrate

Zero changes to functionality

Vulnerabilities this addresses:

PALChain:
- https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5752409
- https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5759265

SQLDatabaseChain
- https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5759268

`load_prompt` (Python files only)
- https://security.snyk.io/vuln/SNYK-PYTHON-LANGCHAIN-5725807
1 year ago
Bagatur 08c658d3f8
fix api ref (#8083) 1 year ago
Harrison Chase da04760de1
Harrison/move experimental (#8084) 1 year ago
Harrison Chase f35db9f43e
(WIP) set up experimental (#7959) 1 year ago