Commit Graph

2279 Commits (0ad76c3380055bfbd518348c3ec47548b9868d76)
 

Author SHA1 Message Date
Raduan Al-Shedivat 00c6ec8a2d
fix(document_loaders/telegram): fix pandas calls + add tests (#4806)
# Fix Telegram API loader + add tests.
I was testing this integration and it was broken with next error:
```python
message_threads = loader._get_message_threads(df)
KeyError: False
```
Also, this particular loader didn't have any tests / related group in
poetry, so I added those as well.

@hwchase17 / @eyurtsev please take a look on this fix PR.

---------

Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
Zander Chase 206c87d525
Change server start name (#4811)
to `langchain plus start/stop`
1 year ago
Eugene Yurtsev 255690d78e
Catch changes to test group (#4802)
# Catch changes to test group

Add test to catch changes to test group.
1 year ago
Eugene Yurtsev c3b6129beb
Block sockets for unit-tests (#4803)
# Block usage of sockets during unit tests

Catch any tests that attempt to use the network.
1 year ago
了空 f7e3d97b19
Remove unnecessary spaces from document object’s page_content of BiliBiliLoader (#4619)
- Remove unnecessary spaces from document object’s page_content of
BiliBiliLoader
- Fix BiliBiliLoader document and test file
1 year ago
Eugene Yurtsev f47ec5b4b6
Docugami docs: First cell should be a title cell (#4735)
# Make first cell a title in docugami docs

This makes the first cell a title cell in docugami notebook
1 year ago
Eugene Yurtsev d403f659ea
Update google protobuf dep (#4798)
# Update google protobuf dep

Resolve: https://github.com/hwchase17/langchain/security/dependabot/11
1 year ago
Eugene Yurtsev 3ecd7c9641
Add check to verify poetry.toml (#4794)
# Add poetry check to github action

Check poetry toml file during tests for errors
1 year ago
Ikko Eltociear Ashimine f5a476fdd4
Fix typo in dataframe.py (#4786)
# Fix typo in dataframe.py (#4786)

Fixed typo.
```
yeild -> yield
```
1 year ago
Eugene Yurtsev 14bedf1cc5
Github Action: Fix poetry lock file checking (#4789)
Fix how poetry lock file is checked to avoid skipping caches silently.
1 year ago
Davis Chase 7ce43372c3
Version 171 (#4788) 1 year ago
Zander Chase bee136efa4
Update Tracing Walkthrough (#4760)
Add client methods to read / list runs and sessions.

Update walkthrough to:
- Let the user create a dataset from the runs without going to the UI
- Use the new CLI command to start the server

Improve the error message when `docker` isn't found
1 year ago
Zander Chase fc0a3c8500
Persist Volume After Stop (#4763)
Previously, the data would be removed after shutting down the server.
This mounts a db volume that isn't erased between calls
1 year ago
Harrison Chase a7af32c274
Cassandra support for chat history (#4378) (#4764)
# Cassandra support for chat history

### Description

- Store chat messages in cassandra

### Dependency

- cassandra-driver - Python Module

## Before submitting

- Added Integration Test

## Who can review?

@hwchase17
@agola11

# Your PR Title (What it does)

<!--
Thank you for contributing to LangChain! Your PR will appear in our next
release under the title you set. Please make sure it highlights your
valuable contribution.

Replace this with a description of the change, the issue it fixes (if
applicable), and relevant context. List any dependencies required for
this change.

After you're done, someone will review your PR. They may suggest
improvements. If no one reviews your PR within a few days, feel free to
@-mention the same people again, as notifications can get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting

<!-- If you're adding a new integration, include an integration test and
an example notebook showing its use! -->

## Who can review?

Community members can review the PR once tests pass. Tag
maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

        @hwchase17 - project lead

        Tracing / Callbacks
        - @agola11

        Async
        - @agola11

        DataLoaders
        - @eyurtsev

        Models
        - @hwchase17
        - @agola11

        Agents / Tools / Toolkits
        - @vowelparrot
        
        VectorStores / Retrievers / Memory
        - @dev2049
        
 -->

Co-authored-by: Jinto Jose <129657162+jj701@users.noreply.github.com>
1 year ago
Harrison Chase c4c7936caa
Harrison/wiki loader (#4765)
Co-authored-by: Guillermo Segovia <T1b4lt@users.noreply.github.com>
1 year ago
Filip Haltmayer c632f7fc4e
Add Milvus and Zilliz Retrievals (#4416)
Adds the basic retrievers for Milvus and Zilliz. Hybrid search support
will be added in the future.

Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
1 year ago
Bradley James 2e43954bc3
fixed on_llm issue (#4717)
Fixes #4714
1 year ago
Zander Chase bf0904b676
Add Server Command (#4695)
Add Support for `langchain server {start|stop}` commands, with support for using ngrok to tunnel to a remote notebook
1 year ago
Anirudh Suresh 03ac39368f
Fixing DeepLake Overwrite Flag (#4683)
# Fix DeepLake Overwrite Flag Issue

Fixes Issue #4682: essentially, setting overwrite to False in the
DeepLake constructor still triggers an overwrite, because the logic is
just checking for the presence of "overwrite" in kwargs. The fix is
simple--just add some checks to inspect if "overwrite" in kwargs AND
kwargs["overwrite"]==True.

Added a new test in
tests/integration_tests/vectorstores/test_deeplake.py to reflect the
desired behavior.


Co-authored-by: Anirudh Suresh <ani@Anirudhs-MBP.cable.rcn.com>
Co-authored-by: Anirudh Suresh <ani@Anirudhs-MacBook-Pro.local>
Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
d 3 n 7 8bb32d77d0
Update utils.py to make headless an optional argument (#4745)
Making headless an optional argument for
create_async_playwright_browser() and create_sync_playwright_browser()
By default no functionality is changed.

This allows for disabled people to use a web browser intelligently with
their voice, for example, while still seeing the content on the screen.
As well as many other use cases

---------

Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
Mose Tronci a9dbe90447
Exponential back-off support for Google PaLM api (#4001)
This PR adds exponential back-off to the Google PaLM api to gracefully
handle rate limiting errors.

---------

Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
Leonid Ganeline a6f3ec94bc
docs: added `additional_resources` folder (#4748)
# docs: added `additional_resources` folder

The additional resource files were inside the doc top-level folder,
which polluted the top-level folder.
- added the `additional_resources` folder and moved correspondent files
to this folder;
- fixed a broken link to the "Model comparison" page (model_laboratory
notebook)
- fixed a broken link to one of the YouTube videos (sorry, it is not
directly related to this PR)

## Who can review?

@dev2049
1 year ago
Zander Chase a128d95aeb
Fix Async Shared Resource Bug (#4751)
Use an async queue to distribute tracers rather than inappropriately
sharing a single one
1 year ago
whuwxl 3f0357f94a
Add summarization task type for HuggingFace APIs (#4721)
# Add summarization task type for HuggingFace APIs

Add summarization task type for HuggingFace APIs.
This task type is described by [HuggingFace inference
API](https://huggingface.co/docs/api-inference/detailed_parameters#summarization-task)

My project utilizes LangChain to connect multiple LLMs, including
various HuggingFace models that support the summarization task.
Integrating this task type is highly convenient and beneficial.

Fixes #4720
1 year ago
Zander Chase 580861e7f2
Revert "Make serpapi base url configurable via env (#4402)" (#4750)
This reverts commit 5111bec540.

This PR introduced a bug in the async API (the `url` param isn't bound);
it also didn't update the synchronous API correctly, which makes it
error-prone (the behavior of the async and sync endpoints would be
different)
1 year ago
shiyu22 21b9397342
Update the milvus example (#4706)
# Fix issue when running example

- add the query content
- update the `user` parameter with Zilliz

Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
1 year ago
hilarious-viking 7d15669b41
llama-cpp: add gpu layers parameter (#4739)
Adds gpu layers parameter to llama.cpp wrapper

Co-authored-by: andrew.khvalenski <andrew.khvalenski@behavox.com>
Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
Davis Chase 36c9fd1af7
Dev2049/docs edit0 (#4699) 1 year ago
Jinto Jose 1e467d9fc4
Jupyter Notebook Example for using Mongodb to store Chat Message History (#4436)
# Jupyter Notebook Example for using Mongodb Chat Message History

@dev2049
1 year ago
Leonid Ganeline 6060505a9d
Add new links to `Tutorials` and `YouTube` pages (#4746)
- added an official LangChain YouTube channel :)
- added new tutorials and videos (only videos with enough subscriber or
view numbers)
- added a "New video" icon 

## Who can review?

@dev2049
1 year ago
Eduard van Valkenburg 47657fe01a
Tweaks to the PowerBI toolkit and utility (#4442)
Fixes some bugs I found while testing with more advanced datasets and
queries. Includes using the output of PowerBI to parse the error and
give that back to the LLM.
1 year ago
mvhensbergen e363e709cb
Add source field to metadata (#4462)
This is needed if one want to use index.query_with_sources on git files.
Without a source field, index.query_with_sources fails with an
exception.
1 year ago
vinoyang 5111bec540
Make serpapi base url configurable via env (#4402)
Fixes #4328

Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
Roma cb802edf75
[Feature] Add GraphQL Query Tool (#4409)
# Add GraphQL Query Support

This PR introduces a GraphQL API Wrapper tool that allows LLM agents to
query GraphQL databases. The tool utilizes the httpx and gql Python
packages to interact with GraphQL APIs and provides a simple interface
for running queries with LLM agents.

@vowelparrot

---------

Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
Eugene Yurtsev 49ce5ce1ca
Only run linkcheck against docs dir on PR (#4741)
# Only run linkchecker on direct changes to docs

This is a stop-gap that will speed up PRs.

Some broken links can slip through if they're embedded in doc-strings
inside the codebase.

But we'll still be running the linkchecker on master.
1 year ago
Eugene Yurtsev 99cfe71cd0
Check poetry lock file (#4740)
# Check poetry lock file on CI

This PR checks that the lock file is up to date using poetry lock
--check.

As part of this PR, a new lock file was generated.
1 year ago
Eugene Yurtsev 09587a3201
Clean up tests for pdf parsers (#4595)
# Organize tests for pdf parsers

Clean up tests for pdf parsers, remove duplicate tests, convert to unit
tests.
1 year ago
Leonid Ganeline 70fd7cda14
docs: `Concepts` (#4734)
# glossary.md renamed as concepts.md and moved under the Getting Started

small PR.
`Concepts` looks right to the point. It is moved under Getting Started
(typical place). Previously it was lost in the Additional Resources
section.

## Who can review?

 @hwchase17
1 year ago
Harrison Chase 8de81d34a1
bump version to 170 (#4733) 1 year ago
Harrison Chase dd95f0892d
Harrison/add top k (#4707)
Co-authored-by: blc16 <benlc@umich.edu>
1 year ago
Harrison Chase 0551594722
add async default (#4701)
a spin on
https://github.com/hwchase17/langchain/pull/4300/files#diff-4f16071d58cd34fb3ec5cd5089e9dbd6fb06574c25c76b4d573827f8a2f48e96
1 year ago
Zander Chase 97434a64c5
Add Environment Info to Run (#4691)
Store the environment info within the `extra` fields of the Run
1 year ago
Eugene Yurtsev d3300bd799
YouTube Loader: Replace regexp with built-in parsing (#4729) 1 year ago
Daniel Barker c70ae562b4
Added support for streaming output response to HuggingFaceTextgenInference LLM class (#4633)
# Added support for streaming output response to
HuggingFaceTextgenInference LLM class

Current implementation does not support streaming output. Updated to
incorporate this feature. Tagging @agola11 for visibility.
1 year ago
d 3 n 7 435b70da47
Update click.py to pass errors back to Agent (#4723)
Instead of halting the entire program if this tool encounters an error,
it should pass the error back to the agent to decide what to do.

This may be best suited for @vowelparrot to review.
1 year ago
Eugene Yurtsev 3c490b5ba3
Docugami DataLoader (#4727)
### Adds a document loader for Docugami

Specifically:

1. Adds a data loader that talks to the [Docugami](http://docugami.com)
API to download processed documents as semantic XML
2. Parses the semantic XML into chunks, with additional metadata
capturing chunk semantics
3. Adds a detailed notebook showing how you can use additional metadata
returned by Docugami for techniques like the [self-querying
retriever](https://python.langchain.com/en/latest/modules/indexes/retrievers/examples/self_query_retriever.html)
4. Adds an integration test, and related documentation

Here is an example of a result that is not possible without the
capabilities added by Docugami (from the notebook):

<img width="1585" alt="image"
src="https://github.com/hwchase17/langchain/assets/749277/bb6c1ce3-13dc-4349-a53b-de16681fdd5b">

---------

Co-authored-by: Taqi Jaffri <tjaffri@docugami.com>
Co-authored-by: Taqi Jaffri <tjaffri@gmail.com>
1 year ago
KNiski c2761aa8f4
Improve video_id extraction in YoutubeLoader (#4452)
# Improve video_id extraction in `YoutubeLoader`

`YoutubeLoader.from_youtube_url` can only deal with one specific url
format. I've introduced `YoutubeLoader.extract_video_id` which can
extract video id from common YT urls.

Fixes #4451 


@eyurtsev

---------

Co-authored-by: Kamil Niski <kamil.niski@gmail.com>
1 year ago
sqr 8b42e8a510
Update Makefile (typo) (#4725)
# Update minor typo in makefile
1 year ago
Lester Yang cd3f9865f3
Feature: pdfplumber PDF loader with BaseBlobParser (#4552)
# Feature: pdfplumber PDF loader with BaseBlobParser

* Adds pdfplumber as a PDF loader
* Adds pdfplumber as a blob parser.
1 year ago
Harrison Chase b6e3ac17c4
Harrison/sitemap local (#4704)
Co-authored-by: Lukas Bauer <lukas.bauer@mayflower.de>
1 year ago