Commit Graph

1888 Commits (parallel_dir_loader)
 

Author SHA1 Message Date
blob42 c78075d0bb DirectoryLoader parallel file loading 1 year ago
Tim Asp ed0d557ede
docs: fix pdf docs hierarchy and formatting (#4593)
# Fix pdf loader docs page


![image](https://github.com/hwchase17/langchain/assets/707699/4a11f379-00ed-4f7a-9870-71f74e0cadc6)

Using h1's messes with hierarchy, this fixes that, and moves the
PyPDFium2 loader out of the middle of PDFMiner docs
1 year ago
Davis Chase 36f9e9a0ba
Skip flaky unit test (#4591) 1 year ago
Eugene Yurtsev 08ed927c32
Turn on extended tests (#4588)
# Turn on strict extended tests

This PR turns on strict testing for extended tests.
1 year ago
Zander Chase d96f6a106b
Add Steamship Image Generation Tool (#4580)
Co-authored-by: Enias Cailliau <enias@steamship.com>
1 year ago
Davis Chase 739c297c94
Release 167 (#4589) 1 year ago
Davis Chase a4a9d1f403
Improve vespa interface (#4546)
![Screenshot 2023-05-11 at 7 50 31
PM](https://github.com/hwchase17/langchain/assets/130488702/bc8ab4bb-8006-44fc-ba07-df54e84ee2c1)
1 year ago
vinoyang 72f18fd08b
Provide get current date function dialect for other DBs (#4576)
# Provide get current date function dialect for other DBs

<!--
Thank you for contributing to LangChain! Your PR will appear in our next
release under the title you set. Please make sure it highlights your
valuable contribution.

Replace this with a description of the change, the issue it fixes (if
applicable), and relevant context. List any dependencies required for
this change.

After you're done, someone will review your PR. They may suggest
improvements. If no one reviews your PR within a few days, feel free to
@-mention the same people again, as notifications can get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting

<!-- If you're adding a new integration, include an integration test and
an example notebook showing its use! -->

## Who can review?

Community members can review the PR once tests pass. Tag
maintainers/contributors who might be interested:

@eyurtsev

<!-- For a quicker response, figure out the right person to tag with @

        @hwchase17 - project lead

        Tracing / Callbacks
        - @agola11

        Async
        - @agola11

        DataLoaders
        - @eyurtsev

        Models
        - @hwchase17
        - @agola11

        Agents / Tools / Toolkits
        - @vowelparrot
        
        VectorStores / Retrievers / Memory
        - @dev2049
        
 -->
1 year ago
Neil Ruaro 3a2855945b
added documentation on retrieving a PG vectorstore (#4578)
This PR adds in documentation on querying an existing vectorstore in PG 

Fixes 3191 (issue)
1 year ago
Andrea Pinto 1e5d25b93c
Improve error messages formatting in doc loaders (#4586)
# Cosmetic in errors formatting

Added appropriate spacing to the `ImportError` message in a bunch of
document loaders to enhance trace readability (including Google Drive,
Youtube, Confluence and others). This change ensures that the error
messages are not displayed as a single line block, and that the `pip
install xyz` commands can be copied to clipboard from terminal easily.

## Who can review?

@eyurtsev
1 year ago
kYLe 570d057db4
Expose AnyScale LLM in langchain.llms (#4585)
# Expose AnyScale LLM in  langchain.llms

Fixes # update init.py so we can from langchain.llms import Anyscale
1 year ago
Eugene Yurtsev a5371a0fa2
Add pytest --only-extended and --only-core options (#4494)
# Adds testing options to pytest

This PR adds the following options: 

* `--only-core` will skip all extended tests, running all core tests.
* `--only-extended` will skip all core tests. Forcing alll extended
tests to be run.

Running `py.test` without specifying either option will remain
unaffected. Run
all tests that can be run within the unit_tests direction. Extended
tests will
run if required packages are installed.

## Before submitting

## Who can review?
1 year ago
Harrison Chase 5ad151ed44
Add constitutional principles from paper (#4554)
Add constitutional principles from https://arxiv.org/pdf/2212.08073.pdf

---------

Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
Sai Vinay G cf4c1394a2
feat: Added class to support huggingface text generation inference server (#4447)
[Text Generation
Inference](https://github.com/huggingface/text-generation-inference) is
a Rust, Python and gRPC server for generating text using LLMs.

This pull request add support for self hosted Text Generation Inference
servers.

feature: #4280

---------

Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
Zander Chase 258c319855
Dereference Messages (#4557)
Update how we parse the messages now that the server splits prompts /
messages up
1 year ago
Leonid Ganeline e17d0319d5
Add `arxiv` retriever (#4538) 1 year ago
vinoyang 25cd6e060a
Enhance the prompt to make the LLM generate right date for real today (#4505)
# Enhance the prompt to make the LLM generate right date for real today

Fixes # (issue)

Currently, if the user's question contains `today`, the clickhouse
always points to an old date. This may be related to the fact that the
GPT training data is relatively old.
1 year ago
vinoyang e942db3e78
Add prestodb prompt (#4516)
Add a PrestoDB prompt
1 year ago
SimFG 7bcf238a1a
Optimize the initialization method of GPTCache (#4522)
Optimize the initialization method of GPTCache, so that users can use GPTCache more quickly.
1 year ago
Zander Chase f4d3cf2dfb
Add Invocation Params (#4509)
### Add Invocation Params to Logged Run


Adds an llm type to each chat model as well as an override of the dict()
method to log the invocation parameters for each call

---------

Co-authored-by: Ankush Gola <ankush.gola@gmail.com>
1 year ago
Ankush Gola 59853fc876
add invocation params as extra params in llm callbacks (#4506)
# Your PR Title (What it does)

<!--
Thank you for contributing to LangChain! Your PR will appear in our next
release under the title you set. Please make sure it highlights your
valuable contribution.

Replace this with a description of the change, the issue it fixes (if
applicable), and relevant context. List any dependencies required for
this change.

After you're done, someone will review your PR. They may suggest
improvements. If no one reviews your PR within a few days, feel free to
@-mention the same people again, as notifications can get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting

<!-- If you're adding a new integration, include an integration test and
an example notebook showing its use! -->

## Who can review?

Community members can review the PR once tests pass. Tag
maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

        @hwchase17 - project lead

        Tracing / Callbacks
        - @agola11

        Async
        - @agola11

        DataLoader Abstractions
        - @eyurtsev

        LLM/Chat Wrappers
        - @hwchase17
        - @agola11

        Tools / Toolkits
        - @vowelparrot
 -->
1 year ago
Ofey Chan 1c0ec26e40
[pyproject.toml] add `tiktoken` when install `langchain[openai]` (#4514)
# Add `tiktoken` as dependency when installed as `langchain[openai]`

Fixes #4513 (issue)

## Who can review?

Community members can review the PR once tests pass. Tag
maintainers/contributors who might be interested:

@vowelparrot 

<!-- For a quicker response, figure out the right person to tag with @

        @hwchase17 - project lead

        Tracing / Callbacks
        - @agola11

        Async
        - @agola11

        DataLoaders
        - @eyurtsev

        Models
        - @hwchase17
        - @agola11

        Agents / Tools / Toolkits
        - @vowelparrot
        
        VectorStores / Retrievers / Memory
        - @dev2049
        
 -->
1 year ago
Zander Chase 4ee47926ca
Add on_chat_message_start (#4499)
### Add on_chat_message_start to callback manager and base tracer

Goal: trace messages directly to permit reloading as chat messages
(store in an integration-agnostic way)

Add an `on_chat_message_start` method. Fall back to `on_llm_start()` for
handlers that don't have it implemented.

Does so in a non-backwards-compat breaking way (for now)
1 year ago
Yu Le bbf76dbb52
fix typos in the prompts of LLMSummarizationCheckerChain (#4518) 1 year ago
Jonas Nelle 97e7dc1502
Make BaseStringMessagePromptTemplate.from_template return type generic (#4523)
# Make BaseStringMessagePromptTemplate.from_template return type generic

I use mypy to check type on my code that uses langchain. Currently after
I load a prompt and convert it to a system prompt I have to explicitly
cast it which is quite ugly (and not necessary):
```
prompt_template = load_prompt("prompt.yaml")
system_prompt_template = cast(
    SystemMessagePromptTemplate,
    SystemMessagePromptTemplate.from_template(prompt_template.template),
)
```

With this PR, the code would simply be: 
```
prompt_template = load_prompt("prompt.yaml")
system_prompt_template = SystemMessagePromptTemplate.from_template(prompt_template.template)
```

Given how much langchain uses inheritance, I think this type hinting
could be applied in a bunch more places, e.g. load_prompt also return a
`FewShotPromptTemplate` or a `PromptTemplate` but without typing the
type checkers aren't able to infer that. Let me know if you agree and I
can take a look at implementing that as well.

        @hwchase17 - project lead

        DataLoaders
        - @eyurtsev
1 year ago
kYLe 446b60d803
Fix a typo in langchain/docs/modules/models/llms/integrations/anyscale.ipynb (#4526) 1 year ago
Davis Chase 0f93de0a59
Release 0.0.166 (#4510) 1 year ago
Sunish Sheth 812e5f43f5
Add _type for all parsers (#4189)
Used for serialization. Also add test that recurses through
our subclasses to check they have them implemented

Would fix https://github.com/hwchase17/langchain/issues/3217
Blocking: https://github.com/mlflow/mlflow/pull/8297

---------

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>
Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
Akshaya Annavajhala b21d7c138c
Callback Handler for MLflow (#4150)
Rebased Mahmedk's PR with the callback refactor and added the example
requested by hwchase plus a couple minor fixes

---------

Co-authored-by: Ahmed K <77802633+mahmedk@users.noreply.github.com>
Co-authored-by: Ahmed K <mda3k27@gmail.com>
Co-authored-by: Davis Chase <130488702+dev2049@users.noreply.github.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
kYLe 0d51a1f12b
Add LLMs support for Anyscale Service (#4350)
Add Anyscale service integration under LLM

Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
Kristóf Dombi 99b2400048
[Docs]: Add Kinsta to the list of deployment providers (#4445)
We're fans of the LangChain framework thus we wanted to make sure we
provide an easy way for our customers to be able to utilize this
framework for their LLM-powered applications at our platform.
1 year ago
Evan Jones f668251948
parameterized distance metrics; lint; format; tests (#4375)
# Parameterize Redis vectorstore index

Redis vectorstore allows for three different distance metrics: `L2`
(flat L2), `COSINE`, and `IP` (inner product). Currently, the
`Redis._create_index` method hard codes the distance metric to COSINE.

I've parameterized this as an argument in the `Redis.from_texts` method
-- pretty simple.

Fixes #4368 

## Before submitting

I've added an integration test showing indexes can be instantiated with
all three values in the `REDIS_DISTANCE_METRICS` literal. An example
notebook seemed overkill here. Normal API documentation would be more
appropriate, but no standards are in place for that yet.

## Who can review?

Not sure who's responsible for the vectorstore module... Maybe @eyurtsev
/ @hwchase17 / @agola11 ?
1 year ago
Nick Omeyer f46710d408
Fix minor issues in self-query retriever prompt formatting (#4450)
# Fix minor issues in self-query retriever prompt formatting

I noticed a few minor issues with the self-query retriever's prompt
while using it, so here's PR to fix them 😇

## Who can review?

Community members can review the PR once tests pass. Tag
maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

        @hwchase17 - project lead

        Tracing / Callbacks
        - @agola11

        Async
        - @agola11

        DataLoader Abstractions
        - @eyurtsev

        LLM/Chat Wrappers
        - @hwchase17
        - @agola11

        Tools / Toolkits
        - @vowelparrot
 -->
1 year ago
Zander Chase d969f43ed8
Load HuggingFace Tool (#4475)
# Add option to `load_huggingface_tool`

Expose a method to load a huggingface Tool from the HF hub

---------

Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
1 year ago
Davis Chase cd01de49cf
Update contribution guidelines (#4431)
provide more guidance on pr's
1 year ago
Eugene Yurtsev 146616aa5d
Test workflow, fix minor typos (#4495)
# Fix 2 minor typos in test workflow.

This PR does not result in any functional changes.
1 year ago
Eugene Yurtsev f373883c1a
Refactor test workflow (#4457)
# Refactor the test workflow

This PR refactors the tests to run using a single test workflow. This
makes it easier to relaunch failing tests and see in the UI which test
failed since the jobs are grouped together.

## Before submitting

## Who can review?
1 year ago
Davis Chase b77e103ca6
Add aleph alpha api key attribute (#4489)
@tugot17 applied your change to master
1 year ago
Harrison Chase 3ce29cb4a6
Harrison/new search (#4359)
Co-authored-by: Jiaping(JP) Zhang <vincentzhangv@gmail.com>
1 year ago
Jakob Heyder 545ae8b756
Fix: Add run_manager on all AgentFinish returns in AgentExecutor (#4466) 1 year ago
Ankush Gola ae8d6d5a89
Add docs for tracing environment variable (#4477) 1 year ago
Davis Chase 9ec60ad832
Add azure cognitive search retriever (#4467)
All credit to @UmerHA, made a couple small changes

---------

Co-authored-by: UmerHA <40663591+UmerHA@users.noreply.github.com>
1 year ago
Davis Chase 46b100ea63
Add DocArray vector stores (#4483)
Thanks to @anna-charlotte and @jupyterjazz for the contribution! Made
few small changes to get it across the finish line

---------

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: jupyterjazz <saba.sturua@jina.ai>
Co-authored-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Co-authored-by: jupyterjazz <saba.sturua@jina.ai>
Co-authored-by: Saba Sturua <45267439+jupyterjazz@users.noreply.github.com>
1 year ago
Davis Chase f2a536b445
release 165 (#4486)
bump version
1 year ago
Harrison Chase b2f920e891
add tracing v2 env var (#4465)
Co-authored-by: Ankush Gola <ankush.gola@gmail.com>
1 year ago
Zander Chase 9231143f91
Fix Duplicate trust_remote_code in pipeline (#4369)
### Fix issue with duplicate specification of `trust_remote_code` in
HuggingFacePipeline

Fixes # 4351
1 year ago
Davis Chase 6fbdb9ce51
Release 0.0.164 (#4454) 1 year ago
Davis Chase 04475bea7d
Mv plan and execute to experimental (#4459) 1 year ago
netseye 1ad180f6de
Add request timeout to openai embedding (#4144)
Add request_timeout field to openai embedding. Defaults to None

---------

Co-authored-by: Jeakin <Jeakin@botu.cc>
1 year ago
zvrr 274dc4bc53
add clickhouse prompt (#4456)
# Add clickhouse prompt

Add clickhouse database sql prompt
1 year ago