Commit Graph

3789 Commits

Author SHA1 Message Date
Nuno Campos
3bdc273ab3
Implement .transform() in RunnablePassthrough() (#9032)
- This ensures passthrough doesnt break streaming
---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-10 10:41:19 -07:00
Bagatur
206f809366
fix sched ci (more) (#9056) 2023-08-10 10:39:29 -07:00
Aashish Saini
8a320e55a0
Corrected grammatical errors and spelling mistakes in the index.mdx file. (#9026)
Expressing gratitude to the creator for crafting this remarkable
application. 🙌, Would like to Enhance grammar and spelling in the
documentation for a polished reader experience.

Your feedback is valuable as always 

@baskaryan , @hwchase17 , @eyurtsev
2023-08-10 10:17:09 -07:00
Bagatur
e5db8a16c0
Bagatur/fix sched (#9054) 2023-08-10 09:34:44 -07:00
Bagatur
e162fd418a
fix sched ci (#9053) 2023-08-10 09:29:46 -07:00
Ismail Pelaseyed
abb1264edf
Fix issue with Metaphor Search Tool throwing error on missing keys in API response (#9051)
- Description: Fixes an issue with Metaphor Search Tool throwing when
missing keys in API response.
  - Issue: #9048 
  - Tag maintainer: @hinthornw @hwchase17 
  - Twitter handle: @pelaseyed
2023-08-10 09:07:00 -07:00
Eugene Yurtsev
5e05ba2140
Add embeddings cache (#8976)
This PR adds the ability to temporarily cache or persistently store
embeddings. 

A notebook has been included showing how to set up the cache and how to
use it with a vectorstore.
2023-08-10 11:15:30 -04:00
Bagatur
6e14f9548b
bump 261 (#9041) 2023-08-10 07:59:27 -07:00
Lance Martin
2380492c8e
API use case (#8546)
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-10 07:52:54 -07:00
Eugene Yurtsev
d21333d710
Add redis storage (#8980)
Add a redis implementation of a BaseStore
2023-08-10 10:48:35 -04:00
Luca Foppiano
dfb93dd2b5
Improved grobid documentation (#9025)
- Description: Improvement in the Grobid loader documentation, typos and
suggesting to use the docker image instead of installing Grobid in local
(the documentation was also limited to Mac, while docker allow running
in any platform)
  - Tag maintainer: @rlancemartin, @eyurtsev
  - Twitter handle: @whitenoise
2023-08-10 10:47:22 -04:00
Hiroshige Umino
2c7297d243
Fix a broken code block display (#9034)
- Description: Fix a broken code block in this page:
https://python.langchain.com/docs/modules/model_io/prompts/prompt_templates/
- Issue: N/A
- Dependencies: None
- Tag maintainer: @baskaryan
- Twitter handle: yaotti
2023-08-10 10:39:01 -04:00
Bagatur
434a96415b
make runnable dir (#9016)
Co-authored-by: Nuno Campos <nuno@boringbits.io>
2023-08-10 08:56:37 +01:00
Nuno Campos
c7a489ae0d
Small improvements for tracer and debug output of runnables (#8683)
<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the
same people again.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
2023-08-10 07:24:12 +01:00
EricFan
618cf5241e
Open file in UTF-8 encoding (#6919) (#8943)
FileCallbackHandler cannot handle some language, for example: Chinese. 
Open file using UTF-8 encoding can fix it.
@agola11
  
**Issue**: #6919 
**Dependencies**: NO dependencies,

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-09 17:54:21 -07:00
colegottdank
f4a47ec717
Add optional model kwargs to ChatAnthropic to allow overrides (#9013)
---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-09 17:34:00 -07:00
Piyush Jain
3b51817706
Updating port and ssl use in sample notebook (#8995)
## Description
This PR updates the sample notebook to use the default port (8182) and
the ssl for the Neptune database connection.
2023-08-09 17:08:48 -07:00
Kaizen
bbbd2b076f
DirectoryLoader slicing (#8994)
DirectoryLoader can now return a random sample of files in a directory.
Parameters added are:
sample_size
randomize_sample
sample_seed


@rlancemartin, @eyurtsev

---------

Co-authored-by: Andrew Oseen <amovfx@protonmail.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-09 16:05:16 -07:00
IanRogers-101Ways
d248481f13
skip over empty google spreadsheets (#8974)
- Description: Allow GoogleDriveLoader to handle empty spreadsheets  
- Issue: Currently GoogleDriveLoader will crash if it tries to load a
spreadsheet with an empty sheet
  - Dependencies: n/a
  - Tag maintainer: @rlancemartin, @eyurtsev
2023-08-09 16:05:02 -07:00
Eugene Yurtsev
efa02ed768
Suppress divide by zero wranings for cosine similarity (#9006)
Suppress run time warnings for divide by zero as the downstream code
handles the scenario (handling inf and nan)
2023-08-09 15:56:51 -07:00
Leonid Ganeline
5454591b0a
docstrings cleanup (#8993)
Added/Updated docstrings

 @baskaryan
2023-08-09 15:49:06 -07:00
Massimiliano Pronesti
c72da53c10
Add logprobs to SamplingParameters in vllm (#9010)
This PR aims at amending #8806 , that I opened a few days ago, adding
the extra `logprobs` parameter that I accidentally forgot
2023-08-09 15:48:29 -07:00
Bagatur
8dd071ad08
import airbyte loaders (#9009) 2023-08-09 14:51:15 -07:00
Bagatur
96d064e305
bump 260 (#9002) 2023-08-09 13:40:49 -07:00
Michael Shen
c2f46b2cdb
Fixed wrong paper reference (#8970)
The ReAct reference references to MRKL paper. Corrected so that it
points to the actual ReAct paper #8964.
2023-08-09 16:17:46 -04:00
Nuno Campos
808248049d
Implement a router for openai functions (#8589) 2023-08-09 21:17:04 +01:00
Eugene Yurtsev
a6e6e9bb86
Fix airbyte loader (#8998)
Fix airbyte loader

https://github.com/langchain-ai/langchain/issues/8996
2023-08-09 16:13:06 -04:00
William FH
90579021f8
Update Key Check (#8948)
In eval loop. It needn't be done unless you are creating the
corresponding evaluators
2023-08-09 12:33:00 -07:00
Jerzy Czopek
539672a7fd
Feature/fix azureopenai model mappings (#8621)
This pull request aims to ensure that the `OpenAICallbackHandler` can
properly calculate the total cost for Azure OpenAI chat models. The
following changes have resolved this issue:

- The `model_name` has been added to the ChatResult llm_output. Without
this, the default values of `gpt-35-turbo` were applied. This was
causing the total cost for Azure OpenAI's GPT-4 to be significantly
inaccurate.
- A new parameter `model_version` has been added to `AzureChatOpenAI`.
Azure does not include the model version in the response. With the
addition of `model_name`, this is not a significant issue for GPT-4
models, but it's an issue for GPT-3.5-Turbo. Version 0301 (default) of
GPT-3.5-Turbo on Azure has a flat rate of 0.002 per 1k tokens for both
prompt and completion. However, version 0613 introduced a split in
pricing for prompt and completion tokens.
- The `OpenAICallbackHandler` implementation has been updated with the
proper model names, versions, and cost per 1k tokens.

Unit tests have been added to ensure the functionality works as
expected; the Azure ChatOpenAI notebook has been updated with examples.

Maintainers: @hwchase17, @baskaryan

Twitter handle: @jjczopek

---------

Co-authored-by: Jerzy Czopek <jerzy.czopek@avanade.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-09 10:56:15 -07:00
Bagatur
269f85b7b7
scheduled gha fix (#8977) 2023-08-09 09:44:25 -07:00
shibuiwilliam
3adb1e12ca
make trajectory eval chain stricter and add unit tests (#8909)
- update trajectory eval logic to be stricter
- add tests to trajectory eval chain
2023-08-09 10:57:18 -04:00
Nuno Campos
b8df15cd64
Adds transform support for runnables (#8762)
<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the
same people again.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-09 12:34:23 +01:00
Harrison Chase
4d72288487
async output parser (#8894)
Co-authored-by: Nuno Campos <nuno@boringbits.io>
2023-08-09 08:25:38 +01:00
Bagatur
3c6eccd701
bump 259 (#8951) 2023-08-09 00:07:47 -07:00
Harrison Chase
7de6a1b78e
parent document retriever (#8941) 2023-08-08 22:39:08 -07:00
arjunbansal
a2681f950d
add instructions on integrating Log10 (#8938)
- Description: Instruction for integration with Log10: an [open
source](https://github.com/log10-io/log10) proxiless LLM data management
and application development platform that lets you log, debug and tag
your Langchain calls
  - Tag maintainer: @baskaryan
  - Twitter handle: @log10io @coffeephoenix

Several examples showing the integration included
[here](https://github.com/log10-io/log10/tree/main/examples/logging) and
in the PR
2023-08-08 19:15:31 -07:00
Aarav Borthakur
3f64b8a761
Integrate Rockset as a chat history store (#8940)
Description: Adds Rockset as a chat history store
Dependencies: no changes
Tag maintainer: @hwchase17

This PR passes linting and testing. 

I added a test for the integration and an example notebook showing its
use.
2023-08-08 18:54:07 -07:00
Bagatur
0a1be1d501
document lcel fallbacks (#8942) 2023-08-08 18:49:33 -07:00
William FH
e3056340da
Add id in error in tracer (#8944) 2023-08-08 18:25:27 -07:00
Molly Cantillon
99b5a7226c
Weaviate: adding auth example + fixing spelling in ReadME (#8939)
Added basic auth example to Weaviate notebook @baskaryan
2023-08-08 16:24:17 -07:00
Bagatur
95cf7de112
scheduled tests GHA (#8879)
Adding scheduled daily GHA that runs marked integration tests. To start
just marking some tests in test_openai
2023-08-08 14:55:25 -07:00
Joe Reuter
8f0cd91d57
Airbyte based loaders (#8586)
This PR adds 8 new loaders:
* `AirbyteCDKLoader` This reader can wrap and run all python-based
Airbyte source connectors.
* Separate loaders for the most commonly used APIs:
  * `AirbyteGongLoader`
  * `AirbyteHubspotLoader`
  * `AirbyteSalesforceLoader`
  * `AirbyteShopifyLoader`
  * `AirbyteStripeLoader`
  * `AirbyteTypeformLoader`
  * `AirbyteZendeskSupportLoader`

## Documentation and getting started
I added the basic shape of the config to the notebooks. This increases
the maintenance effort a bit, but I think it's worth it to make sure
people can get started quickly with these important connectors. This is
also why I linked the spec and the documentation page in the readme as
these two contain all the information to configure a source correctly
(e.g. it won't suggest using oauth if that's avoidable even if the
connector supports it).

## Document generation
The "documents" produced by these loaders won't have a text part
(instead, all the record fields are put into the metadata). If a text is
required by the use case, the caller needs to do custom transformation
suitable for their use case.

## Incremental sync
All loaders support incremental syncs if the underlying streams support
it. By storing the `last_state` from the reader instance away and
passing it in when loading, it will only load updated records.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-08 14:49:25 -07:00
Eugene Yurtsev
15f650ae8c
Add base storage interface, 2 implementations and utility encoder (#8895)
This PR defines an abstract interface for key value stores.

It provides 2 implementations: 
1. Local File System
2. In memory -- used to facilitate testing

It also provides an encoder utility to help take care of serialization
from arbitrary data to data that can be stored by the given store
2023-08-08 17:29:06 -04:00
Harrison Chase
7543a3d70e
Harrison/image (#845)
Co-authored-by: Ashutosh Sanzgiri <sanzgiri@gmail.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-08 13:58:27 -07:00
Bagatur
ab193338aa
bump 258 (#8932) 2023-08-08 12:54:51 -07:00
Eugene Yurtsev
bb12184551
Internal code deprecation API (#8763)
Proposal for an internal API to deprecate LangChain code.

This PR is heavily based on:
https://github.com/matplotlib/matplotlib/blob/main/lib/matplotlib/_api/deprecation.py

This PR only includes deprecation functionality (no renaming etc.). 
Additional functionality can be added on a need basis (e.g., renaming
parameters), but best to roll out as an MVP to test this
out.

DeprecationWarnings are ignored by default. We can change the policy for
the deprecation warnings, but we'll need to make sure we're not creating
noise for users due to internal code invoking deprecated functionality.
2023-08-08 15:42:22 -04:00
Leonid Ganeline
33a2f58fbf
tensoflow_datasets document loader (#8721)
This PR adds `tensoflow_datasets` document loader
2023-08-08 15:19:28 -04:00
Holt Skinner
fad26e79a3
fix: Resolve AttributeError in Google Cloud Enterprise Search retriever (#8872)
- Reverting some of the changes made in
https://github.com/langchain-ai/langchain/pull/8369
2023-08-08 12:11:12 -07:00
William FH
b2eb4ff0fc
Relax Validation in Eval (#8902)
Just check for missing keys
2023-08-08 11:59:30 -07:00
Leonid Ganeline
2d078c7767
PubMed document loader (#8893)
- added `PubMed Document Loader` artifacts; ut-s; examples 
- fixed `PubMed utility`; ut-s

@hwchase17
2023-08-08 14:26:03 -04:00