Commit Graph

1810 Commits (0fc3af89327b89e2d5c164525d914487dec4738b)

Author SHA1 Message Date
Govind.S.B 9024593468
added system prompt and template fields to ollama (#13022)
**Description**
the ollama api now supports passing system prompt and template directly
instead of modifying the model file , but the ollama integration in
langchain did not have this change updated . The update just adds these
two parameters to it ( there are 2 more parameters that are pending to
be updated, I was not sure about their utility wrt to langchain )
Refer :
8713ac23a8

**Issue** : None Applicable

**Dependencies** : None Changed

**Twitter handle** : https://twitter.com/violetto96

---------

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
11 months ago
langchain-infra f55f67055f
Add dockerfile template (#13240) 11 months ago
Guillem Orellana Trullols 0f31cd8b49
Remove `_get_kwarg_value` function (#13184)
`_get_kwarg_value` function is useless, one can rely on python builtin
functionalities to do the exact same thing.

- **Description:** Removed `_get_kwarg_value`. Helps with code
readability.
  - **Issue:** the issue # it fixes (if applicable),
  - **Twitter handle:** @Guillem_96
11 months ago
SuperDa Fu e1c020dfe1
dalle add model parameter (#13201)
- **Description:** dalle_image_generator adding a new model parameter,
  - **Issue:** N/A,
  - **Dependencies:** 
  - **Tag maintainer: @hwchase17
  - **Twitter handle:**

---------

Co-authored-by: dafu <xiangbingze@wenru.wang>
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
Co-authored-by: Erick Friis <erickfriis@gmail.com>
11 months ago
Dennis de Greef 64e11592bb
Improve CSV reader which can't call .strip() on NoneType (#13079)
Improve CSV reader which can't call .strip() on NoneType if there are
less cells in the row compared to the header

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** 
I have a CSV file as followed

```
headerA,headerB,headerC
v1A,v1B,v1C,
v2A,v2B
v3A,v3B,v3C
```
In this case, row 2 is missing a value, which results in reading a None
type. The strip() method can not be called on None, hence raising. In
this PR I am making the change to only call strip if the value if not
None.

  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
11 months ago
glad4enkonm 339973db47
Update ollama.py (#12895)
duplicate option removed
**Description:**  An issue fix, http stop option duplicate removed.
**Issue:** the issue #12892 fix
**Dependencies:** no
**Tag maintainer:** @eyurtsev

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
11 months ago
Isak Nyberg 8f81703d76
Add new models to openai callback (#13244)
**Description:** Adding the new models to the openai callback function,
info taken from [model
announcement](https://platform.openai.com/docs/models) and
[pricing](https://openai.com/pricing)

A short description for a short PR :)
11 months ago
Bagatur ea6dd3a550
bump 335 (#13261) 11 months ago
William FH a837b03e55
Update langsmith version 0.63 (#13208) 11 months ago
Harrison Chase 7f1d26160d
update tools (#13243) 11 months ago
Nuno Campos 8d6faf5665
Make it easier to subclass runnable binding with custom init args (#13189) 11 months ago
Peter Vandenabeele 7f1964b264
Fix BeautifulSoupTransformer: no more duplicates and correct order of tags + tests (#12596) 11 months ago
Erick Friis 9c7afa8adb
Upgrade cohere embedding model to v3 (#13219)
Just updates API docs, doesn't change default param from 2.0 (could be
breaking change)
11 months ago
Erick Friis 8fdf15c023
Fix Document Loader Unit Test - Docusaurus (#13228) 11 months ago
Lee 72ad448daa
feat: Docusaurus Loader (#9138)
Added a Docusaurus Loader

Issue: #6353

I had to implement this for working with the Ionic documentation, and
wanted to open this up as a draft to get some guidance on building this
out further. I wasn't sure if having it be a light extension of the
SitemapLoader was in the spirit of a proper feature for the library --
but I'm grateful for the opportunities Langchain has given me and I'd
love to build this out properly for the sake of the community.

Any feedback welcome!
11 months ago
Tomaz Bratanic 0dc4ab0be1
Neo4j chat message history (#13008) 11 months ago
fyasla d266b3ea4a
issue #12165 mask API key in chat_models/azureml_endpoint module (#12836)
- **Description:** `AzureMLChatOnlineEndpoint` object from
langchain/chat_models/azureml_endpoint.py safe to print
without having any secrets included in raw format in the string
representation.
  - **Issue:** #12165,
  - **Tag maintainer:** @eyurtsev

---------

Co-authored-by: Faysal Bougamale <faysal.bougamale@horiba.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
11 months ago
Anush 52f34de9b7
feat: FastEmbed embedding provider (#13109)
## Description:
This PR intends to add
[Qdrant/FastEmbed](https://qdrant.github.io/fastembed/) as a local
embeddings provider, associated tests and documentation.

**Documentation preview:**
https://langchain-git-fork-anush008-master-langchain.vercel.app/docs/integrations/text_embedding/fastembed

---------

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
11 months ago
Eugene Yurtsev b0e8cbe0b3
Add RunnableSequence documentation (#13094)
Add RunnableSequence documentation
11 months ago
Eugene Yurtsev 869df62736
Document RunnableWithFallbacks (#13088)
Add documentation to RunnableWithFallbacks
11 months ago
Eugene Yurtsev 8313c218da
Add more runnable documentation (#13083)
- Adding documentation to the runnable.
- Documentation is not organized in the best way for the runnable; i.e.,
in
terms of LCEL vs. other standard methods, will follow up with more
edits.
11 months ago
Bagatur 24386e0860
bump 334, exp 40 (#13211) 11 months ago
Lance Martin d2e50b3108
Add Chroma multimodal cookbook (#12952)
Pending:
* https://github.com/chroma-core/chroma/pull/1294
* https://github.com/chroma-core/chroma/pull/1293

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
Co-authored-by: Bagatur <baskaryan@gmail.com>
11 months ago
The1Bill 55912868da
Update toolkit.py to remove single quotes around table names (#12445)
**Description:** Removing the single quote wrapper around the table
names in the SQL agent toolkit.py file as it misleads the LLM into
querying against tables with single quotes around their names.
**Issue:** #7457 
**Dependencies:** None
**Tag maintainer:** @hwchase17 
**Twitter handle:** None
11 months ago
Nuno Campos 362a446999
Changes to root listener (#12174)
- Implement config_specs to include session_id
- Remove Runnable method and update notebook
- Add more details to notebook, eg. show input schema and config schema
before and after adding message history

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
11 months ago
Nuno Campos b2b94424db
Update return type for Runnable.__or__ (#12880)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
11 months ago
Harrison Chase 0a2b1c7471
improve duck duck go tool (#13165) 11 months ago
Shinya Maeda 28cc60b347
Fix langchain.llms OpenAI completion doesn't work due to v1 client update (#13099)
This commit fixes the issue that langchain.llms OpenAI completion
stopped working since the V1 openai client update.

Replace this entire comment with:
- **Description:** This PR fixes the issue [AttributeError: module
'openai' has no attribute
'Completion'](https://github.com/langchain-ai/langchain/issues/12967)
similar to
8e0cb2eb84
and https://github.com/langchain-ai/langchain/pull/12969,
  - **Issue:** https://github.com/langchain-ai/langchain/issues/12967,
  - **Dependencies:** `openai` v1.x.x client,
  - **Tag maintainer:** @baskaryan,
  - **Twitter handle:** @dosuken123 

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
Co-authored-by: Bagatur <baskaryan@gmail.com>
11 months ago
Bagatur ff43cd6701
OpenAI remove httpx typing (#13154)
Addresses #13124
11 months ago
Bagatur 8b2a82b5ce
Bagatur/docs smith context (#13139) 11 months ago
Bagatur f04cc4b7e1
bump 333 (#13131) 11 months ago
billytrend-cohere b346d4a455
Add message to documents (#12552)
This adds the response message as a document to the rag retriever so
users can choose to use this. Also drops document limit.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
11 months ago
Harrison Chase 5f38770161
Support oai tool call (#13110)
Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Nuno Campos <nuno@boringbits.io>
11 months ago
Holt Skinner 0fc8fd12bd
feat: Vertex AI Search - Add Snippet Retrieval for Non-Advanced Website Data Stores (#13020)
https://cloud.google.com/generative-ai-app-builder/docs/snippets#snippets

---------

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
11 months ago
Jacob Lee 76283e9625
Adds embeddings filter option to return scores in state (#12489)
CC @baskaryan @assafelovic
11 months ago
jakerachleff 18601bd4c8
Get project from langchain sdk (#13100)
## Description
We need to centralize the API we use to get the project name for our
tracers. This PR makes it so we always get this from a shared function
in the langsmith sdk.

## Dependencies
Upgraded langsmith from 0.52 to 0.62 to include the new API
`get_tracer_project`
11 months ago
Bagatur 72e12f6bcf
update more azure docs (#13093) 11 months ago
Bagatur 1703f132c6
update azure embedding docs (#13091) 11 months ago
Bagatur 9fdfac22c2
bump 332 (#13089) 11 months ago
Bagatur 1f85ec34d5
bump 331rc3 exp 39 (#13086) 11 months ago
Anton Troynikov 9f077270c8
Don't pass EF to chroma (#13085)
- **Description:** 

Recently Chroma rolled out a breaking change on the way we handle
embedding functions, in order to support multi-modal collections.

This broke the way LangChain's `Chroma` objects get created, because we
were passing the EF down into the Chroma collection:
https://docs.trychroma.com/migration#migration-to-0416---november-7-2023

However, internally, we are never actually using embeddings on the
chroma collection - LangChain's `Chroma` object calls it instead. Thus
we just don't pass an `embedding_function` to Chroma itself, which fixes
the issue.
11 months ago
Erick Friis f15f8e01cf
Azure OpenAI Embeddings (#13039)
Co-authored-by: Bagatur <baskaryan@gmail.com>
11 months ago
David Peterson 37561d8986
Add Proper Import Error (#13042)
- **Description:** The issue was not listing the proper import error for
amazon textract loader.
- **Issue:** Time wasted trying to figure out what to install...
(langchain docs don't list the dependency either)
  - **Dependencies:** N/A
  - **Tag maintainer:** @sbusso 
  - **Twitter handle:** @h9ste

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
11 months ago
Eugene Yurtsev 06c503f672
Add RunnableRetry Documentation (#13074) 11 months ago
Bagatur 55aeff6777
oai assistant multiple actions (#13068) 11 months ago
Erick Friis a9b70baef9
cli updates, 0.0.16 (#13034)
- confirm flags, serve detection
- 0.0.16
- always gen code
- pip bool
11 months ago
Erick Friis 506f81563f
Update Deps in Experimental (#13029) 11 months ago
Stefano Lottini 4f4b020582
Add "Astra DB" vector store integration (#12966)
# Astra DB Vector store integration

- **Description:** This PR adds a `VectorStore` implementation for
DataStax Astra DB using its HTTP API
  - **Issue:** (no related issue)
- **Dependencies:** A new required dependency is `astrapy` (`>=0.5.3`)
which was added to pyptoject.toml, optional, as per guidelines
- **Tag maintainer:** I recently mentioned to @baskaryan this
integration was coming
  - **Twitter handle:** `@rsprrs` if you want to mention me

This PR introduces the `AstraDB` vector store class, extensive
integration test coverage, a reworking of the documentation which
conflates Cassandra and Astra DB on a single "provider" page and a new,
completely reworked vector-store example notebook (common to the
Cassandra store, since parts of the flow is shared by the two APIs). I
also took care in ensuring docs (and redirects therein) are behaving
correctly.

All style, linting, typechecks and tests pass as far as the `AstraDB`
integration is concerned.

I could build the documentation and check it all right (but ran into
trouble with the `api_docs_build` makefile target which I could not
verify: `Error: Unable to import module
'plan_and_execute.agent_executor' with error: No module named
'langchain_experimental'` was the first of many similar errors)

Thank you for a review!
Stefano

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
11 months ago
Yang, Bo 600caff03c
Add `Memorize` tool (#11722)
- **Description:** Add `Memorize` tool
  - **Tag maintainer:** @hwchase17

This PR added a new tool `Memorize` so that an agent can use it to
fine-tune itself. This tool requires `TrainableLLM` introduced in #11721

DEMO:
6a9003d5db

![image](https://github.com/langchain-ai/langchain/assets/601530/d6f0cb45-54df-4dcf-b143-f8aefb1e76e3)
11 months ago
Bagatur cf481c9418
bump exp 38 (#13016) 11 months ago
Bagatur 57e19989f6
Bagatur/oai assistant (#13010) 11 months ago
Erick Friis 74134dd7e1
cli pyproject updating (#12945)
`langchain app add` and `langchain app remove` will now keep the
dependencies list updated.

---------

Co-authored-by: Nuno Campos <nuno@boringbits.io>
11 months ago
Bagatur 6175dc30aa
bump 331rc2 (#13006) 11 months ago
Erick Friis 0c81cd923e
oai v1 embeddings (#12969)
Initial PR to get OpenAIEmbeddings working with the new sdk

fyi @rlancemartin 

Fixes #12943

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
11 months ago
Bagatur fdbb45d79e
bump 331rc1 (#12965) 11 months ago
Bagatur 3bb8030a6e
fix max_tokens (#12964) 11 months ago
Bagatur a9002a82b8
bump 331rc0 (#12963) 11 months ago
Harrison Chase c27400efeb
Support multimodal messages (#11320)
Co-authored-by: Bagatur <baskaryan@gmail.com>
11 months ago
Bagatur 4f7dff9d66
Record system fingerprint chat openai (#12960) 11 months ago
Bagatur 8e0cb2eb84
ChatOpenAI and AzureChatOpenAI openai>=1 compatible (#12948) 11 months ago
Kacper Łukawski 52d0055a91
Add support of Cohere Embed v3 (#12940)
Cohere released the new embedding API (Embed v3:
https://txt.cohere.com/introducing-embed-v3/) that treats document and
query embeddings differently. This PR updated the `CohereEmbeddings` to
use them appropriately. It also works with the old models.
11 months ago
Praveen Venkateswaran 8e0dcb37d2
Add SecretStr for Symbl.ai Nebula API (#12896)
Description: This PR masks API key secrets for the Nebula model from
Symbl.ai
Issue: #12165 
Maintainer: @eyurtsev

---------

Co-authored-by: Praveen Venkateswaran <praveen.venkateswaran@ibm.com>
11 months ago
Vinzenz Klass 59d0bd2150
feat: acquire advisory lock before creating extension in pgvector (#12935)
- **Description:** Acquire advisory lock before attempting to create
extension on postgres server, preventing errors in concurrent
executions.
  - **Issue:** #12933
  - **Dependencies:** None

---------

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
11 months ago
Eugene Yurtsev b376854b26
Fix for anyscale chat model api key (#12938)
* ChatAnyscale was missing coercion to SecretStr for anyscale api key
* The model inherits from ChatOpenAI so it should not force the openai
api key to be secret str until openai model has the same changes

https://github.com/langchain-ai/langchain/issues/12841
11 months ago
hmasdev 622bf12c2e
fix regex pattern of structured output parser (#12929)
- **Description:** fix the regex pattern of
[StructuredChatOutputParser](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/agents/structured_chat/output_parser.py#L18)
and add unit tests for the code change.
- **Issue:** #12158 #12922
- **Dependencies:** None
- **Tag maintainer:** 
- **Twitter handle:** @hmdev3
- **NOTE:** This PR conflicts #7495 . After #7495 is merged, I am going
to update PR.
11 months ago
wemysschen 8d7144e6a6
fix baiducloud directory loader import file loader (#12924)
**Issue:** 
fix baiducloud BOS directory loader imports its file loader

---------

Co-authored-by: wemysschen <root@icoding-cwx.bcc-szzj.baidu.com>
11 months ago
Kacper Łukawski 621419f71e
Fix normalizing the cosine distance in Qdrant (#12934)
Qdrant was incorrectly calculating the cosine similarity and returning
`0.0` for the best match, instead of `1.0`. Internally Qdrant returns a
cosine score from `-1.0` (worst match) to `1.0` (best match), and the
current formula reflects it.
11 months ago
Hech 8fe6bcc662
Fix return metadata when searching for DingoDB (#12937) 11 months ago
Jakub Novák ada3d2cbd1
Add possibility to pass on_artifacts for a specific conversation (#12687)
Possibility to pass on_artifacts to a conversation. It can be then
achieved by adding this way:

```python
result = agent.run(
    input=message.text,
    metadata={
        "on_artifact": CALLBACK_FUNCTION
    },
)
```
11 months ago
Bagatur 53f453f01a
bump 331 (#12932) 11 months ago
Erick Friis 5000c7308e
cli template gitignores (#12914)
- ap gitignore
- package
11 months ago
Harrison Chase aba407f774
use keys not items (#12918) 11 months ago
wemysschen e14aa37d59
fix bes vector store search (#12828)
**Issue:** 
fix search body in baidu cloud vectorsearch

---------

Co-authored-by: wemysschen <root@icoding-cwx.bcc-szzj.baidu.com>
11 months ago
Lance Martin ea1ab391d4
Open Clip multimodal embeddings (#12754) 11 months ago
Bagatur ebee616822
bump 330 (#12853) 11 months ago
Erick Friis 6c237716c4
Update readmes with new cli install (#12847)
Old command still works. Just simplifying.

Merge after releasing CLI 0.0.15
11 months ago
Erick Friis 7db49d3842
Confirm sys.path includes current dir for app serve (#12851)
- Make sure sys.path is set properly for langchain app serve
- bump
11 months ago
Erick Friis 1bc35f61cb
CLI 0.0.14, Uvicorn update and no more [serve] (#12845)
Calls uvicorn directly from cli:
Reload works if you define app by import string instead of object.
(was doing subprocess in order to get reloading)

Version bump to 0.0.14

Remove the need for [serve] for simplicity.

Readmes are updated in #12847 to avoid cluttering this PR
11 months ago
William FH 18005c6384
Disable trace_on_chain_group auto-tracing (#12807)
Previously we treated trace_on_chain_group as a command to always start
tracing. This is unintuitive (makes the function do 2 things), and makes
it harder to toggle tracing
11 months ago
Erick Friis 0da75b9ebd
Autopopulate module name in cli init (#12814) 11 months ago
William FH 98aff29fbd
Add Dataset Page to printout (#12816) 11 months ago
Manuel Rech 2e2b9c76d9
Keep also original query - multi_query.py (#12696)
When you use a MultiQuery it might be useful to use the original query
as well as the newly generated ones to maximise the changes to retriever
the correct document. I haven't created an issue, it seems a very small
and easy thing.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
11 months ago
Bagatur 658a3a8607
FEAT: Merge TileDB vecstore (#12811) 11 months ago
Akio Nishimura c04647bb4e
Correct number of elements in config list in `batch()` and `abatch()` of `BaseLLM` (#12713)
- **Description:** Correct number of elements in config list in
`batch()` and `abatch()` of `BaseLLM` in case `max_concurrency` is not
None.
- **Issue:** #12643
- **Twitter handle:** @akionux

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
11 months ago
James Braza 88b506b321
Adds missing `urllib.parse` for IDE warning of `PubMedAPIWrapper` (#12808)
Resolves an IDE (PyCharm 2023.2.3 PE) warning around
`urllib.parse.quote`, also enabling CTRL-click
11 months ago
Bagatur a2bb0dd445 TileDB update import unit tests 11 months ago
Nikos Papailiou 2fdaa1e5fd
Add TileDB vectorstore implementation (#12624)
- **Description:** Add [TileDB](https://tiledb.com) vectorstore
implementation. TileDB offers ANN search capabilities using the
[TileDB-Vector-Search](https://github.com/TileDB-Inc/TileDB-Vector-Search)
module. It provides serverless execution of ANN queries and storage of
vector indexes both on local disk and cloud object stores (i.e. AWS S3).
More details in:
- [Why TileDB as a Vector
Database](https://tiledb.com/blog/why-tiledb-as-a-vector-database)
- [TileDB 101: Vector
Search](https://tiledb.com/blog/tiledb-101-vector-search)
- **Twitter handle:** @tiledb
11 months ago
盐粒 Yanli 1b233798a0
feat: Supprt pgvecto.rs as a VectorStore (#12718)
Supprt [pgvecto.rs](https://github.com/tensorchord/pgvecto.rs) as a new
VectorStore type.

This introduces a new dependency
[pgvecto_rs](https://pypi.org/project/pgvecto_rs/) and upgrade
SQLAlchemy to ^2.

Relate to https://github.com/tensorchord/pgvecto.rs/issues/11

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
11 months ago
Daniel Chalef 0cbdba6a9b
zep: VectorStore: Use Native MMR (#12690)
- refactor to use Zep's native MMR; update example
- 
@baskaryan @eyurtsev
11 months ago
Daniel Chalef cc3d3920e3
Zep: Summary Search and Example (#12686)
Zep now has the ability to search over chat history summaries. This PR
adds support for doing so. More here: https://blog.getzep.com/zep-v0-17/

@baskaryan @eyurtsev
11 months ago
Bagatur 526313002c
add import tests to all modules (#12806) 11 months ago
Harrison Chase 6609a6033f
fix vectorstore imports (#12804)
Co-authored-by: Bagatur <baskaryan@gmail.com>
11 months ago
Nuno Campos f66a9d2adf
Automatically add configurable key to config_schema if config_specs i… (#12798)
…s present

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
11 months ago
Praveen Venkateswaran 21eeba075c
enable the device_map parameter in huggingface pipeline (#12731)
### Enabling `device_map` in HuggingFacePipeline 

For multi-gpu settings with large models, the
[accelerate](https://huggingface.co/docs/accelerate/usage_guides/big_modeling#using--accelerate)
library provides the `device_map` parameter to automatically distribute
the model across GPUs / disk.

The [Transformers
pipeline](3520e37e86/src/transformers/pipelines/__init__.py (L543))
enables users to specify `device` (or) `device_map`, and handles cases
(with warnings) when both are specified.

However, Langchain's HuggingFacePipeline only supports specifying
`device` when calling transformers which limits large models and
multi-gpu use-cases.
Additionally, the [default
value](8bd3ce59cd/libs/langchain/langchain/llms/huggingface_pipeline.py (L72))
of `device` is initialized to `-1` , which is incompatible with the
transformers pipeline when `device_map` is specified.

This PR addresses the addition of `device_map` as a parameter , and
solves the incompatibility of `device = -1` when `device_map` is also
specified.
An additional test has been added for this feature. 

Additionally, some existing tests no longer work since 
1. `max_new_tokens` has to be specified under `pipeline_kwargs` and not
`model_kwargs`
2. The GPT2 tokenizer raises a `ValueError: Pipeline with tokenizer
without pad_token cannot do batching`, since the `tokenizer.pad_token`
is `None` ([related
issue](https://github.com/huggingface/transformers/issues/19853) on the
transformers repo).

This PR handles fixing these tests as well.

Co-authored-by: Praveen Venkateswaran <praveen.venkateswaran@ibm.com>
11 months ago
Mark Bell 3276aa3e17
__getattr__ should rase AttributeError not ImportError on missing attributes (#12801)
[The python
spec](https://docs.python.org/3/reference/datamodel.html#object.__getattr__)
requires that `__getattr__` throw `AttributeError` for missing
attributes but there are several places throwing `ImportError` in the
current code base. This causes a specific problem with `hasattr` since
it calls `__getattr__` then looks only for `AttributeError` exceptions.
At present, calling `hasattr` on any of these modules will raise an
unexpected exception that most code will not handle as `hasattr`
throwing exceptions is not expected.

In our case this is triggered by an exception tracker (Airbrake) that
attempts to collect the version of all installed modules with code that
looks like: `if hasattr(mod, "__version__"):`. With `HEAD` this is
causing our exception tracker to fail on all exceptions.

I only changed instances of unknown attributes raising `ImportError` and
left instances of known attributes raising `ImportError`. It feels a
little weird but doesn't seem to break anything.
11 months ago
Illia 71d1a48b66
Use data from all Google search results in SerpApi.com wrapper (#12770)
- **Description:** Use all Google search results data in SerpApi.com
wrapper instead of the first one only
  - **Tag maintainer:** @hwchase17 

_P.S. `libs/langchain/tests/integration_tests/utilities/test_serpapi.py`
are not executed during the `make test`._
11 months ago
Nuno Campos c4fdf78d03
Fix AddableDict raising exception when used with non-addable values (#12785)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
11 months ago
Erick Friis 49e283a0cd
CLI 0.0.13, Configurable Template Demo (#12796) 11 months ago
Nuno Campos d1c6ad7769
Fix on_llm_new_token(chunk=) for some chat models (#12784)
It was passing in message instead of generation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
- **Tag maintainer:** for a quicker response, tag the relevant
maintainer (see below),
- **Twitter handle:** we announce bigger features on Twitter. If your PR
gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in `docs/extras`
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17.
 -->
11 months ago
Erick Friis 070823f294
CLI 0.0.12 (#12787) 11 months ago