langchain

Commit Graph

Author	SHA1	Message	Date
Eugene Yurtsev	b88dfcb42a	Add indexing support (#9614 ) This PR introduces a persistence layer to help with indexing workflows into vectostores. The indexing code helps users to: 1. Avoid writing duplicated content into the vectostore 2. Avoid over-writing content if it's unchanged Importantly, this keeps on working even if the content being written is derived via a set of transformations from some source content (e.g., indexing children documents that were derived from parent documents by chunking.) The two main components are: 1. Persistence layer that keeps track of which keys were updated and when. Keeping track of the timestamp of updates, allows to clean up old content safely, and with minimal complexity. 2. HashedDocument which is used to hash the contents (including metadata) of the documents. We rely on the hashes for identifying duplicates. The indexing code works with ANY document loader. To add transformations to the documents, users for now can add a custom document loader that composes an existing loader together with document transformers. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
刘方瑞	c215481531	Update default index type and metric type for MyScale vector store (#9353 ) We update the default index type from `IVFFLAT` to `MSTG`, a new vector type developed by MyScale.	1 year ago
Joshua Sundance Bailey	a9c86774da	Anthropic: Allow the use of kwargs consistent with ChatOpenAI. (#9515 ) - Description: ~~Creates a new root_validator in `_AnthropicCommon` that allows the use of `model_name` and `max_tokens` keyword arguments.~~ Adds pydantic field aliases to support `model_name` and `max_tokens` as keyword arguments. Ultimately, this makes `ChatAnthropic` more consistent with `ChatOpenAI`, making the two classes more interchangeable for the developer. - Issue: https://github.com/langchain-ai/langchain/issues/9510 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	342087bdfa	fix integration test imports (#9669 )	1 year ago
Keras Conv3d	cbaea8d63b	tair fix distance_type error, and add hybrid search (#9531 ) - fix: distance_type error, - feature: Tair add hybrid search --------- Co-authored-by: thw <hanwen.thw@alibaba-inc.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Eugene Yurtsev	cd81e8a8f2	Add exclude to GenericLoader.from_file_system (#9539 ) support exclude param in GenericLoader.from_filesystem --------- Co-authored-by: Kyle Pancamo <50267605+KylePancamo@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Jacob Lee	278ef0bdcf	Adds ChatOllama (#9628 ) @rlancemartin --------- Co-authored-by: Adilkhan Sarsen <54854336+adolkhan@users.noreply.github.com> Co-authored-by: Kim Minjong <make.dirty.code@gmail.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Lance Martin <lance@langchain.dev> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Nuno Campos	20ce283fa7	Format	1 year ago
Nuno Campos	6424b3cde0	Add another test	1 year ago
William FH	da18e177f1	Update libs/langchain/langchain/schema/runnable/base.py Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	1 year ago
Nuno Campos	c326751085	Lint	1 year ago
Nuno Campos	6d19709b65	RunnableLambda, if func returns a Runnable, run it	1 year ago
Nuno Campos	677da6a0fd	Add support for async funcs in RunnableSequence	1 year ago
Nuno Campos	1751fe114d	Add one more test	1 year ago
Nuno Campos	882b97cfd2	Lint	1 year ago
Nuno Campos	3ddabe8b2c	Code review	1 year ago
Nuno Campos	fdcd50aab4	Extend test	1 year ago
Nuno Campos	9777c2801d	Update method and docstring	1 year ago
Nuno Campos	93bbf67afc	WIP Add test Add test Lint	1 year ago
Nuno Campos	c184be5511	Use a shared executor for all parallel calls	1 year ago
Nuno Campos	db4b256a28	Add error for batch of 0	1 year ago
Nuno Campos	3458489936	Lint	1 year ago
Nuno Campos	e420bf22b6	Lint	1 year ago
Nuno Campos	cc83f54694	L:int	1 year ago
Nuno Campos	d414d47c78	Use a shared executor for all parallel calls	1 year ago
Bagatur	a40c12bb88	Update the nlpcloud connector after some changes on the NLP Cloud API (#9586 ) - Description: remove some text generation deprecated parameters and update the embeddings doc, - Tag maintainer: @rlancemartin	1 year ago
Bagatur	e2e582f1f6	Fixed source key name for docugami loader (#8598 ) The Docugami loader was not returning the source metadata key. This was triggering this exception when used with retrievers, per https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/schema/prompt_template.py#L193C1-L195C41 The fix is simple and just updates the metadata key name for the document each chunk is sourced from, from "name" to "source" as expected. I tested by running the python notebook that has an end to end scenario in it. Tagging DataLoader maintainers @rlancemartin @eyurtsev	1 year ago
karynzv	5508baf1eb	Add CrateDB prompt (#9657 ) Adds a prompt template for the CrateDB SQL dialect.	1 year ago
Bagatur	a8e8a31b41	Merge branch 'master' into bagatur/locals_in_config	1 year ago
Bagatur	ef2500584c	fmt	1 year ago
Zizhong Zhang	8a03836160	docs: fix PromptGuard docs (#9659 ) Fix PromptGuard docs. Noticed several trivial issues on the docs when integrating the new class. cc @baskaryan	1 year ago
Guy Korland	39a5d02225	Cleanup of ruff warnings use isinstance() instead of type() (#9655 ) Minor cosmetic PR just cleanup of `ruff` warnings use `isinstance()` instead of `type()`	1 year ago
Joseph McElroy	2a06e7b216	ElasticsearchStore: improve error logging for adding documents (#9648 ) Not obvious what the error is when you cannot index. This pr adds the ability to log the first errors reason, to help the user diagnose the issue. Also added some more documentation for when you want to use the vectorstore with an embedding model deployed in elasticsearch. Credit: @elastic and @phoey1	1 year ago
Julien Salinas	f1072cc31f	Merge branch 'master' into master	1 year ago
Jun Liu	b379c5f9c8	Fixed the error on ConfluenceLoader when content_format=VIEW and `keep_markdown_format`=True (#9633 ) - Description: a description of the change when I set `content_format=ContentFormat.VIEW` and `keep_markdown_format=True` on ConfluenceLoader, it shows the following error: ``` langchain/document_loaders/confluence.py", line 459, in process_page page["body"]["storage"]["value"], heading_style="ATX" KeyError: 'storage' ``` The reason is because the content format was set to `view` but it was still trying to get the content from `page["body"]["storage"]["value"]`. Also added the other content formats which are supported by Atlassian API https://stackoverflow.com/questions/34353955/confluence-rest-api-expanding-page-body-when-retrieving-page-by-title/34363386#34363386 - Issue: the issue # it fixes (if applicable), Not applicable. - Dependencies: any dependencies required for this change, Added optional dependency `markdownify` if anyone wants to extract in markdown format. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Gabriel Fu	b2d9970fc1	Allow specifying dtype in `langchain.llms.VLLM` (#9635 ) - Description: add `dtype` argument for VLLM - Issue: #9593 - Dependencies: none - Tag maintainer: @hwchase17, @baskaryan	1 year ago
anifort	900c1f3e8d	Add support for structured data sources with google enterprise search (#9037 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Added the capability to handles structured data from google enterprise search, - Issue: Retriever failed when underline search engine was integrated with structured data, - Dependencies: google-api-core - Tag maintainer: @jarokaz - Twitter handle: anifort Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> --------- Co-authored-by: Christos Aniftos <aniftos@google.com> Co-authored-by: Holt Skinner <13262395+holtskinner@users.noreply.github.com> Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	1 year ago
Harrison Chase	02545a54b3	python repl improvement for csv agent (#9618 )	1 year ago
Erick Friis	fc64e6349e	Hub stub updates (#9577 ) Updates the hub stubs to not fail when no api key is found. For supporting singleton tenants and default values from sdk 0.1.6. Also adds the ability to define is_public and description for backup repo creation on push.	1 year ago
Kim Minjong	ca8232a3c1	Update BaseChatModel.astream to respect generation_info (#9430 ) Currently, generation_info is not respected by only reflecting messages in chunks. Change it to add generations so that generation chunks are merged properly. --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Bagatur	81163e3c0c	parent retriever nit (#9570 ) if ids are nullable seems like they should have default val None. mirrors VectorStore interface as well. cc @mcantillon21 @jacoblee93	1 year ago
Myeongseop Kim	f1e602996a	import tqdm.auto instead of tqdm tqdm for OpenAIEmbeddings (#9584 ) - Description: current code does not work very well on jupyter notebook, so I changed the code so that it imports `tqdm.auto` instead. - Issue: #9582 - Dependencies: N/A - Tag maintainer: @hwchase17, @baskaryan - Twitter handle: N/A Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	1 year ago
Predrag Gruevski	d564ec944c	`poetry lock` the experimental package. (#9478 )	1 year ago
Predrag Gruevski	65e893b9cd	`poetry lock` on langchain. (#9476 )	1 year ago
Predrag Gruevski	3c7cc4d440	Test experimental package with `langchain` on `master` branch. (#9621 ) It's possible that langchain-experimental works fine with the latest published langchain, but is broken with the langchain on `master`. Unfortunately, you can see this is currently the case — this is why this PR also includes a minor fix for the `langchain` package itself. We want to catch situations like that before releasing a new langchain, hence this test.	1 year ago
Eugene Yurtsev	3408810748	Add batch util (#9620 ) Add `batch` utility to langchain	1 year ago
Bagatur	2b663089b5	bump 271 (#9615 )	1 year ago
klae01	b868ef23bc	Add AINetwork blockchain toolkit integration (#9527 ) # Description This PR introduces a new toolkit for interacting with the AINetwork blockchain. The toolkit provides a set of tools for performing various operations on the AINetwork blockchain, such as transferring AIN, reading and writing values to the blockchain database, managing apps, setting rules and owners. # Dependencies [ain-py](https://github.com/ainblockchain/ain-py) >= 1.0.2 # Misc The example notebook (langchain/docs/extras/integrations/toolkits/ainetwork.ipynb) is in the PR --------- Co-authored-by: kriii <kriii@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	e99ef12cb1	Bagatur/litellm model name (#9613 ) Co-authored-by: ishaan-jaff <ishaanjaffer0324@gmail.com>	1 year ago
Harrison Chase	1720e99397	add variables for field names (#9563 )	1 year ago

1 2 3 4 5 ...

550 Commits (b88dfcb42a52eab78ccdcd38c1f1833c2acf3caf)