langchain

Commit Graph

Author	SHA1	Message	Date
Mike Wang	c3044b1bf0	[test] Add integration_test for PandasAgent (#4056 ) - confirm creation - confirm functionality with a simple dimension check. The test now is calling OpenAI API directly, but learning from @vowelparrot that we’re caching the requests, so that it’s not that expensive. I also found we’re calling OpenAI api in other integration tests. Please lmk if there is any concern of real external API calls. I can alternatively make a fake LLM for this test. Thanks	1 year ago
Aivin V. Solatorio	6567b73e1a	JSON loader (#4067 ) This implements a loader of text passages in JSON format. The `jq` syntax is used to define a schema for accessing the relevant contents from the JSON file. This requires dependency on the `jq` package: https://pypi.org/project/jq/. --------- Signed-off-by: Aivin V. Solatorio <avsolatorio@gmail.com>	1 year ago
hp0404	2a3c5f8353	Update WhatsAppChatLoader regex to handle multiple date-time formats (#4186 ) This PR updates the `message_line_regex` used by `WhatsAppChatLoader` to support different date-time formats used in WhatsApp chat exports; resolves #4153. The new regex handles the following input formats: ```terminal [05.05.23, 15:48:11] James: Hi here [11/8/21, 9:41:32 AM] User name: Message 123 1/23/23, 3:19 AM - User 2: Bye! 1/23/23, 3:22_AM - User 1: And let me know if anything changes ``` Tests have been added to verify that the loader works correctly with all formats.	1 year ago
Zander Chase	84cfa76e00	Update Cohere Reranker (#4180 ) The forward ref annotations don't get updated if we only iimport with type checking --------- Co-authored-by: Abhinav Verma <abhinav_win12@yahoo.co.in>	1 year ago
Zander Chase	6032a051e9	Add Tenant ID to V2 Tracer (#4135 ) Update the V2 tracer to - use UUIDs instead of int's - load a tenant ID and use that when saving sessions	1 year ago
Zander Chase	2f087d63af	Fix Python RePL Tool (#4137 ) Filter out kwargs from inferred schema when determining if a tool is single input. Add a couple unit tests. Move tool unit tests to the tools dir	1 year ago
Harrison Chase	d4cf1eb60a	Add firestore memory (#3792 ) (#3941 ) If you have any other suggestions or feedback, please let me know. --------- Co-authored-by: yakigac <10434946+yakigac@users.noreply.github.com>	1 year ago
Mike Wang	67db495fcf	[agent] Add Spark Agent (#4020 ) - added support for spark through pyspark library. - added jupyter notebook as example.	1 year ago
rogerserper	b1446bea5f	google-serper: async + full json results + support for Google Images, Places and News (#4078 ) * implemented arun, results, and aresults. Reuses aiosession if available. * helper tools GoogleSerperRun and GoogleSerperResults * support for Google Images, Places and News (examples given) and filtering based on time (e.g. past hour) * updated docs	1 year ago
Zander Chase	65c3b146c9	Accept str or list[str] for shell (#4060 ) Relax the requirements	1 year ago
hp0404	374725a715	Refactor TelegramChatLoader and FacebookChatLoader classes and add tests (#3863 ) This PR includes two main changes: - Refactor the `TelegramChatLoader` and `FacebookChatLoader` classes by removing the dependency on pandas and simplifying the message filtering process. - Add test cases for the `TelegramChatLoader` and `FacebookChatLoader` classes. This test ensures that the class correctly loads and processes the example chat data, providing better test coverage for this functionality.	1 year ago
Jon Saginaw	ea64b1716d	Enhancement: option to Get All Tokens with a single Blockchain Document Loader call (#3797 ) The Blockchain Document Loader's default behavior is to return 100 tokens at a time which is the Alchemy API limit. The Document Loader exposes a startToken that can be used for pagination against the API. This enhancement includes an optional get_all_tokens param (default: False) which will: - Iterate over the Alchemy API until it receives all the tokens, and return the tokens in a single call to the loader. - Manage all/most tokenId formats (this can be int, hex16 with zero or all the leading zeros). There aren't constraints as to how smart contracts can represent this value, but these three are most common. Note that a contract with 10,000 tokens will issue 100 calls to the Alchemy API, and could take about a minute, which is why this param will default to False. But I've been using the doc loader with these utilities on the side, so figured it might make sense to build them in for others to use.	1 year ago
Zander Chase	afa9d1292b	Re-Permit Partials in `Tool` (#4058 ) Resolved issue #4053 Now that StructuredTool is a separate class, this constraint is no longer needed. Added/updated a unit test	1 year ago
Harrison Chase	a5dd73c1a6	Revert "[agent][property type] Change allowed_tools to Set as Duplicate doesn’t make sense" (#4014 ) Reverts hwchase17/langchain#3840	1 year ago
Davis Chase	f08a76250f	Better custom model handling OpenAICallbackHandler (#4009 ) Thanks @maykcaldas for flagging! think this should resolve #3988. Let me know if you still see issues after next release.	1 year ago
Harrison Chase	48ea27ba60	Harrison/blockwise sitemap (#3940 ) Co-authored-by: Martin Holzhauer <martin@holzhauer.eu>	1 year ago
Harrison Chase	f04faf8496	Harrison/spreedly (#3937 ) Co-authored-by: Esmit Pérez <esmitperez@users.noreply.github.com>	1 year ago
Harrison Chase	cd3f8582cb	Harrison/combined memory (#3935 ) Co-authored-by: engkheng <60956360+outday29@users.noreply.github.com>	1 year ago
Zander Chase	c4cb55a0c5	[Breaking] Migrate GPT4All to use PyGPT4All (#3934 ) Seems the pyllamacpp package is no longer the supported bindings from gpt4all. Tested that this works locally. Given that the older models weren't very performant, I think it's better to migrate now without trying to include a lot of try / except blocks --------- Co-authored-by: Nissan Pow <npow@users.noreply.github.com> Co-authored-by: Nissan Pow <pownissa@amazon.com>	1 year ago
Zander Chase	c582f2e9e3	Add Structure Chat Agent (#3912 ) Create a new chat agent that is compatible with the Multi-input tools	1 year ago
Mike Wang	ec21b7126c	[agent][property type] Change allowed_tools to Set as Duplicate doesn’t make sense (#3840 ) - ActionAgent has a property called, `allowed_tools`, which is declared as `List`. It stores all provided tools which is available to use during agent action. - This collection shouldn’t allow duplicates. The original datatype List doesn’t make sense. Each tool should be unique. Even when there are variants (assuming in the future), it would be named differently in load_tools. Test: - confirm the functionality in an example by initializing an agent with a list of 2 tools and confirm everything works. ```python3 def test_agent_chain_chat_bot(): from langchain.agents import load_tools from langchain.agents import initialize_agent from langchain.agents import AgentType from langchain.chat_models import ChatOpenAI from langchain.llms import OpenAI from langchain.utilities.duckduckgo_search import DuckDuckGoSearchAPIWrapper chat = ChatOpenAI(temperature=0) llm = OpenAI(temperature=0) tools = load_tools(["ddg-search", "llm-math"], llm=llm) agent = initialize_agent(tools, chat, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True) agent.run("Who is Olivia Wilde's boyfriend? What is his current age raised to the 0.23 power?") test_agent_chain_chat_bot() ``` Result: <img width="863" alt="Screenshot 2023-05-01 at 7 58 11 PM" src="https://user-images.githubusercontent.com/62768671/235572157-0937594c-ddfb-4760-acb2-aea4cacacd89.png">	1 year ago
Davis Chase	e7e29f9937	Dev2049/add modern treasury (#3924 ) Modified Modern Treasury and Strip slightly so credentials don't have to be passed in explicitly. Thanks @mattgmarcus for adding Modern Treasury! --------- Co-authored-by: Matt Marcus <matt.g.marcus@gmail.com>	1 year ago
Davis Chase	5db6b796cf	Dev2049/hf emb encode kwargs (#3925 ) Thanks @amogkam for the addition! Refactored slightly --------- Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>	1 year ago
Zander Chase	9b9b231e10	Update some Tools Docs (#3913 ) Haven't gotten to all of them, but this: - Updates some of the tools notebooks to actually instantiate a tool (many just show a 'utility' rather than a tool. More changes to come in separate PR) - Move the `Tool` and decorator definitions to `langchain/tools/base.py` (but still export from `langchain.agents`) - Add scene explain to the load_tools() function - Add unit tests for public apis for the langchain.tools and langchain.agents modules	1 year ago
Zander Chase	84ea17b786	Move Tool Validation (#3923 ) Move tool validation to each implementation of the Agent. Another alternative would be to adjust the `_validate_tools()` signature to accept the output parser (and format instructions) and add logic there. Something like `parser.outputs_structured_actions(format_instructions)` But don't think that's needed right now.	1 year ago
Eugene Yurtsev	7cce68a051	Add minimal file system blob loader (#3669 ) This adds a minimal file system blob loader. If looks good, this PR will be merged and a few additional enhancements will be made.	1 year ago
Zura Isakadze	647bbf61c1	Add SQLiteChatMessageHistory (#3534 ) It's based on already existing `PostgresChatMessageHistory` Use case somewhere in between multiple files and Postgres storage.	1 year ago
James Brotchie	921894960b	Add ChatModel, LLM, and Embeddings for Google's PaLM APIs (#3575 ) - Add langchain.llms.GooglePalm for text completion, - Add langchain.chat_models.ChatGooglePalm for chat completion, - Add langchain.embeddings.GooglePalmEmbeddings for sentence embeddings, - Add example field to HumanMessage and AIMessage so that users can feed in examples into the PaLM Chat API, - Add system and unit tests. Note async completion for the Text API is not yet supported and will be included in a future PR. Happy for feedback on any aspect of this PR, especially our choice of adding an example field to Human and AI Message objects to enable passing example messages to the API.	1 year ago
Roma	d15f481352	Add unit test to output parsers (#3911 ) This pull request adds unit tests for various output parsers (BooleanOutputParser, CommaSeparatedListOutputParser, and StructuredOutputParser) to ensure their correct functionality and to increase code reliability and maintainability. The tests cover both valid and invalid input cases. Changes: Added unit tests for BooleanOutputParser. Added unit tests for CommaSeparatedListOutputParser. Added unit tests for StructuredOutputParser. Testing: All new unit tests have been executed, and they pass successfully. The overall test suite has been run, and all tests pass. Notes: These tests cover both successful parsing scenarios and error handling for invalid inputs. If any new output parsers are added in the future, corresponding unit tests should also be created to maintain coverage.	1 year ago
Davis Chase	2451310975	Chroma fix mmr (#3897 ) Fixes #3628, thanks @derekmoeller for the issue!	1 year ago
Davis Chase	52e4fba897	Fix self query pinecone translation (#3892 ) Enum to string conversion handled differently between python 3.9 and 3.11, currently breaking in 3.11 (see #3788). Thanks @peter-brady for catching this!	1 year ago
Zander Chase	19912d755e	Vwp/arxiv (#3855 ) Co-authored-by: Mike Wang <62768671+skcoirz@users.noreply.github.com>	1 year ago
Zander Chase	d7e17fc8fe	Deprecate StdInquireTool (#3850 ) - Deprecate StdInInquire tool (dup of HumanInputRun) - Expose missing tools from `langchain.tools`	1 year ago
Ankush Gola	d3ec00b566	Callbacks Refactor [base] (#3256 ) Co-authored-by: Nuno Campos <nuno@boringbits.io> Co-authored-by: Davis Chase <130488702+dev2049@users.noreply.github.com> Co-authored-by: Zander Chase <130414180+vowelparrot@users.noreply.github.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	bd7e0a534c	Harrison/csv loader (#3771 ) Co-authored-by: mrT23 <tal.r@codium.ai>	1 year ago
Mike Wang	ce4fea983b	[simple] added test case and improve self class return type annotation (#3773 ) a simple follow up of https://github.com/hwchase17/langchain/pull/3748 - added test case - improve annotation when function return type is class itself.	1 year ago
Harrison Chase	0c0f14407c	Harrison/tair (#3770 ) Co-authored-by: Seth Huang <848849+seth-hg@users.noreply.github.com>	1 year ago
Mike Wang	512c24fc9c	[annotation improvement] Make AgentType->Class Conversion More Scalable (#3749 ) In the current solution, AgentType and AGENT_TO_CLASS are placed in two separate files and both manually maintained. This might cause inconsistency when we update either of them. — latest — based on the discussion with hwchase17, we don’t know how to further use the newly introduced AgentTypeConfig type, so it doesn’t make sense yet to add it. Instead, it’s better to move the dictionary to another file to keep the loading.py file clear. The consistency is a good point. Instead of asserting the consistency during linting, we added a unittest for consistency check. I think it works as auto unittest is triggered every time with clear failure notice. (well, force push is possible, but we all know what we are doing, so let’s show trust. :>) ~~This PR includes~~ - ~~Introduced AgentTypeConfig as the source of truth of all AgentType related meta data.~~ - ~~Each AgentTypeConfig is a annotated class type which can be used for annotation in other places.~~ - ~~Each AgentTypeConfig can be easily extended when we have more meta data needs.~~ - ~~Strong assertion to ensure AgentType and AGENT_TO_CLASS are always consistent.~~ - ~~Made AGENT_TO_CLASS automatically generated.~~ ~~Test Plan:~~ - ~~since this change is focusing on annotation, lint is the major test focus.~~ - ~~lint, format and test passed on local.~~	1 year ago
Harrison Chase	be7a8e0824	Harrison/redis cache (#3766 ) Co-authored-by: Tyler Hutcherson <tyler.hutcherson@redis.com>	1 year ago
Mike Wang	b588446bf9	[simple][test] Added test case for schema.py (#3692 ) - added unittest for schema.py covering utility functions and token counting. - fixed a nit. based on huggingface doc, the tokenizer model is gpt-2. [link](https://huggingface.co/transformers/v4.8.2/_modules/transformers/models/gpt2/tokenization_gpt2_fast.html) - make lint && make format, passed on local - screenshot of new test running result <img width="1283" alt="Screenshot 2023-04-27 at 9 51 55 PM" src="https://user-images.githubusercontent.com/62768671/235057441-c0ac3406-9541-453f-ba14-3ebb08656114.png">	1 year ago
Jon Saginaw	f8d69e4e52	Enhancement: Blockchain Document Loader with better Metadata support (#3710 ) This PR includes some minor alignment updates, including: - metadata object extended to support contractAddress, blockchainType, and tokenId - notebook doc better aligned to standard langchain format - startToken changed from int to str to support multiple hex value types on the Alchemy API The updated metadata will look like the below. It's possible for a single contractAddress to exist across multiple blockchains (e.g. Ethereum, Polygon, etc.) so it's important to include the blockchainType. ``` metadata = {"source": self.contract_address, "blockchain": self.blockchainType, "tokenId": tokenId} ```	1 year ago
Davis Chase	220a7076ac	Add Mathpix pdf loader (#3727 ) Inspo https://twitter.com/danielgross/status/1651695062307274754?s=46&t=1zHLap5WG4I_kQPPjfW9fA Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	40f6e60e68	Harrison/stripe (#3762 ) Co-authored-by: Ismail Pelaseyed <homanp@gmail.com>	1 year ago
Rafal Wojdyla	160bfae93f	Add `DocstoreFn` - lookup doc via arbitrary function (#3760 ) This partially addresses https://github.com/hwchase17/langchain/issues/1524, but it's also useful for some of our use cases. This `DocstoreFn` allows to lookup a document given a function that accepts the `search` string without the need to implement a custom `Docstore`. This could be useful when: * you don't want to implement a `Docstore` just to provide a custom `search` * it's expensive to construct an `InMemoryDocstore`/dict * you retrieve documents from remote sources * you just want to reuse existing objects	1 year ago
Zander Chase	5042bd40d3	Add Shell Tool (#3335 ) Create an official bash shell tool to replace the dynamically generated one	1 year ago
Zander Chase	334c162f16	Add Other File Utilities (#3209 ) Add other File Utilities, include - List Directory - Search for file - Move - Copy - Remove file Bundle as toolkit Add a notebook that connects to the Chat Agent, which somewhat supports multi-arg input tools Update original read/write files to return the original dir paths and better handle unsupported file paths. Add unit tests	1 year ago
Zander Chase	da7b51455c	Dynamic tool -> single purpose (#3697 ) I think the logic of https://github.com/hwchase17/langchain/pull/3684#pullrequestreview-1405358565 is too confusing. I prefer this alternative because: - All `Tool()` implementations by default will be treated the same as before. No breaking changes. - Less reliance on pydantic magic - The decorator (which only is typed as returning a callable) can infer schema and generate a structured tool - Either way, the recommended way to create a custom tool is through inheriting from the base tool	1 year ago
Zander Chase	4654c58f72	Add validation on agent instantiation for multi-input tools (#3681 ) Tradeoffs here: - No lint-time checking for compatibility - Differs from JS package - The signature inference, etc. in the base tool isn't simple - The `args_schema` is optional Pros: - Forwards compatibility retained - Doesn't break backwards compatibility - User doesn't have to think about which class to subclass (single base tool or dynamic `Tool` interface regardless of input) - No need to change the load_tools, etc. interfaces Co-authored-by: Hasan Patel <mangafield@gmail.com>	1 year ago
Davis Chase	b807a114e4	Add query parsing unit tests (#3672 )	1 year ago
Eugene Yurtsev	708787dddb	Blob: Add validator and use future annotations (#3650 ) Minor changes to the Blob schema. --------- Co-authored-by: Zander Chase <130414180+vowelparrot@users.noreply.github.com>	1 year ago
Eugene Yurtsev	c5a4b4fea1	Suppress duckdb warning in unit tests explicitly (#3653 ) This catches the warning raised when using duckdb, asserts that it's as expected. The goal is to resolve all existing warnings to make unit-testing much stricter.	1 year ago
Eugene Yurtsev	e6c8cce050	Add unit-test to catch changes to required deps (#3662 ) This adds a unit test that can catch changes to required dependencies	1 year ago
Eugene Yurtsev	055f58960a	Fix pytest collection warning (#3651 ) Fixes a pytest collection warning because the test class starts with the prefix "Test"	1 year ago
plutopulp	6d6fd1b9e1	Add PipelineAI LLM integration (#3644 ) Add PipelineAI LLM integration	1 year ago
Harrison Chase	a35bbbfa9e	Harrison/lancedb (#3634 ) Co-authored-by: Minh Le <minhle@canva.com>	1 year ago
Eugene Yurtsev	5d02010763	Introduce Blob and Blob Loader interface (#3603 ) This PR introduces a Blob data type and a Blob loader interface. This is the first of a sequence of PRs that follows this proposal: https://github.com/hwchase17/langchain/pull/2833 The primary goals of these abstraction are: * Decouple content loading from content parsing code. * Help duplicated content loading code from document loaders. * Make lazy loading a default for langchain.	1 year ago
Harrison Chase	ab749fa1bb	Harrison/opensearch logic (#3631 ) Co-authored-by: engineer-matsuo <95115586+engineer-matsuo@users.noreply.github.com>	1 year ago
Ehsan M. Kermani	4a246e2fd6	Allow clearing cache and fix gptcache (#3493 ) This PR * Adds `clear` method for `BaseCache` and implements it for various caches * Adds the default `init_func=None` and fixes gptcache integtest * Since right now integtest is not running in CI, I've verified the changes by running `docs/modules/models/llms/examples/llm_caching.ipynb` (until proper e2e integtest is done in CI)	1 year ago
cs0lar	440c98e24b	Fix/issue 2695 (#3608 ) ## Background fixes #2695 ## Changes The `add_text` method uses the internal embedding function if one was passes to the `Weaviate` constructor. NOTE: the latest merge on the `Weaviate` class made the specification of a `weaviate_api_key` mandatory which might not be desirable for all users and connection methods (for example weaviate also support Embedded Weaviate which I am happy to add support to here if people think it's desirable). I wrapped the fetching of the api key into a try catch in order to allow the `weaviate_api_key` to be unspecified. Do let me know if this is unsatisfactory. ## Test Plan added test for `add_texts` method.	1 year ago
leo-gan	36c59e0c25	`Arxiv` document loader (#3627 ) It makes sense to use `arxiv` as another source of the documents for downloading. - Added the `arxiv` document_loader, based on the `utilities/arxiv.py:ArxivAPIWrapper` - added tests - added an example notebook - sorted `__all__` in `__init__.py` (otherwise it is hard to find a class in the very long list)	1 year ago
Zander Chase	443a893ffd	Align names of search tools (#3620 ) Tools for Bing, DDG and Google weren't consistent even though the underlying implementations were. All three services now have the same tools and implementations to easily switch and experiment when building chains.	1 year ago
Maciej Bryński	aa345a4bb7	Add get_text_separator parameter to BSHTMLLoader (#3551 ) By default get_text doesn't separate content of different HTML tag. Adding option for specifying separator helps with document splitting.	1 year ago
Zander Chase	ee670c448e	Persistent Bash Shell (#3580 ) Clean up linting and make more idiomatic by using an output parser --------- Co-authored-by: FergusFettes <fergusfettes@gmail.com>	1 year ago
Davis Chase	d18b0caf0e	Add Anthropic default request timeout (#3540 ) thanks @hitflame! --------- Co-authored-by: Wenqiang Zhao <hitzhaowenqiang@sina.com> Co-authored-by: delta@com <delta@com>	1 year ago
Roma	2b4e9a3efa	Add unit test for _merge_splits function (#3513 ) This commit adds a new unit test for the _merge_splits function in the text splitter. The new test verifies that the function merges text into chunks of the correct size and overlap, using a specified separator. The test passes on the current implementation of the function.	1 year ago
yakigac	f338d6251c	Add a test for cosmos db memory (#3525 ) Test for #3434 @eavanvalkenburg Initially, I was unaware and had submitted a pull request #3450 for the same purpose, but I have now repurposed the one I used for that. And it worked.	1 year ago
Harrison Chase	0fc0aa62f2	Harrison/blockchain docloader (#3491 ) Co-authored-by: Jon Saginaw <saginawj@users.noreply.github.com>	1 year ago
Harrison Chase	707741de58	Harrison/prediction guard (#3490 ) Co-authored-by: Daniel Whitenack <whitenack.daniel@gmail.com>	1 year ago
Harrison Chase	7257f9e015	Harrison/tfidf parameters (#3481 ) Co-authored-by: pao <go5kuramubon@gmail.com> Co-authored-by: KyoHattori <kyo.hattori@abejainc.com>	1 year ago
Harrison Chase	eda69b13f3	openai embeddings (#3488 )	1 year ago
Harrison Chase	408a0183cd	Harrison/weaviate (#3494 ) Co-authored-by: Nick Rubell <nick@rubell.com>	1 year ago
Mindaugas Sharskus	a4d85f7fd5	[Fix #3365 ]: Changed regex to cover new line before action serious (#3367 ) Fix for: [Changed regex to cover new line before action serious.](https://github.com/hwchase17/langchain/issues/3365) --- This PR fixes the issue where `ValueError: Could not parse LLM output:` was thrown on seems to be valid input. Changed regex to cover new lines before action serious (after the keywords "Action:" and "Action Input:"). regex101: https://regex101.com/r/CXl1kB/1 --------- Co-authored-by: msarskus <msarskus@cisco.com>	1 year ago
Davis Chase	b2564a6391	fix #3884 (#3475 ) fixes mar bug #3384	1 year ago
Zander Chase	416f3bdf11	Vwp/alpaca streaming (#3468 ) Co-authored-by: Luke Stanley <306671+lukestanley@users.noreply.github.com>	1 year ago
cs0lar	3033c6b964	fixes #1214 (#3003 ) ### Background Continuing to implement all the interface methods defined by the `VectorStore` class. This PR pertains to implementation of the `max_marginal_relevance_search_by_vector` method. ### Changes - a `max_marginal_relevance_search_by_vector` method implementation has been added in `weaviate.py` - tests have been added to the the new method - vcr cassettes have been added for the weaviate tests ### Test Plan Added tests for the `max_marginal_relevance_search_by_vector` implementation ### Change Safety - [x] I have added tests to cover my changes	1 year ago
Zander Chase	49122a96e7	Structured Tool Bugfixes (#3324 ) - Proactively raise error if a tool subclasses BaseTool, defines its own schema, but fails to add the type-hints - fix the auto-inferred schema of the decorator to strip the unneeded virtual kwargs from the schema dict Helps avoid silent instances of #3297	1 year ago
Davit Buniatyan	2c0023393b	Deep Lake mini upgrades (#3375 ) Improvements * set default num_workers for ingestion to 0 * upgraded notebooks for avoiding dataset creation ambiguity * added `force_delete_dataset_by_path` * bumped deeplake to 3.3.0 * creds arg passing to deeplake object that would allow custom S3 Notes * please double check if poetry is not messed up (thanks!) Asks * Would be great to create a shared slack channel for quick questions --------- Co-authored-by: Davit Buniatyan <d@activeloop.ai>	1 year ago
Zander Chase	20f530e9c5	Add Sentence Transformers Embeddings (#3409 ) Add embeddings based on the sentence transformers library. Add a notebook and integration tests. Co-authored-by: khimaros <me@khimaros.com>	1 year ago
Luke Harris	b4de839ed8	Several confluence loader improvements (#3300 ) This PR addresses several improvements: - Previously it was not possible to load spaces of more than 100 pages. The `limit` was being used both as an overall page limit and as a per request pagination limit. This, in combination with the fact that atlassian seem to use a server-side hard limit of 100 when page content is expanded, meant it wasn't possible to download >100 pages. Now `limit` is used only as a per-request pagination limit and `max_pages` is introduced as the way to limit the total number of pages returned by the paginator. - Document metadata now includes `source` (the source url), making it compatible with `RetrievalQAWithSourcesChain`. - It is now possible to include inline and footer comments. - It is now possible to pass `verify_ssl=False` and other parameters to the confluence object for use cases that require it.	1 year ago
Harrison Chase	a6664be79c	Harrison/myscale (#3352 ) Co-authored-by: Fangrui Liu <fangruil@moqi.ai> Co-authored-by: 刘方瑞 <fangrui.liu@outlook.com> Co-authored-by: Fangrui.Liu <fangrui.liu@ubc.ca>	1 year ago
Filip Haltmayer	215dcc2d26	Refactor Milvus/Zilliz (#3047 ) Refactoring milvus/zilliz to clean up and have a more consistent experience. Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>	1 year ago
Richy Wang	88a8f59aa7	Add a full PostgresSQL syntax database 'AnalyticDB' as vector store. (#3135 ) Hi there！ I'm excited to open this PR to add support for using a fully Postgres syntax compatible database 'AnalyticDB' as a vector. As AnalyticDB has been proved can be used with AutoGPT, ChatGPT-Retrieve-Plugin, and LLama-Index, I think it is also good for you. AnalyticDB is a distributed Alibaba Cloud-Native vector database. It works better when data comes to large scale. The PR includes: - [x] A new memory: AnalyticDBVector - [x] A suite of integration tests verifies the AnalyticDB integration I have read your [contributing guidelines](`72b7d76d79/.github/CONTRIBUTING.md`). And I have passed the tests below - [x] make format - [x] make lint - [x] make coverage - [x] make test	1 year ago
Zander Chase	05a8aa5447	Fix linting on master (#3327 )	1 year ago
Varun Srinivas	d2f922f525	Change in method name for creating an issue on JIRA (#3307 ) The awesome JIRA tool created by @zywilliamli calls the `create_issue()` method to create issues, however, the actual method is `issue_create()`. Details in the Documentation here: https://atlassian-python-api.readthedocs.io/jira.html#manage-issues	1 year ago
Paul Garner	aa9d5707e0	Add PythonLoader which auto-detects encoding of Python files (#3311 ) This PR contributes a `PythonLoader`, which inherits from `TextLoader` but detects and sets the encoding automatically.	1 year ago
Davis Chase	2fd24d31a4	Cleanup integration test dir (#3308 )	1 year ago
Naveen Tatikonda	bb6c459f7a	OpenSearch: Add Support for Lucene Filter (#3201 ) ### Description Add Support for Lucene Filter. When you specify a Lucene filter for a k-NN search, the Lucene algorithm decides whether to perform an exact k-NN search with pre-filtering or an approximate search with modified post-filtering. This filter is supported only for approximate search with the indexes that are created using `lucene` engine. OpenSearch Documentation - https://opensearch.org/docs/latest/search-plugins/knn/filter-search-knn/#lucene-k-nn-filter-implementation Signed-off-by: Naveen Tatikonda <navtat@amazon.com>	1 year ago
Davis Chase	46542dc774	Contextual compression retriever (#2915 ) Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Gabriel Altay	34fb56b633	fix copy/pasta typos wikipedia->arxiv (#3222 ) just updates a few module level docstrings from Wikipedia -> Arxiv	1 year ago
Harrison Chase	d2520a5f1e	Harrison/ddg (#3206 ) Co-authored-by: itai <itai.marks@gmail.com> Co-authored-by: Itai Marks <itaim@users.noreply.github.com> Co-authored-by: Tianyi Pan <60060750+tipani86@users.noreply.github.com> Co-authored-by: Tianyi Pan <tianyi.pan@clobotics.com> Co-authored-by: Adilzhan Ismailov <13088690+aismlv@users.noreply.github.com> Co-authored-by: Justin Flick <Justinjayflick@gmail.com> Co-authored-by: Justin Flick <jflick@homesite.com>	1 year ago
Harrison Chase	9a0356d276	Harrison/file chat history (#3198 ) Co-authored-by: Young Lee <joybro201@gmail.com>	1 year ago
Harrison Chase	9181cd9b22	Harrison/playwright selector (#3185 ) Co-authored-by: zhyuri <4649294+zhyuri@users.noreply.github.com>	1 year ago
Harrison Chase	68cd37175e	Harrison/arxiv tool (#3186 ) Co-authored-by: leo-gan <leo.gan.57@gmail.com>	1 year ago
Zander Chase	4adfd790f0	Update File Management Tools to Include Root Directory (#3112 ) - Permit the specification of a `root_dir` to the read/write file tools to specify a working directory - Add validation for attempts to read/write outside the directory (e.g., through `../../` or symlinks or `/abs/path`'s that don't lie in the correct path) - Add some tests for all One question is whether we should make a default root directory for these? tradeoffs either way	1 year ago
engkheng	dbbc340f25	Validate `input_variables` when using `jinja2` templates (#3140 ) `langchain.prompts.PromptTemplate` and `langchain.prompts.FewShotPromptTemplate` do not validate `input_variables` when initialized as `jinja2` template. ```python # Using langchain v0.0.144 template = """"\ Your variable: {{ foo }} {% if bar %} You just set bar boolean variable to true {% endif %} """ # Missing variable, should raise ValueError prompt_template = PromptTemplate(template=template, input_variables=["bar"], template_format="jinja2", validate_template=True) # Extra variable, should raise ValueError prompt_template = PromptTemplate(template=template, input_variables=["bar", "foo", "extra", "thing"], template_format="jinja2", validate_template=True) ```	1 year ago
Zander Chase	90ef705ced	Update Tool Input (#3103 ) - Remove dynamic model creation in the `args()` property. _Only infer for the decorator (and add an argument to NOT infer if someone wishes to only pass as a string)_ - Update the validation example to make it less likely to be misinterpreted as a "safe" way to run a repl There is one example of "Multi-argument tools" in the custom_tools.ipynb from yesterday, but we could add more. The output parsing for the base MRKL agent hasn't been adapted to handle structured args at this point in time --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	aad0a498ac	Harrison/output error (#3094 ) Co-authored-by: yummydum <sumita@nowcast.co.jp>	1 year ago
Harrison Chase	db968284f8	tools refactor (#2961 ) Co-authored-by: vowelparrot <130414180+vowelparrot@users.noreply.github.com>	1 year ago
engkheng	19febc77d6	Support inference of `input_variables` from `jinja2` template (#3013 ) `langchain.prompts.PromptTemplate` is unable to infer `input_variables` from jinja2 template. ```python # Using langchain v0.0.141 template_string = """\ Hello world Your variable: {{ var }} {# This will not get rendered #} {% if verbose %} Congrats! You just turned on verbose mode and got extra messages! {% endif %} """ template = PromptTemplate.from_template(template_string, template_format="jinja2") print(template.input_variables) # Output ['# This will not get rendered #', '% endif %', '% if verbose %'] ``` --------- Co-authored-by: engkheng <ongengkheng929@example.com>	1 year ago
Nuno Campos	dac32c59e5	Nc/combining output parser (#3014 ) Co-authored-by: vowelparrot <130414180+vowelparrot@users.noreply.github.com>	1 year ago
Davis Chase	19c85aa990	Factor out doc formatting and add validation (#3026 ) @cnhhoang850 slightly more generic fix for #2944, works for whatever the expected metadata keys are not just `source`	1 year ago
Naveen Tatikonda	3453b7457c	OpenSearch: Add Support for Boolean Filter with ANN search (#3038 ) ### Description Add Support for Boolean Filter with ANN search Documentation - https://opensearch.org/docs/latest/search-plugins/knn/filter-search-knn/#boolean-filter-with-ann-search ### Issues Resolved https://github.com/hwchase17/langchain/issues/2924 Signed-off-by: Naveen Tatikonda <navtat@amazon.com>	1 year ago
Harrison Chase	afd3e70ae5	Harrison/confluent loader (#2994 ) Co-authored-by: Justin Flick <Justinjayflick@gmail.com>	1 year ago
vowelparrot	99c0382209	Generative Characters (#2859 ) Add a time-weighted memory retriever and a notebook that approximates a Generative Agent from https://arxiv.org/pdf/2304.03442.pdf The "daily plan" components are removed for now since they are less useful without a virtual world, but the memory is an interesting component to build off. --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Jan Backes	a9310a3e8b	Add Annoy as VectorStore (#2939 ) Adds Annoy (https://github.com/spotify/annoy) as vector Store. RESOLVES hwchase17/langchain#2842 discord ref: https://discord.com/channels/1038097195422978059/1051632794427723827/1096089994168377354 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: vowelparrot <130414180+vowelparrot@users.noreply.github.com>	1 year ago
Harrison Chase	e12e00df12	use output parsers in agents (#2987 )	1 year ago
cs0lar	8b9e02da9d	Fix/issue 1213 (#2932 ) ### Background Continuing to implement all the interface methods defined by the `VectorStore` class. This PR pertains to implementation of the `max_marginal_relevance_search` method. ### Changes - a `max_marginal_relevance_search` method implementation has been added in `weaviate.py` - tests have been added to the the new method - vcr cassettes have been added for the weaviate tests ### Test Plan Added tests for the `max_marginal_relevance_search` implementation ### Change Safety - [x] I have added tests to cover my changes	1 year ago
vowelparrot	5ca7ce77cd	Remove pythonrepl from LLM-MathChain (#2943 ) Use numexpr evaluate instead of the python REPL to avoid malicious code injection. Tested against the (limited) math dataset and got the same score as before. For more permissive tools (like the REPL tool itself), other approaches ought to be provided (some combination of Sanitizer + Restricted python + unprivileged-docker + ...), but for a calculator tool, only mathematical expressions should be permitted. See https://github.com/hwchase17/langchain/issues/814	1 year ago
vowelparrot	4ffc58e07b	Add similarity_search_with_normalized_similarities (#2916 ) Add a method that exposes a similarity search with corresponding normalized similarity scores. Implement only for FAISS now. ### Motivation: Some memory definitions combine `relevance` with other scores, like recency , importance, etc. While many (but not all) of the `VectorStore`'s expose a `similarity_search_with_score` method, they don't all interpret the units of that score (depends on the distance metric and whether or not the the embeddings are normalized). This PR proposes a `similarity_search_with_normalized_similarities` method that lets consumers of the vector store not have to worry about the metric and embedding scale. Most providers default to euclidean distance, with Pinecone being one exception (defaults to cosine _similarity_). --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
dev2049	36aa7f30e4	Move PythonRepl -> langchain.utilities (#2917 )	1 year ago
Davit Buniatyan	b3a5b51728	[minor] Deep Lake auth improvements in docs, kwargs pass, faster tests (#2927 ) Minor cosmetic changes - Activeloop environment cred authentication in notebooks with `getpass.getpass` (instead of CLI which not always works) - much faster tests with Deep Lake pytest mode on - Deep Lake kwargs pass Notes - I put pytest environment creds inside `vectorstores/conftest.py`, but feel free to suggest a better location. For context, if I put in `test_deeplake.py`, `ruff` doesn't let me to set them before import deeplake --------- Co-authored-by: Davit Buniatyan <d@activeloop.ai>	1 year ago
Ankush Gola	ec59e9d886	Fix ChatAnthropic stop_sequences error (#2919 ) (#2920 ) Note to self: Always run integration tests, even on "that last minute change you thought would be safe" :) --------- Co-authored-by: Mike Lambert <mike.lambert@anthropic.com>	1 year ago
Mike Lambert	392f1b3218	Add Anthropic ChatModel to langchain (#2293 ) * Adds an Anthropic ChatModel * Factors out common code in our LLMModel and ChatModel * Supports streaming llm-tokens to the callbacks on a delta basis (until a future V2 API does that for us) * Some fixes	1 year ago
Harrison Chase	1e9378d0a8	Harrison/weaviate fixes (#2872 ) Co-authored-by: cs0lar <cristiano.solarino@gmail.com> Co-authored-by: cs0lar <cristiano.solarino@brightminded.com>	1 year ago
Harrison Chase	705596b46a	Harrison/fix create sql agent (#2870 ) Co-authored-by: Timothé Pearce <timothe.pearce@gmail.com>	1 year ago
sergerdn	04c458a270	feat: improve pinecone tests (#2806 ) Improve the integration tests for Pinecone by adding an `.env.example` file for local testing. Additionally, add some dev dependencies specifically for integration tests. This change also helps me understand how Pinecone deals with certain things, see related issues https://github.com/hwchase17/langchain/issues/2484 https://github.com/hwchase17/langchain/issues/2816	1 year ago
vowelparrot	bf0887c486	Add Slack Directory Loader (#2841 ) Fixes linting issue from #2835 Adds a loader for Slack Exports which can be a very valuable source of knowledge to use for internal QA bots and other use cases. ```py # Export data from your Slack Workspace first. from langchain.document_loaders import SLackDirectoryLoader SLACK_WORKSPACE_URL = "https://awesome.slack.com" loader = ("Slack_Exports", SLACK_WORKSPACE_URL) docs = loader.load() ```	1 year ago
Tim Asp	be4fb24b32	OpenAI LLM: update `modelname_to_contextsize` with new models (#2843 ) Token counts pulled from https://openai.com/pricing	1 year ago
KullTC	802363eb6a	Remove print statement from test (#2809 ) Remove unnecessary print statement.	1 year ago
KullTC	64596b23b9	Return output of PythonAstREPLTool when falling back to exec() (#2780 ) When the code ran by the PythonAstREPLTool contains multiple statements it will fallback to exec() instead of using eval(). With this change, it will also return the output of the code in the same way the PythonREPLTool will.	1 year ago
Harrison Chase	e49f1e628c	Harrison/gpt cache (#2744 ) Co-authored-by: SimFG <bang.fu@zilliz.com>	1 year ago
Joshua Snyder	59d054308c	Add type inference for output parsers (#2769 ) Currently, the output type of a number of OutputParser's `parse` methods is `Any` when it can in fact be inferred. This PR makes BaseOutputParser use a generic type and fixes the output types of the following parsers: - `PydanticOutputParser` - `OutputFixingParser` - `RetryOutputParser` - `RetryWithErrorOutputParser` The output of the `StructuredOutputParser` is corrected from `BaseModel` to `Any` since there are no type guarantees provided by the parser. Fixes issue #2715	1 year ago
Johnny Lee	0ab364404e	add continue to fix 'continue_on_failure' parameter for URL doc loader (#2735 ) Currently, the function still fails if `continue_on_failure` is set to True, because `elements` is not set. --------- Co-authored-by: leecjohnny <johnny-lee1255@users.noreply.github.com>	1 year ago
sergerdn	4bdcedab54	fix: some imports for integration tests (#2612 ) Add more missed imports for integration tests. Bump `pytest` to the current latest version. Fix `tests/integration_tests/vectorstores/test_elasticsearch.py` to update its cassette(easy fix). Related PR: https://github.com/hwchase17/langchain/pull/2560	1 year ago
Ankush Gola	c1521ddbdb	Add workaround for not having async vector store methods (#2733 ) This allows us to use the async API for the Retrieval chains, though it is not guaranteed to be thread safe.	1 year ago
vowelparrot	709f26b69e	Added bilibili loader (#2673 ) (#2724 ) I've added a bilibili loader, bilibili is a very active video site in China and I think we need this loader. Example: ```python from langchain.document_loaders.bilibili import BiliBiliLoader loader = BiliBiliLoader( ["https://www.bilibili.com/video/BV1xt411o7Xu/", "https://www.bilibili.com/video/av330407025/"] ) docs = loader.load() ``` Co-authored-by: 了空 <568250549@qq.com>	1 year ago
Abhik Singla	955bd2e1db	Fixed Ast Python Repl for Chatgpt multiline commands (#2406 ) Resolves issue https://github.com/hwchase17/langchain/issues/2252 --------- Co-authored-by: Abhik Singla <abhiksingla@microsoft.com>	1 year ago
Naveen Tatikonda	4364d3316e	Add custom vector fields and text fields for OpenSearch (#2652 ) Description Add custom vector field name and text field name while indexing and querying for OpenSearch Issues https://github.com/hwchase17/langchain/issues/2500 Signed-off-by: Naveen Tatikonda <navtat@amazon.com>	1 year ago
Ankush Gola	b82cbd1be0	Use `run` and `arun` in place of `combine_docs` and `acombine_docs` (#2635 ) `combine_docs` does not go through the standard chain call path which means that chain callbacks won't be triggered, meaning QA chains won't be traced properly, this fixes that. Also fix several errors in the chat_vector_db notebook	1 year ago
Chetanya Rastogi	50c511d75f	Add new loader to load pdf as html content (#2607 ) Adds a new pdf loader using the existing dependency on PDFMiner. The new loader can be helpful for chunking texts semantically into sections as the output html content can be parsed via `BeautifulSoup` to get more structured and rich information about font size, page numbers, pdf headers/footers, etc. which may not be available otherwise with other pdf loaders	1 year ago
sergerdn	cd9336469e	fix: missed deps integrations tests (#2560 ) Almost all integration tests have failed, but we haven't encountered any import errors yet. Some tests failed due to lazy import issues. It doesn't seem like a problem to resolve some of these errors in the next PR. I have a headache from resolving conflicts with `deeplake` and `boto3`, so I will temporarily comment out `boto3`. fix https://github.com/hwchase17/langchain/issues/2426	1 year ago
sergerdn	6dc86ad48f	feat: add pytest-vcr for recording HTTP interactions in integration tests (#2445 ) Using `pytest-vcr` in integration tests has several benefits. Firstly, it removes the need to mock external services, as VCR records and replays HTTP interactions on the fly. Secondly, it simplifies the integration test setup by eliminating the need to set up and tear down external services in some cases. Finally, it allows for more reliable and deterministic integration tests by ensuring that HTTP interactions are always replayed with the same response. Overall, `pytest-vcr` is a valuable tool for simplifying integration test setup and improving their reliability This commit adds the `pytest-vcr` package as a dependency for integration tests in the `pyproject.toml` file. It also introduces two new fixtures in `tests/integration_tests/conftest.py` files for managing cassette directories and VCR configurations. In addition, the `tests/integration_tests/vectorstores/test_elasticsearch.py` file has been updated to use the `@pytest.mark.vcr` decorator for recording and replaying HTTP interactions. Finally, this commit removes the `documents` fixture from the `test_elasticsearch.py` file and replaces it with a new fixture defined in `tests/integration_tests/vectorstores/conftest.py` that yields a list of documents to use in any other tests. This also includes my second attempt to fix issue : https://github.com/hwchase17/langchain/issues/2386 Maybe related https://github.com/hwchase17/langchain/issues/2484	1 year ago
Vashisht Madhavan	aa439ac2ff	Adding an in-context QA evaluation chain + chain of thought reasoning chain for improved accuracy (#2444 ) Right now, eval chains require an answer for every question. It's cumbersome to collect this ground truth so getting around this issue with 2 things: * Adding a context param in `ContextQAEvalChain` and simply evaluating if the question is answered accurately from context * Adding chain of though explanation prompting to improve the accuracy of this w/o GT. This also gets to feature parity with openai/evals which has the same contextual eval w/o GT. TODO in follow-up: * Better prompt inheritance. No need for seperate prompt for CoT reasoning. How can we merge them together --------- Co-authored-by: Vashisht Madhavan <vashishtmadhavan@Vashs-MacBook-Pro.local>	1 year ago
Alex Rad	bd780a8223	Add support for rwkv (#2422 ) This adds support for running RWKV with pytorch. https://github.com/hwchase17/langchain/issues/2398 This does not yet support rwkv.cpp	1 year ago
William FH	f240651bd8	Add Request body (#2507 ) This still doesn't handle the following - non-JSON media types - anyOf, allOf, oneOf's And doesn't emit the typescript definitions for referred types yet, but that can be saved for a separate PR. Also, we could have better support for Swagger 2.0 specs and OpenAPI 3.0.3 (can use the same lib for the latter) recommend offline conversion for now.	1 year ago
Zach Jones	13d1df2140	Feature: AgentExecutor execution time limit (#2399 ) `AgentExecutor` already has support for limiting the number of iterations. But the amount of time taken for each iteration can vary quite a bit, so it is difficult to place limits on the execution time. This PR adds a new field `max_execution_time` to the `AgentExecutor` model. When called asynchronously, the agent loop is wrapped in an `asyncio.timeout()` context which triggers the early stopping response if the time limit is reached. When called synchronously, the agent loop checks for both the max_iteration limit and the time limit after each iteration. When used asynchronously `max_execution_time` gives really tight control over the max time for an execution chain. When used synchronously, the chain can unfortunately exceed max_execution_time, but it still gives more control than trying to estimate the number of max_iterations needed to cap the execution time. --------- Co-authored-by: Zachary Jones <zjones@zetaglobal.com>	1 year ago
Davit Buniatyan	b4914888a7	Deep Lake upgrade to include attribute search, distance metrics, returning scores and MMR (#2455 ) ### Features include - Metadata based embedding search - Choice of distance metric function (`L2` for Euclidean, `L1` for Nuclear, `max` L-infinity distance, `cos` for cosine similarity, 'dot' for dot product. Defaults to `L2` - Returning scores - Max Marginal Relevance Search - Deleting samples from the dataset ### Notes - Added numerous tests, let me know if you would like to shorten them or make smarter --------- Co-authored-by: Davit Buniatyan <d@activeloop.ai>	1 year ago
leo-gan	fd69cc7e42	Removed duplicate BaseModel dependencies (#2471 ) Removed duplicate BaseModel dependencies in class inheritances. Also, sorted imports by `isort`.	1 year ago
Harrison Chase	1e19e004af	Harrison/openapi spec (#2474 ) Co-authored-by: William Fu-Hinthorn <13333726+hinthornw@users.noreply.github.com>	1 year ago
Harrison Chase	26314d7004	Harrison/openapi parser (#2461 ) Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>	1 year ago
William FH	007babb363	Add a mock server (#2443 ) It's useful to evaluate API Chains against a mock server. This PR makes an example "robot" server that exposes endpoints for the following: - Path, Query, and Request Body argument passing - GET, PUT, and DELETE endpoints exposed OpenAPI spec. Relies on FastAPI + Uvicorn - I could add to the dev dependencies list if you'd like	1 year ago
sergerdn	b410dc76aa	fix: elasticsearch (#2402 ) - Create a new docker-compose file to start an Elasticsearch instance for integration tests. - Add new tests to `test_elasticsearch.py` to verify Elasticsearch functionality. - Include an optional group `test_integration` in the `pyproject.toml` file. This group should contain dependencies for integration tests and can be installed using the command `poetry install --with test_integration`. Any new dependencies should be added by running `poetry add some_new_deps --group "test_integration" ` Note: New tests running in live mode, which involve end-to-end testing of the OpenAI API. In the future, adding `pytest-vcr` to record and replay all API requests would be a nice feature for testing process.More info: https://pytest-vcr.readthedocs.io/en/latest/ Fixes https://github.com/hwchase17/langchain/issues/2386	1 year ago
Ankush Gola	4d730a9bbc	improve `AsyncCallbackManager` (#2410 )	1 year ago
Harrison Chase	c7b083ab56	bump version to 131 (#2391 )	1 year ago
Harrison Chase	0a9f04bad9	Harrison/gpt4all (#2366 ) Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com> Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	1 year ago
Harrison Chase	e90d007db3	Harrison/msg files (#2375 ) Co-authored-by: Sahil Masand <masand.sahil@gmail.com> Co-authored-by: Sahil Masand <masands@cbh.com.au>	1 year ago
Kacper Łukawski	585f60a5aa	Qdrant update to 1.1.1 & docs polishing (#2388 ) This PR updates Qdrant to 1.1.1 and introduces local mode, so there is no need to spin up the Qdrant server. By that occasion, the Qdrant example notebooks also got updated, covering more cases and answering some commonly asked questions. All the Qdrant's integration tests were switched to local mode, so no Docker container is required to launch them.	1 year ago
Harrison Chase	fe1eb8ca5f	requests wrapper (#2367 )	1 year ago
Shrined	10dab053b4	Add Enum for agent types (#2321 ) This pull request adds an enum class for the various types of agents used in the project, located in the `agent_types.py` file. Currently, the project is using hardcoded strings for the initialization of these agents, which can lead to errors and make the code harder to maintain. With the introduction of the new enums, the code will be more readable and less error-prone. The new enum members include: - ZERO_SHOT_REACT_DESCRIPTION - REACT_DOCSTORE - SELF_ASK_WITH_SEARCH - CONVERSATIONAL_REACT_DESCRIPTION - CHAT_ZERO_SHOT_REACT_DESCRIPTION - CHAT_CONVERSATIONAL_REACT_DESCRIPTION In this PR, I have also replaced the hardcoded strings with the appropriate enum members throughout the codebase, ensuring a smooth transition to the new approach.	1 year ago
Harrison Chase	d85f57ef9c	Harrison/llama (#2314 ) Co-authored-by: RJ Adriaansen <adriaansen@eshcc.eur.nl>	1 year ago
akmhmgc	94b2f536f3	Modify output for wikipedia api wrapper (#2287 ) ## Description Thanks for the quick maintenance for great repository!! I modified wikipedia api wrapper ## Details - Add output for missing search results - Add tests	1 year ago
Harrison Chase	acfda4d1d8	Harrison/multiline commands (#2280 ) Co-authored-by: Marc Päpper <mpaepper@users.noreply.github.com>	1 year ago
leo-gan	579ad85785	skip unit tests that fail in Windows (#2238 ) Issue #2174 Several unit tests fail in Windows. Added pytest attribute to skip these tests automatically.	1 year ago
Sam Cordner-Matthews	1ddd6dbf0b	Add ability to pass kwargs to loader classes in `DirectoryLoader`, add ability to modify encoding and BeautifulSoup behaviour in `BSHTMLLoader` (#2275 ) Solves #2247. Noted that the only test I added checks for the BeautifulSoup behaviour change. Happy to add a test for `DirectoryLoader` if deemed necessary.	1 year ago
Harrison Chase	2d3918c152	make requests more general (#2209 )	1 year ago
Harrison Chase	5c907d9998	Harrison/base agent without docs (#2166 )	1 year ago
Harrison Chase	f5a4bf0ce4	remove prep (#2136 ) agents should be stateless or async stuff may not work	1 year ago
Harrison Chase	b35260ed47	Harrison/memory base (#2122 ) @3coins + @zoltan-fedor.... heres the pr + some minor changes i made. thoguhts? can try to get it into tmrws release --------- Co-authored-by: Zoltan Fedor <zoltan.0.fedor@gmail.com> Co-authored-by: Piyush Jain <piyushjain@duck.com>	1 year ago
Ankush Gola	ccee1aedd2	add async support for anthropic (#2114 ) should not be merged in before https://github.com/anthropics/anthropic-sdk-python/pull/11 gets released	1 year ago
Harrison Chase	e2c26909f2	Harrison/memory check (#2119 ) Co-authored-by: JIAQIA <jqq1716@gmail.com>	1 year ago
Harrison Chase	3e879b47c1	Harrison/gitbook (#2044 ) Co-authored-by: Irene López <45119610+ireneisdoomed@users.noreply.github.com>	1 year ago
Honkware	aff33d52c5	Add OpenWeatherMap API Tool (#2083 ) Added tool for OpenWeatherMap API	1 year ago
Charlie Holtz	f16c1fb6df	Add replicate take 2 (#2077 ) This PR adds a replicate integration to langchain. It's an updated version of https://github.com/hwchase17/langchain/pull/1993, but with updates to match latest replicate-python code. https://github.com/replicate/replicate-python. --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Zeke Sikelianos <zeke@sikelianos.com>	1 year ago
Harrison Chase	f281033362	rm pandas dependency (#2102 )	1 year ago
Harrison Chase	410bf37fb8	Harrison/big query (#2100 ) Co-authored-by: lu-cashmoney <lucas.corley@gmail.com>	1 year ago
Harrison Chase	eff5eed719	Harrison/jina (#2043 ) Co-authored-by: numb3r3 <wangfelix87@gmail.com> Co-authored-by: felix-wang <35718120+numb3r3@users.noreply.github.com>	1 year ago
Harrison Chase	9e74df2404	Fix issue#1645: Parse llm_output even there's newline (#2092 ) (#2099 ) Fix issue#1645: Parse either whitespace or newline after 'Action Input:' in llm_output in mrkl agent. Unittests added accordingly. Co-authored-by: ₿ingnan.ΞTH <brillliantz@outlook.com>	1 year ago
blob42	b7f392fdd6	[agent_executor] convenience func: lookup tool by name (#2001 ) A quick convenience function to lookup a tool by name Co-authored-by: blob42 <spike@w530>	1 year ago
Harrison Chase	f74a1bebf5	Harrison/duckdb (#2064 ) Co-authored-by: Trent Hauck <trent@trenthauck.com>	1 year ago
Ankush Gola	b7ebb8fe30	enable streaming in anthropic llm wrapper (#2065 )	1 year ago
Harrison Chase	30e3b31b04	Harrison/document cleanup (#2062 ) Co-authored-by: Delip Rao <delip@users.noreply.github.com>	1 year ago
Harrison Chase	a0cd6672aa	Harrison/site map (#2061 ) Co-authored-by: Tim Asp <707699+timothyasp@users.noreply.github.com>	1 year ago
Daniel Chalef	6598beacdb	PydanticOutputParser unit test (#2047 ) Unit test for PydanticOutputParser --------- Co-authored-by: Daniel Chalef <daniel.chalef@private.org>	1 year ago
Harrison Chase	705431aecc	big docs refactor (#1978 ) Co-authored-by: Ankush Gola <ankush.gola@gmail.com>	1 year ago
Mario Kostelac	e7d6de6b1c	(ChatOpenAI) Add model_name to LLMResult.llm_output (#1960 ) This makes sure OpenAI and ChatOpenAI have the same llm_output, and allow tracking usage per model. Same work for OpenAI was done in https://github.com/hwchase17/langchain/pull/1713.	1 year ago
Harrison Chase	6e1b5b8f7e	Harrison/figma doc loader (#1908 ) Co-authored-by: Ismail Pelaseyed <homanp@gmail.com>	1 year ago
Eli	12f868b292	Propagate "filter" arg in Chroma similarity_search (#1869 ) Technically a duplicate fix to #1619 but with unit tests and a small documentation update - Propagate `filter` arg in Chroma `similarity_search` to delegated call to `similarity_search_with_score` - Add `filter` arg to `similarity_search_by_vector` - Clarify doc strings on FakeEmbeddings	1 year ago
Maurício Maia	f155d9d3ec	Add metadata filter to PGVector search (#1872 ) Add ability to filter pgvector documents by metadata.	1 year ago
Maurício Maia	2212520a6c	Add PGVector collection metadata (#1887 ) The `CollectionStore` for `PGVector` has a `cmetadata` field but it's never used. This PR add the ability to save metadata information to the collection.	1 year ago
Harrison Chase	ce5d97bcb3	Harrison/guarded output parser (#1804 ) Co-authored-by: jerwelborn <jeremy.welborn@gmail.com>	1 year ago
Matt Tucker	a92344f476	Use regex match for bash process error output test assertion. (#1837 ) I was getting the same issue reported in #1339 by [MacYang555](https://github.com/MacYang555) when running the test suite on my Mac. I implemented the fix they suggested to use a regex match in the output assertion for the scenario under test. Resolves #1339	1 year ago
LeoGrin	3701b2901e	use namespace argument in Pinecone constructor (#1757 ) Fix #1756 Use the `namespace` argument of `Pinecone.from_exisiting_index` to set the default value of `namespace` for other methods. Leads to more expected behavior and easier integration in chains. For the test, I've added a line to delete and rebuild the `langchain-demo` index at the beginning of the test. I'm not 100% sure if it's a good idea but it makes the test reproducible.	1 year ago
Mario Kostelac	aff44d0a98	(OpenAI) Add model_name to LLMResult.llm_output (#1713 ) Given that different models have very different latencies and pricings, it's benefitial to pass the information about the model that generated the response. Such information allows implementing custom callback managers and track usage and price per model. Addresses https://github.com/hwchase17/langchain/issues/1557.	1 year ago
Daniel Chalef	b157e0c1c3	Add HTML document_loader that includes page title metadata (#1720 ) This `BSHTMLLoader` document_loader loads an HTML document, extracts text and adds the page title to the returned Document's metadata. The loader uses the already installed bs4 package to extract both text content and the page title. Included in this PR is an example HTML file and an integration test that tests against this file. --------- Co-authored-by: Daniel Chalef <daniel.chalef@private.org>	1 year ago
Kacper Łukawski	4a327dd1d6	Implement basic metadata filtering in Qdrant (#1689 ) This PR implements a basic metadata filtering mechanism similar to the ones in Chroma and Pinecone. It still cannot express complex conditions, as there are no operators, but some users requested to have that feature available.	1 year ago
Harrison Chase	0b29e68c17	Harrison/pgvector (#1679 ) Co-authored-by: Aman Kumar <krsingh.aman@gmail.com>	1 year ago
Xin Qiu	4e13cef05a	feat: add redisearch vectorstore (#1307 ) # Description Add `RediSearch` vectorstore for LangChain RediSearch: [RediSearch quick start](https://redis.io/docs/stack/search/quick_start/) # How to use ``` from langchain.vectorstores.redisearch import RediSearch rds = RediSearch.from_documents(docs, embeddings,redisearch_url="redis://localhost:6379") ```	1 year ago
Jon Luo	0a1b1806e9	sql: do not hard code the LIMIT clause in the table_info section (#1563 ) Seeing a lot of issues in Discord in which the LLM is not using the correct LIMIT clause for different SQL dialects. ie, it's using `LIMIT` for mssql instead of `TOP`, or instead of `ROWNUM` for Oracle, etc. I think this could be due to us specifying the LIMIT statement in the example rows portion of `table_info`. So the LLM is seeing the `LIMIT` statement used in the prompt. Since we can't specify each dialect's method here, I think it's fine to just replace the `SELECT... LIMIT 3;` statement with `3 rows from table_name table:`, and wrap everything in a block comment directly following the `CREATE` statement. The Rajkumar et al paper wrapped the example rows and `SELECT` statement in a block comment as well anyway. Thoughts @fpingham?	1 year ago
Tim Asp	b3234bf3b0	cleanup: unify 3 different pdf loaders, rename PagedPDFSplitter (#1615 ) `OnlinePDFLoader` and `PagedPDFSplitter` lived separate from the rest of the pdf loaders. Because they're all similar, I propose moving all to `pdy.py` and the same docs/examples page. Additionally, `PagedPDFSplitter` naming doesn't match the pattern the rest of the loaders follow, so I renamed to `PyPDFLoader` and had it inherit from `BasePDFLoader` so it can now load from remote file sources.	1 year ago
Luis	562d9891ea	Add regex dict: (#1616 ) This class enables us to send a dictionary containing an output key and the expected format, which in turn allows us to retrieve the result of the matching formats and extract specific information from it. To exclude irrelevant information from our return dictionary, we can prompt the LLM to use a specific command that notifies us when it doesn't know the answer. We refer to this variable as the "no_update_value". Regarding the updated regular expression pattern (r"{}:\s?([^.'\n']).?"), it enables us to retrieve a format as 'Output Key':'value'. We have improved the regex by adding an optional space between ':' and 'value' with "s?", and by excluding points and line jumps from the matches using "[^.'\n']".	1 year ago
Harrison Chase	aed9f9febe	Harrison/return intermediate (#1633 ) Co-authored-by: Mario Kostelac <mario@intercom.io>	1 year ago
yakigac	acd86d33bc	Add read only shared memory (#1491 ) Provide shared memory capability for the Agent. Inspired by #1293 . ## Problem If both Agent and Tools (i.e., LLMChain) use the same memory, both of them will save the context. It can be annoying in some cases. ## Solution Create a memory wrapper that ignores the save and clear, thereby preventing updates from Agent or Tools.	1 year ago
Harrison Chase	c9b5a30b37	move output parsing (#1605 )	1 year ago
Harrison Chase	cb04ba0136	Add support for intermediate steps to SQLDatabaseSequentialChain (#1583 ) (#1601 ) for https://github.com/hwchase17/langchain/issues/1582 I simply added the `return_intermediate_steps` and changed the `output_keys` function. I added 2 simple tests, 1 for SQLDatabaseSequentialChain without the intermediate steps and 1 with Co-authored-by: brad-nemetski <115185478+brad-nemetski@users.noreply.github.com>	1 year ago
Harrison Chase	f95d551f7a	Harrison/shallow metadata (#1599 ) Co-authored-by: Jesse Zhang <jessetanzhang@gmail.com>	1 year ago
Harrison Chase	9f78717b3c	Harrison/callbacks (#1587 )	1 year ago
Harrison Chase	3ee32a01ea	Harrison/prompt layer (#1547 ) Co-authored-by: Jonathan Pedoeem <jonathanped@gmail.com> Co-authored-by: AbuBakar <abubakarsohail123@gmail.com>	1 year ago
Harrison Chase	c844d1fd46	Harrison/chunk size (#1549 ) Co-authored-by: Florian Leuerer <31259070+floleuerer@users.noreply.github.com>	1 year ago
Harrison Chase	357d808484	Harrison/remote paths pdf (#1544 ) Co-authored-by: Tim Asp <707699+timothyasp@users.noreply.github.com>	1 year ago
Harrison Chase	cc423f40f1	Harrison/youtube loader (#1545 ) Co-authored-by: Julian Wustl <57504258+Julianwustl@users.noreply.github.com>	1 year ago
Harrison Chase	7ade419a0e	allow passing of messages into prompt template (#1505 )	1 year ago
Harrison Chase	064741db58	Harrison/fix text splitter (#1511 ) Co-authored-by: ajaysolanky <ajsolanky@gmail.com> Co-authored-by: Ajay Solanky <ajaysolanky@saw-l14668307kd.myfiosgateway.com>	1 year ago
Ankush Gola	27104d4921	fix `ChatOpenAI.agenerate` (#1504 )	1 year ago
Harrison Chase	7bec461782	Harrison/memory refactor (#1478 ) moves memory to own module, factors out common stuff	1 year ago
Harrison Chase	0e21463f07	(rfc) chat models (#1424 ) Co-authored-by: Ankush Gola <ankush.gola@gmail.com>	1 year ago
Harrison Chase	63a5614d23	Harrison/simple memory (#1435 ) Co-authored-by: Tim Asp <707699+timothyasp@users.noreply.github.com>	2 years ago
Harrison Chase	a1b9dfc099	Harrison/similarity search chroma (#1434 ) Co-authored-by: shibuiwilliam <shibuiyusuke@gmail.com>	2 years ago
Tim Asp	23231d65a9	Add PyMuPDF PDF loader (#1426 ) Different PDF libraries have different strengths and weaknesses. PyMuPDF does a good job at extracting the most amount of content from the doc, regardless of the source quality, extremely fast (especially compared to Unstructured). https://pymupdf.readthedocs.io/en/latest/index.html	2 years ago
Nuno Campos	499e76b199	Allow the regular openai class to be used for ChatGPT models (#1393 ) Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	2 years ago
Kacper Łukawski	9ac442624c	Add Qdrant named arguments (#1386 ) This PR: - Increases `qdrant-client` version to 1.0.4 - Introduces custom content and metadata keys (as requested in #1087) - Moves all the `QdrantClient` parameters into the method parameters to simplify code completion	2 years ago
Ankush Gola	fe30be6fba	add async and streaming support to `OpenAIChat` (#1378 ) title says it all	2 years ago
Harrison Chase	1cd8996074	Harrison/summarizer chain (#1356 ) Co-authored-by: Tim Asp <707699+timothyasp@users.noreply.github.com>	2 years ago
Ankush Gola	82baecc892	Add a SQL agent for interacting with SQL Databases and JSON Agent for interacting with large JSON blobs (#1150 ) This PR adds * `ZeroShotAgent.as_sql_agent`, which returns an agent for interacting with a sql database. This builds off of `SQLDatabaseChain`. The main advantages are 1) answering general questions about the db, 2) access to a tool for double checking queries, and 3) recovering from errors * `ZeroShotAgent.as_json_agent` which returns an agent for interacting with json blobs. * Several examples in notebooks --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	2 years ago
Harrison Chase	786852e9e6	partial variables (#1308 )	2 years ago
Tim Asp	72ef69d1ba	Add new iFixit document loader (#1333 ) iFixit is a wikipedia-like site that has a huge amount of open content on how to fix things, questions/answers for common troubleshooting and "things" related content that is more technical in nature. All content is licensed under CC-BY-SA-NC 3.0 Adding docs from iFixit as context for user questions like "I dropped my phone in water, what do I do?" or "My macbook pro is making a whining noise, what's wrong with it?" can yield significantly better responses than context free response from LLMs.	2 years ago
Harrison Chase	166cda2cc6	Harrison/deeplake (#1316 ) Co-authored-by: Davit Buniatyan <d@activeloop.ai>	2 years ago
Harrison Chase	aaad6cc954	Harrison/atlas db (#1315 ) Co-authored-by: Brandon Duderstadt <brandonduderstadt@gmail.com>	2 years ago
Enrico Shippole	9becdeaadf	Add Writer, Banana, Modal, StochasticAI (#1270 ) Add LLM wrappers and examples for Banana, Writer, Modal, Stochastic AI Added rigid json format for Banana and Modal	2 years ago
Dennis Antela Martinez	53c67e04d4	add aleph alpha llm (#1207 ) Integrate Aleph Alpha's client into Langchain to provide access to the luminous models - more info on latest benchmarks here: https://www.aleph-alpha.com/luminous-performance-benchmarks	2 years ago
Harrison Chase	b7708bbec6	rfc: callback changes (#1165 ) conceptually, no reason a tool should know what an "agent action" is unless any objections, can change in all callback handlers	2 years ago
Harrison Chase	44c8d8a9ac	move serpapi wrapper (#1199 ) Co-authored-by: Tim Asp <707699+timothyasp@users.noreply.github.com>	2 years ago
Naveen Tatikonda	0118706fd6	Add Support for OpenSearch Vector database (#1191 ) ### Description This PR adds a wrapper which adds support for the OpenSearch vector database. Using opensearch-py client we are ingesting the embeddings of given text into opensearch cluster using Bulk API. We can perform the `similarity_search` on the index using the 3 popular searching methods of OpenSearch k-NN plugin: - `Approximate k-NN Search` use approximate nearest neighbor (ANN) algorithms from the [nmslib](https://github.com/nmslib/nmslib), [faiss](https://github.com/facebookresearch/faiss), and [Lucene](https://lucene.apache.org/) libraries to power k-NN search. - `Script Scoring` extends OpenSearch’s script scoring functionality to execute a brute force, exact k-NN search. - `Painless Scripting` adds the distance functions as painless extensions that can be used in more complex combinations. Also, supports brute force, exact k-NN search like Script Scoring. ### Issues Resolved https://github.com/hwchase17/langchain/issues/1054 --------- Signed-off-by: Naveen Tatikonda <navtat@amazon.com>	2 years ago
Andrew White	c5015d77e2	Allow k to be higher than doc size in max_marginal_relevance_search (#1187 ) Fixes issue #1186. For some reason, #1117 didn't seem to fix it.	2 years ago
Harrison Chase	9d6d8f85da	Harrison/self hosted runhouse (#1154 ) Co-authored-by: Donny Greenberg <dongreenberg2@gmail.com> Co-authored-by: John Dagdelen <jdagdelen@users.noreply.github.com> Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net> Co-authored-by: Andrew White <white.d.andrew@gmail.com> Co-authored-by: Peng Qu <82029664+pengqu123@users.noreply.github.com> Co-authored-by: Matt Robinson <mthw.wm.robinson@gmail.com> Co-authored-by: jeff <tangj1122@gmail.com> Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MacBook-Pro.local> Co-authored-by: zanderchase <zander@unfold.ag> Co-authored-by: Charles Frye <cfrye59@gmail.com> Co-authored-by: zanderchase <zanderchase@gmail.com> Co-authored-by: Shahriar Tajbakhsh <sh.tajbakhsh@gmail.com> Co-authored-by: Stefan Keselj <skeselj@princeton.edu> Co-authored-by: Francisco Ingham <fpingham@gmail.com> Co-authored-by: Dhruv Anand <105786647+dhruv-anand-aintech@users.noreply.github.com> Co-authored-by: cragwolfe <cragcw@gmail.com> Co-authored-by: Anton Troynikov <atroyn@users.noreply.github.com> Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com> Co-authored-by: Oliver Klingefjord <oliver@klingefjord.com> Co-authored-by: blob42 <contact@blob42.xyz> Co-authored-by: blob42 <spike@w530> Co-authored-by: Enrico Shippole <henryshippole@gmail.com> Co-authored-by: Ibis Prevedello <ibiscp@gmail.com> Co-authored-by: jped <jonathanped@gmail.com> Co-authored-by: Justin Torre <justintorre75@gmail.com> Co-authored-by: Ivan Vendrov <ivan@anthropic.com> Co-authored-by: Sasmitha Manathunga <70096033+mmz-001@users.noreply.github.com> Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com> Co-authored-by: Matt Robinson <mrobinson@unstructuredai.io> Co-authored-by: Jeff Huber <jeffchuber@gmail.com> Co-authored-by: Akshay <64036106+akshayvkt@users.noreply.github.com> Co-authored-by: Andrew Huang <jhuang16888@gmail.com> Co-authored-by: rogerserper <124558887+rogerserper@users.noreply.github.com> Co-authored-by: seanaedmiston <seane999@gmail.com> Co-authored-by: Hasegawa Yuya <52068175+Hase-U@users.noreply.github.com> Co-authored-by: Ivan Vendrov <ivendrov@gmail.com> Co-authored-by: Chen Wu (吴尘) <henrychenwu@cmu.edu> Co-authored-by: Dennis Antela Martinez <dennis.antela@gmail.com> Co-authored-by: Maxime Vidal <max.vidal@hotmail.fr> Co-authored-by: Rishabh Raizada <110235735+rishabh-ti@users.noreply.github.com>	2 years ago
CG80499	af8f5c1a49	Added constitutional chain. (#1147 ) - Added self-critique constitutional chain based on this [paper](https://www.anthropic.com/constitutional.pdf).	2 years ago
Ankush Gola	7b5e160d28	Make Tools own model, add ToolKit Concept (#1095 ) Follow-up of @hinthornw's PR: - Migrate the Tool abstraction to a separate file (`BaseTool`). - `Tool` implementation of `BaseTool` takes in function and coroutine to more easily maintain backwards compatibility - Add a Toolkit abstraction that can own the generation of tools around a shared concept or state --------- Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Francisco Ingham <fpingham@gmail.com> Co-authored-by: Dhruv Anand <105786647+dhruv-anand-aintech@users.noreply.github.com> Co-authored-by: cragwolfe <cragcw@gmail.com> Co-authored-by: Anton Troynikov <atroyn@users.noreply.github.com> Co-authored-by: Oliver Klingefjord <oliver@klingefjord.com> Co-authored-by: William Fu-Hinthorn <whinthorn@Williams-MBP-3.attlocal.net> Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>	2 years ago
Francisco Ingham	3f29742adc	Sql alchemy commands used in table info (#1135 ) This approach has several advantages: * it improves the readability of the code * removes incompatibilities between SQL dialects * fixes a bug with `datetime` values in rows and `ast.literal_eval` Huge thanks and credits to @jzluo for finding the weaknesses in the current approach and for the thoughtful discussion on the best way to implement this. --------- Co-authored-by: Francisco Ingham <> Co-authored-by: Jon Luo <20971593+jzluo@users.noreply.github.com>	2 years ago
Noah Gundotra	8c5fbab72d	[Integration Tests] Cast fake embeddings to ALL float values (#1102 ) Pydantic validation breaks tests for example (`test_qdrant.py`) because fake embeddings contain an integer. This PR casts the embeddings array to all floats. Now the `qdrant` test passes, `poetry run pytest tests/integration_tests/vectorstores/test_qdrant.py`	2 years ago
yakigac	1ed708391e	Fix a bug that shows "KeyError 'items'" (#1118 ) Fix KeyError 'items' when no result found. ## Problem When no result found for a query, google search crashed with `KeyError 'items'`. ## Solution I added a check for an empty response before accessing the 'items' key. It will handle the case correctly. ## Other my twitter: yakigac (I don't mind even if you don't mention me for this PR. But just because last time my real name was shout out :) )	2 years ago
Harrison Chase	5e10e19bfe	Harrison/align table (#1081 ) Co-authored-by: Francisco Ingham <fpingham@gmail.com>	2 years ago
Hasegawa Yuya	e08961ab25	Fixed openai embeddings to be safe by batching them based on token size calculation. (#991 ) I modified the logic of the batch calculation for embedding according to this cookbook https://github.com/openai/openai-cookbook/blob/main/examples/Embedding_long_inputs.ipynb	2 years ago
seanaedmiston	f0a258555b	Support similarity search by vector (in FAISS) (#961 ) Alternate implementation to PR #960 Again - only FAISS is implemented. If accepted can add this to other vectorstores or leave as NotImplemented? Suggestions welcome...	2 years ago
rogerserper	e46cd3b7db	Google Search API integration with serper.dev (wrapper, tests, docs, … (#909 ) Adds Google Search integration with [Serper](https://serper.dev) a low-cost alternative to SerpAPI (10x cheaper + generous free tier). Includes documentation, tests and examples. Hopefully I am not missing anything. Developers can sign up for a free account at [serper.dev](https://serper.dev) and obtain an api key. ## Usage ```python from langchain.utilities import GoogleSerperAPIWrapper from langchain.llms.openai import OpenAI from langchain.agents import initialize_agent, Tool import os os.environ["SERPER_API_KEY"] = "" os.environ['OPENAI_API_KEY'] = "" llm = OpenAI(temperature=0) search = GoogleSerperAPIWrapper() tools = [ Tool( name="Intermediate Answer", func=search.run ) ] self_ask_with_search = initialize_agent(tools, llm, agent="self-ask-with-search", verbose=True) self_ask_with_search.run("What is the hometown of the reigning men's U.S. Open champion?") ``` ### Output ``` Entering new AgentExecutor chain... Yes. Follow up: Who is the reigning men's U.S. Open champion? Intermediate answer: Current champions Carlos Alcaraz, 2022 men's singles champion. Follow up: Where is Carlos Alcaraz from? Intermediate answer: El Palmar, Spain So the final answer is: El Palmar, Spain > Finished chain. 'El Palmar, Spain' ```	2 years ago
Ankush Gola	caa8e4742e	Enable streaming for OpenAI LLM (#986 ) * Support a callback `on_llm_new_token` that users can implement when `OpenAI.streaming` is set to `True`	2 years ago
Harrison Chase	88bebb4caa	Harrison/llm integrations (#1039 ) Co-authored-by: jped <jonathanped@gmail.com> Co-authored-by: Justin Torre <justintorre75@gmail.com> Co-authored-by: Ivan Vendrov <ivan@anthropic.com>	2 years ago
Harrison Chase	ec727bf166	Align table info (#999 ) (#1034 ) Currently the chain is getting the column names and types on the one side and the example rows on the other. It is easier for the llm to read the table information if the column name and examples are shown together so that it can easily understand to which columns do the examples refer to. For an instantiation of this, please refer to the changes in the `sqlite.ipynb` notebook. Also changed `eval` for `ast.literal_eval` when interpreting the results from the sample row query since it is a better practice. --------- Co-authored-by: Francisco Ingham <> --------- Co-authored-by: Francisco Ingham <fpingham@gmail.com>	2 years ago
Enrico Shippole	f30dcc6359	Add GooseAI, CerebriumAI, Petals, ForefrontAI (#981 ) Add GooseAI, CerebriumAI, Petals, ForefrontAI	2 years ago
Anton Troynikov	d43d430d86	Chroma persistence (#1028 ) This PR adds persistence to the Chroma vector store. Users can supply a `persist_directory` with any of the `Chroma` creation methods. If supplied, the store will be automatically persisted at that directory. If a user creates a new `Chroma` instance with the same persistence directory, it will get loaded up automatically. If they use `from_texts` or `from_documents` in this way, the documents will be loaded into the existing store. There is the chance of some funky behavior if the user passes a different embedding function from the one used to create the collection - we will make this easier in future updates. For now, we log a warning.	2 years ago
Anton Troynikov	78abd277ff	Chroma in LangChain (#1010 ) Chroma is a simple to use, open-source, zero-config, zero setup vectorstore. Simply `pip install chromadb`, and you're good to go. Out-of-the-box Chroma is suitable for most LangChain workloads, but is highly flexible. I tested to 1M embs on my M1 mac, with out issues and reasonably fast query times. Look out for future releases as we integrate more Chroma features with LangChain!	2 years ago
Shahriar Tajbakhsh	b7747017d7	Import of `declarative_base` when SQLAlchemy <1.4 (#883 ) In [pyproject.toml](https://github.com/hwchase17/langchain/blob/master/pyproject.toml), the expectation is `SQLAlchemy = "^1"`. But, the way `declarative_base` is imported in [cache.py](https://github.com/hwchase17/langchain/blob/master/langchain/cache.py) will only work with SQLAlchemy >=1.4. This PR makes sure Langchain can be run in environments with SQLAlchemy <1.4	2 years ago
Harrison Chase	c64f98e2bb	Harrison/format agent instructions (#973 ) Co-authored-by: Andrew White <white.d.andrew@gmail.com> Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net> Co-authored-by: Peng Qu <82029664+pengqu123@users.noreply.github.com>	2 years ago
Harrison Chase	91c6cea227	Harrison/batch embeds (#972 ) Co-authored-by: John Dagdelen <jdagdelen@users.noreply.github.com> Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>	2 years ago
Ankush Gola	bc7e56e8df	Add asyncio support for LLM (OpenAI), Chain (LLMChain, LLMMathChain), and Agent (#841 ) Supporting asyncio in langchain primitives allows for users to run them concurrently and creates more seamless integration with asyncio-supported frameworks (FastAPI, etc.) Summary of changes: LLM * Add `agenerate` and `_agenerate` * Implement in OpenAI by leveraging `client.Completions.acreate` Chain * Add `arun`, `acall`, `_acall` * Implement them in `LLMChain` and `LLMMathChain` for now Agent * Refactor and leverage async chain and llm methods * Add ability for `Tools` to contain async coroutine * Implement async SerpaPI `arun` Create demo notebook. Open questions: * Should all the async stuff go in separate classes? I've seen both patterns (keeping the same class and having async and sync methods vs. having class separation)	2 years ago
Harrison Chase	bc53c928fc	Harrison/athropic (#921 ) Co-authored-by: Mike Lambert <mlambert@gmail.com> Co-authored-by: mrbean <sam@you.com> Co-authored-by: mrbean <43734688+sam-h-bean@users.noreply.github.com> Co-authored-by: Ivan Vendrov <ivendrov@gmail.com>	2 years ago
Harrison Chase	1e56879d38	Harrison/save faiss (#916 ) Co-authored-by: Shrey Joshi <shreyjoshi2004@gmail.com>	2 years ago
Harrison Chase	e2b834e427	Harrison/prompt template prefix (#888 ) Co-authored-by: Gabriel Simmons <simmons.gabe@gmail.com>	2 years ago
Harrison Chase	f95cedc443	Harrison/sql rows (#915 ) Co-authored-by: Jon Luo <20971593+jzluo@users.noreply.github.com>	2 years ago
Harrison Chase	ba5a2f06b9	Harrison/inference endpoint (#861 ) Co-authored-by: Eno Reyes <enoreyes@gmail.com>	2 years ago
Kevin Huo	31b054f69d	Add pinecone integration test (#911 ) Basic integration test for pinecone	2 years ago
Harrison Chase	93a091cfb8	Optionally return shell output on incorrect command (#894 ) (#899 ) This allows the LLM to correct its previous command by looking at the error message output to the shell. Additionally, this uses subprocess.run because that is now recommended over subprocess.check_output: https://docs.python.org/3/library/subprocess.html#using-the-subprocess-module Co-authored-by: Amos Ng <me@amos.ng>	2 years ago

... 3 4 5 6 7 ...

571 Commits (289e9aeb9d122d689d68b2e77236ce3dfcd606a7)