langchain

Commit Graph

Author	SHA1	Message	Date
Eugene Yurtsev	ae0c3382dd	Add MimeType based parser (#4376 ) # Add MimeType Based Parser This PR adds a MimeType Based Parser. The parser inspects the mime-type of the blob it is parsing and based on the mime-type can delegate to the sub parser. ## Before submitting Waiting on adding notebooks until more implementations are landed. ## Who can review? Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested: @hwchase17 @vowelparrot	1 year ago
Eugene Yurtsev	aa11f7c89b	Add progress bar to filesystemblob loader, update pytest config for unit tests (#4212 ) This PR adds: * Option to show a tqdm progress bar when using the file system blob loader * Update pytest run configuration to be stricter * Adding a new marker that checks that required pkgs exist	1 year ago
Zander Chase	8b284f9ad0	Pass parsed inputs through to tool _run (#4309 )	1 year ago
Zander Chase	35c9e6ab40	Pass Callbacks through load_tools (#4298 ) - Update the load_tools method to properly accept `callbacks` arguments. - Add a deprecation warning when `callback_manager` is passed - Add two unit tests to check the deprecation warning is raised and to confirm the callback is passed through. Closes issue #4096	1 year ago
Harrison Chase	c8b0b6e6c1	add youtube tools (#4320 )	1 year ago
Eugene Yurtsev	423f497168	Add BlobParser abstraction (#3979 ) This PR adds the BlobParser abstraction. It follows the proposal described here: https://github.com/hwchase17/langchain/pull/2833#issuecomment-1509097756	1 year ago
Zander Chase	1017e5cee2	Add LCP Client (#4198 ) Adding a client to fetch datasets, examples, and runs from a LCP instance and run objects over them.	1 year ago
Zander Chase	a30f42da4e	Update V2 Tracer (#4193 ) - Update the RunCreate object to work with recent changes - Add optional Example ID to the tracer - Adjust default persist_session behavior to attempt to load the session if it exists - Raise more useful HTTP errors for logging - Add unit testing - Fix the default ID to be a UUID for v2 tracer sessions Broken out from the big draft here: https://github.com/hwchase17/langchain/pull/4061	1 year ago
Zander Chase	6032a051e9	Add Tenant ID to V2 Tracer (#4135 ) Update the V2 tracer to - use UUIDs instead of int's - load a tenant ID and use that when saving sessions	1 year ago
Zander Chase	2f087d63af	Fix Python RePL Tool (#4137 ) Filter out kwargs from inferred schema when determining if a tool is single input. Add a couple unit tests. Move tool unit tests to the tools dir	1 year ago
Mike Wang	67db495fcf	[agent] Add Spark Agent (#4020 ) - added support for spark through pyspark library. - added jupyter notebook as example.	1 year ago
rogerserper	b1446bea5f	google-serper: async + full json results + support for Google Images, Places and News (#4078 ) * implemented arun, results, and aresults. Reuses aiosession if available. * helper tools GoogleSerperRun and GoogleSerperResults * support for Google Images, Places and News (examples given) and filtering based on time (e.g. past hour) * updated docs	1 year ago
Zander Chase	65c3b146c9	Accept str or list[str] for shell (#4060 ) Relax the requirements	1 year ago
Zander Chase	afa9d1292b	Re-Permit Partials in `Tool` (#4058 ) Resolved issue #4053 Now that StructuredTool is a separate class, this constraint is no longer needed. Added/updated a unit test	1 year ago
Davis Chase	f08a76250f	Better custom model handling OpenAICallbackHandler (#4009 ) Thanks @maykcaldas for flagging! think this should resolve #3988. Let me know if you still see issues after next release.	1 year ago
Harrison Chase	cd3f8582cb	Harrison/combined memory (#3935 ) Co-authored-by: engkheng <60956360+outday29@users.noreply.github.com>	1 year ago
Zander Chase	c582f2e9e3	Add Structure Chat Agent (#3912 ) Create a new chat agent that is compatible with the Multi-input tools	1 year ago
Zander Chase	9b9b231e10	Update some Tools Docs (#3913 ) Haven't gotten to all of them, but this: - Updates some of the tools notebooks to actually instantiate a tool (many just show a 'utility' rather than a tool. More changes to come in separate PR) - Move the `Tool` and decorator definitions to `langchain/tools/base.py` (but still export from `langchain.agents`) - Add scene explain to the load_tools() function - Add unit tests for public apis for the langchain.tools and langchain.agents modules	1 year ago
Zander Chase	84ea17b786	Move Tool Validation (#3923 ) Move tool validation to each implementation of the Agent. Another alternative would be to adjust the `_validate_tools()` signature to accept the output parser (and format instructions) and add logic there. Something like `parser.outputs_structured_actions(format_instructions)` But don't think that's needed right now.	1 year ago
Eugene Yurtsev	7cce68a051	Add minimal file system blob loader (#3669 ) This adds a minimal file system blob loader. If looks good, this PR will be merged and a few additional enhancements will be made.	1 year ago
Zura Isakadze	647bbf61c1	Add SQLiteChatMessageHistory (#3534 ) It's based on already existing `PostgresChatMessageHistory` Use case somewhere in between multiple files and Postgres storage.	1 year ago
James Brotchie	921894960b	Add ChatModel, LLM, and Embeddings for Google's PaLM APIs (#3575 ) - Add langchain.llms.GooglePalm for text completion, - Add langchain.chat_models.ChatGooglePalm for chat completion, - Add langchain.embeddings.GooglePalmEmbeddings for sentence embeddings, - Add example field to HumanMessage and AIMessage so that users can feed in examples into the PaLM Chat API, - Add system and unit tests. Note async completion for the Text API is not yet supported and will be included in a future PR. Happy for feedback on any aspect of this PR, especially our choice of adding an example field to Human and AI Message objects to enable passing example messages to the API.	1 year ago
Roma	d15f481352	Add unit test to output parsers (#3911 ) This pull request adds unit tests for various output parsers (BooleanOutputParser, CommaSeparatedListOutputParser, and StructuredOutputParser) to ensure their correct functionality and to increase code reliability and maintainability. The tests cover both valid and invalid input cases. Changes: Added unit tests for BooleanOutputParser. Added unit tests for CommaSeparatedListOutputParser. Added unit tests for StructuredOutputParser. Testing: All new unit tests have been executed, and they pass successfully. The overall test suite has been run, and all tests pass. Notes: These tests cover both successful parsing scenarios and error handling for invalid inputs. If any new output parsers are added in the future, corresponding unit tests should also be created to maintain coverage.	1 year ago
Davis Chase	52e4fba897	Fix self query pinecone translation (#3892 ) Enum to string conversion handled differently between python 3.9 and 3.11, currently breaking in 3.11 (see #3788). Thanks @peter-brady for catching this!	1 year ago
Zander Chase	d7e17fc8fe	Deprecate StdInquireTool (#3850 ) - Deprecate StdInInquire tool (dup of HumanInputRun) - Expose missing tools from `langchain.tools`	1 year ago
Ankush Gola	d3ec00b566	Callbacks Refactor [base] (#3256 ) Co-authored-by: Nuno Campos <nuno@boringbits.io> Co-authored-by: Davis Chase <130488702+dev2049@users.noreply.github.com> Co-authored-by: Zander Chase <130414180+vowelparrot@users.noreply.github.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	bd7e0a534c	Harrison/csv loader (#3771 ) Co-authored-by: mrT23 <tal.r@codium.ai>	1 year ago
Mike Wang	512c24fc9c	[annotation improvement] Make AgentType->Class Conversion More Scalable (#3749 ) In the current solution, AgentType and AGENT_TO_CLASS are placed in two separate files and both manually maintained. This might cause inconsistency when we update either of them. — latest — based on the discussion with hwchase17, we don’t know how to further use the newly introduced AgentTypeConfig type, so it doesn’t make sense yet to add it. Instead, it’s better to move the dictionary to another file to keep the loading.py file clear. The consistency is a good point. Instead of asserting the consistency during linting, we added a unittest for consistency check. I think it works as auto unittest is triggered every time with clear failure notice. (well, force push is possible, but we all know what we are doing, so let’s show trust. :>) ~~This PR includes~~ - ~~Introduced AgentTypeConfig as the source of truth of all AgentType related meta data.~~ - ~~Each AgentTypeConfig is a annotated class type which can be used for annotation in other places.~~ - ~~Each AgentTypeConfig can be easily extended when we have more meta data needs.~~ - ~~Strong assertion to ensure AgentType and AGENT_TO_CLASS are always consistent.~~ - ~~Made AGENT_TO_CLASS automatically generated.~~ ~~Test Plan:~~ - ~~since this change is focusing on annotation, lint is the major test focus.~~ - ~~lint, format and test passed on local.~~	1 year ago
Mike Wang	b588446bf9	[simple][test] Added test case for schema.py (#3692 ) - added unittest for schema.py covering utility functions and token counting. - fixed a nit. based on huggingface doc, the tokenizer model is gpt-2. [link](https://huggingface.co/transformers/v4.8.2/_modules/transformers/models/gpt2/tokenization_gpt2_fast.html) - make lint && make format, passed on local - screenshot of new test running result <img width="1283" alt="Screenshot 2023-04-27 at 9 51 55 PM" src="https://user-images.githubusercontent.com/62768671/235057441-c0ac3406-9541-453f-ba14-3ebb08656114.png">	1 year ago
Rafal Wojdyla	160bfae93f	Add `DocstoreFn` - lookup doc via arbitrary function (#3760 ) This partially addresses https://github.com/hwchase17/langchain/issues/1524, but it's also useful for some of our use cases. This `DocstoreFn` allows to lookup a document given a function that accepts the `search` string without the need to implement a custom `Docstore`. This could be useful when: * you don't want to implement a `Docstore` just to provide a custom `search` * it's expensive to construct an `InMemoryDocstore`/dict * you retrieve documents from remote sources * you just want to reuse existing objects	1 year ago
Zander Chase	5042bd40d3	Add Shell Tool (#3335 ) Create an official bash shell tool to replace the dynamically generated one	1 year ago
Zander Chase	334c162f16	Add Other File Utilities (#3209 ) Add other File Utilities, include - List Directory - Search for file - Move - Copy - Remove file Bundle as toolkit Add a notebook that connects to the Chat Agent, which somewhat supports multi-arg input tools Update original read/write files to return the original dir paths and better handle unsupported file paths. Add unit tests	1 year ago
Zander Chase	da7b51455c	Dynamic tool -> single purpose (#3697 ) I think the logic of https://github.com/hwchase17/langchain/pull/3684#pullrequestreview-1405358565 is too confusing. I prefer this alternative because: - All `Tool()` implementations by default will be treated the same as before. No breaking changes. - Less reliance on pydantic magic - The decorator (which only is typed as returning a callable) can infer schema and generate a structured tool - Either way, the recommended way to create a custom tool is through inheriting from the base tool	1 year ago
Zander Chase	4654c58f72	Add validation on agent instantiation for multi-input tools (#3681 ) Tradeoffs here: - No lint-time checking for compatibility - Differs from JS package - The signature inference, etc. in the base tool isn't simple - The `args_schema` is optional Pros: - Forwards compatibility retained - Doesn't break backwards compatibility - User doesn't have to think about which class to subclass (single base tool or dynamic `Tool` interface regardless of input) - No need to change the load_tools, etc. interfaces Co-authored-by: Hasan Patel <mangafield@gmail.com>	1 year ago
Davis Chase	b807a114e4	Add query parsing unit tests (#3672 )	1 year ago
Eugene Yurtsev	708787dddb	Blob: Add validator and use future annotations (#3650 ) Minor changes to the Blob schema. --------- Co-authored-by: Zander Chase <130414180+vowelparrot@users.noreply.github.com>	1 year ago
Eugene Yurtsev	c5a4b4fea1	Suppress duckdb warning in unit tests explicitly (#3653 ) This catches the warning raised when using duckdb, asserts that it's as expected. The goal is to resolve all existing warnings to make unit-testing much stricter.	1 year ago
Eugene Yurtsev	e6c8cce050	Add unit-test to catch changes to required deps (#3662 ) This adds a unit test that can catch changes to required dependencies	1 year ago
Eugene Yurtsev	055f58960a	Fix pytest collection warning (#3651 ) Fixes a pytest collection warning because the test class starts with the prefix "Test"	1 year ago
Eugene Yurtsev	5d02010763	Introduce Blob and Blob Loader interface (#3603 ) This PR introduces a Blob data type and a Blob loader interface. This is the first of a sequence of PRs that follows this proposal: https://github.com/hwchase17/langchain/pull/2833 The primary goals of these abstraction are: * Decouple content loading from content parsing code. * Help duplicated content loading code from document loaders. * Make lazy loading a default for langchain.	1 year ago
Zander Chase	ee670c448e	Persistent Bash Shell (#3580 ) Clean up linting and make more idiomatic by using an output parser --------- Co-authored-by: FergusFettes <fergusfettes@gmail.com>	1 year ago
Roma	2b4e9a3efa	Add unit test for _merge_splits function (#3513 ) This commit adds a new unit test for the _merge_splits function in the text splitter. The new test verifies that the function merges text into chunks of the correct size and overlap, using a specified separator. The test passes on the current implementation of the function.	1 year ago
Mindaugas Sharskus	a4d85f7fd5	[Fix #3365 ]: Changed regex to cover new line before action serious (#3367 ) Fix for: [Changed regex to cover new line before action serious.](https://github.com/hwchase17/langchain/issues/3365) --- This PR fixes the issue where `ValueError: Could not parse LLM output:` was thrown on seems to be valid input. Changed regex to cover new lines before action serious (after the keywords "Action:" and "Action Input:"). regex101: https://regex101.com/r/CXl1kB/1 --------- Co-authored-by: msarskus <msarskus@cisco.com>	1 year ago
Davis Chase	b2564a6391	fix #3884 (#3475 ) fixes mar bug #3384	1 year ago
Zander Chase	49122a96e7	Structured Tool Bugfixes (#3324 ) - Proactively raise error if a tool subclasses BaseTool, defines its own schema, but fails to add the type-hints - fix the auto-inferred schema of the decorator to strip the unneeded virtual kwargs from the schema dict Helps avoid silent instances of #3297	1 year ago
Davis Chase	46542dc774	Contextual compression retriever (#2915 ) Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	9a0356d276	Harrison/file chat history (#3198 ) Co-authored-by: Young Lee <joybro201@gmail.com>	1 year ago
Zander Chase	4adfd790f0	Update File Management Tools to Include Root Directory (#3112 ) - Permit the specification of a `root_dir` to the read/write file tools to specify a working directory - Add validation for attempts to read/write outside the directory (e.g., through `../../` or symlinks or `/abs/path`'s that don't lie in the correct path) - Add some tests for all One question is whether we should make a default root directory for these? tradeoffs either way	1 year ago
engkheng	dbbc340f25	Validate `input_variables` when using `jinja2` templates (#3140 ) `langchain.prompts.PromptTemplate` and `langchain.prompts.FewShotPromptTemplate` do not validate `input_variables` when initialized as `jinja2` template. ```python # Using langchain v0.0.144 template = """"\ Your variable: {{ foo }} {% if bar %} You just set bar boolean variable to true {% endif %} """ # Missing variable, should raise ValueError prompt_template = PromptTemplate(template=template, input_variables=["bar"], template_format="jinja2", validate_template=True) # Extra variable, should raise ValueError prompt_template = PromptTemplate(template=template, input_variables=["bar", "foo", "extra", "thing"], template_format="jinja2", validate_template=True) ```	1 year ago
Zander Chase	90ef705ced	Update Tool Input (#3103 ) - Remove dynamic model creation in the `args()` property. _Only infer for the decorator (and add an argument to NOT infer if someone wishes to only pass as a string)_ - Update the validation example to make it less likely to be misinterpreted as a "safe" way to run a repl There is one example of "Multi-argument tools" in the custom_tools.ipynb from yesterday, but we could add more. The output parsing for the base MRKL agent hasn't been adapted to handle structured args at this point in time --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago

1 2 3 4 5

207 Commits (ea83eed9ba3d20d24ee103320aa08e7369e2f19a)