langchain

Commit Graph

Author	SHA1	Message	Date
Harrison Chase	f95d551f7a	Harrison/shallow metadata (#1599 ) Co-authored-by: Jesse Zhang <jessetanzhang@gmail.com>	2 years ago
Harrison Chase	9f78717b3c	Harrison/callbacks (#1587 )	2 years ago
Harrison Chase	cc423f40f1	Harrison/youtube loader (#1545 ) Co-authored-by: Julian Wustl <57504258+Julianwustl@users.noreply.github.com>	2 years ago
Harrison Chase	7ade419a0e	allow passing of messages into prompt template (#1505 )	2 years ago
Harrison Chase	064741db58	Harrison/fix text splitter (#1511 ) Co-authored-by: ajaysolanky <ajsolanky@gmail.com> Co-authored-by: Ajay Solanky <ajaysolanky@saw-l14668307kd.myfiosgateway.com>	2 years ago
Harrison Chase	7bec461782	Harrison/memory refactor (#1478 ) moves memory to own module, factors out common stuff	2 years ago
Harrison Chase	0e21463f07	(rfc) chat models (#1424 ) Co-authored-by: Ankush Gola <ankush.gola@gmail.com>	2 years ago
Harrison Chase	63a5614d23	Harrison/simple memory (#1435 ) Co-authored-by: Tim Asp <707699+timothyasp@users.noreply.github.com>	2 years ago
Harrison Chase	1cd8996074	Harrison/summarizer chain (#1356 ) Co-authored-by: Tim Asp <707699+timothyasp@users.noreply.github.com>	2 years ago
Ankush Gola	82baecc892	Add a SQL agent for interacting with SQL Databases and JSON Agent for interacting with large JSON blobs (#1150 ) This PR adds * `ZeroShotAgent.as_sql_agent`, which returns an agent for interacting with a sql database. This builds off of `SQLDatabaseChain`. The main advantages are 1) answering general questions about the db, 2) access to a tool for double checking queries, and 3) recovering from errors * `ZeroShotAgent.as_json_agent` which returns an agent for interacting with json blobs. * Several examples in notebooks --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	2 years ago
Harrison Chase	786852e9e6	partial variables (#1308 )	2 years ago
Harrison Chase	b7708bbec6	rfc: callback changes (#1165 ) conceptually, no reason a tool should know what an "agent action" is unless any objections, can change in all callback handlers	2 years ago
CG80499	af8f5c1a49	Added constitutional chain. (#1147 ) - Added self-critique constitutional chain based on this [paper](https://www.anthropic.com/constitutional.pdf).	2 years ago
Ankush Gola	7b5e160d28	Make Tools own model, add ToolKit Concept (#1095 ) Follow-up of @hinthornw's PR: - Migrate the Tool abstraction to a separate file (`BaseTool`). - `Tool` implementation of `BaseTool` takes in function and coroutine to more easily maintain backwards compatibility - Add a Toolkit abstraction that can own the generation of tools around a shared concept or state --------- Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Francisco Ingham <fpingham@gmail.com> Co-authored-by: Dhruv Anand <105786647+dhruv-anand-aintech@users.noreply.github.com> Co-authored-by: cragwolfe <cragcw@gmail.com> Co-authored-by: Anton Troynikov <atroyn@users.noreply.github.com> Co-authored-by: Oliver Klingefjord <oliver@klingefjord.com> Co-authored-by: William Fu-Hinthorn <whinthorn@Williams-MBP-3.attlocal.net> Co-authored-by: Bruno Bornsztein <bruno.bornsztein@gmail.com>	2 years ago
Francisco Ingham	3f29742adc	Sql alchemy commands used in table info (#1135 ) This approach has several advantages: * it improves the readability of the code * removes incompatibilities between SQL dialects * fixes a bug with `datetime` values in rows and `ast.literal_eval` Huge thanks and credits to @jzluo for finding the weaknesses in the current approach and for the thoughtful discussion on the best way to implement this. --------- Co-authored-by: Francisco Ingham <> Co-authored-by: Jon Luo <20971593+jzluo@users.noreply.github.com>	2 years ago
Harrison Chase	5e10e19bfe	Harrison/align table (#1081 ) Co-authored-by: Francisco Ingham <fpingham@gmail.com>	2 years ago
Ankush Gola	caa8e4742e	Enable streaming for OpenAI LLM (#986 ) * Support a callback `on_llm_new_token` that users can implement when `OpenAI.streaming` is set to `True`	2 years ago
Harrison Chase	ec727bf166	Align table info (#999 ) (#1034 ) Currently the chain is getting the column names and types on the one side and the example rows on the other. It is easier for the llm to read the table information if the column name and examples are shown together so that it can easily understand to which columns do the examples refer to. For an instantiation of this, please refer to the changes in the `sqlite.ipynb` notebook. Also changed `eval` for `ast.literal_eval` when interpreting the results from the sample row query since it is a better practice. --------- Co-authored-by: Francisco Ingham <> --------- Co-authored-by: Francisco Ingham <fpingham@gmail.com>	2 years ago
Shahriar Tajbakhsh	b7747017d7	Import of `declarative_base` when SQLAlchemy <1.4 (#883 ) In [pyproject.toml](https://github.com/hwchase17/langchain/blob/master/pyproject.toml), the expectation is `SQLAlchemy = "^1"`. But, the way `declarative_base` is imported in [cache.py](https://github.com/hwchase17/langchain/blob/master/langchain/cache.py) will only work with SQLAlchemy >=1.4. This PR makes sure Langchain can be run in environments with SQLAlchemy <1.4	2 years ago
Ankush Gola	bc7e56e8df	Add asyncio support for LLM (OpenAI), Chain (LLMChain, LLMMathChain), and Agent (#841 ) Supporting asyncio in langchain primitives allows for users to run them concurrently and creates more seamless integration with asyncio-supported frameworks (FastAPI, etc.) Summary of changes: LLM * Add `agenerate` and `_agenerate` * Implement in OpenAI by leveraging `client.Completions.acreate` Chain * Add `arun`, `acall`, `_acall` * Implement them in `LLMChain` and `LLMMathChain` for now Agent * Refactor and leverage async chain and llm methods * Add ability for `Tools` to contain async coroutine * Implement async SerpaPI `arun` Create demo notebook. Open questions: * Should all the async stuff go in separate classes? I've seen both patterns (keeping the same class and having async and sync methods vs. having class separation)	2 years ago
Harrison Chase	e2b834e427	Harrison/prompt template prefix (#888 ) Co-authored-by: Gabriel Simmons <simmons.gabe@gmail.com>	2 years ago
Harrison Chase	f95cedc443	Harrison/sql rows (#915 ) Co-authored-by: Jon Luo <20971593+jzluo@users.noreply.github.com>	2 years ago
Harrison Chase	93a091cfb8	Optionally return shell output on incorrect command (#894 ) (#899 ) This allows the LLM to correct its previous command by looking at the error message output to the shell. Additionally, this uses subprocess.run because that is now recommended over subprocess.check_output: https://docs.python.org/3/library/subprocess.html#using-the-subprocess-module Co-authored-by: Amos Ng <me@amos.ng>	2 years ago
Harrison Chase	a2b699dcd2	prompt template from string (#884 )	2 years ago
Harrison Chase	8df6b68093	fix length based example selector (#862 )	2 years ago
Jason Liu	54f9e4287f	Pass kwargs from initialize_agent into agent classmethod (#799 ) # Problem I noticed that in order to change the prefix of the prompt in the `zero-shot-react-description` agent we had to dig around to subset strings deep into the agent's attributes. It requires the user to inspect a long chain of attributes and classes. `initialize_agent -> AgentExecutor -> Agent -> LLMChain -> Prompt from Agent.create_prompt` ``` python agent = initialize_agent( tools=tools, llm=fake_llm, agent="zero-shot-react-description" ) prompt_str = agent.agent.llm_chain.prompt.template new_prompt_str = change_prefix(prompt_str) agent.agent.llm_chain.prompt.template = new_prompt_str ``` # Implemented Solution `initialize_agent` accepts `*kwargs` but passes it to `AgentExecutor` but not `ZeroShotAgent`, by simply giving the kwargs to the agent class methods we can support changing the prefix and suffix for one agent while allowing future agents to take advantage of `initialize_agent`. ``` agent = initialize_agent( tools=tools, llm=fake_llm, agent="zero-shot-react-description", agent_kwargs={"prefix": prefix, "suffix": suffix} ) ``` To be fair, this was before finding docs around custom agents here: https://langchain.readthedocs.io/en/latest/modules/agents/examples/custom_agent.html?highlight=custom%20#custom-llmchain but i find that my use case just needed to change the prefix a little. # Changes Pass kwargs to Agent class method * Added a test to check suffix and prefix --------- Co-authored-by: Jason Liu <jason@jxnl.coA>	2 years ago
Roy Williams	6086292252	Centralize logic for loading from LangChainHub, add ability to pin dependencies (#805 ) It's generally considered to be a good practice to pin dependencies to prevent surprise breakages when a new version of a dependency is released. This commit adds the ability to pin dependencies when loading from LangChainHub. Centralizing this logic and using urllib fixes an issue identified by some windows users highlighted in this video - https://youtu.be/aJ6IQUh8MLQ?t=537	2 years ago
Harrison Chase	1ad7973cc6	Harrison/tool decorator (#790 ) Co-authored-by: Jason Liu <jxnl@users.noreply.github.com> Co-authored-by: Jason Liu <jason@jxnl.coA>	2 years ago
Harrison Chase	248c297f1b	Sample row in table info for SQLDatabase (#769 ) (#782 ) The agents usually benefit from understanding what the data looks like to be able to filter effectively. Sending just one row in the table info allows the agent to understand the data before querying and get better results. --------- Co-authored-by: Francisco Ingham <> --------- Co-authored-by: Francisco Ingham <fpingham@gmail.com>	2 years ago
Amos Ng	6ad360bdef	Suggestions for better debugging (#765 ) Please feel free to disregard any changes you disagree with	2 years ago
Ankush Gola	57609845df	add tracing support to langchain (#741 ) * add implementations of `BaseCallbackHandler` to support tracing: `SharedTracer` which is thread-safe and `Tracer` which is not and is meant to be used locally. * Tracers persist runs to locally running `langchain-server` Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	2 years ago
Amos Ng	fa6826e417	Fix sqlalchemy warnings when running tests (#733 ) This has been bugging me when running my own tests that call langchain methods :P	2 years ago
scadEfUr	e3df8ab6dc	move hyde into chains (#728 ) Co-authored-by: scadEfUr <>	2 years ago
Harrison Chase	0ffeabd14f	Harrison/serialize llm chain (#671 )	2 years ago
Harrison Chase	cbc146720b	verbose flag (#683 )	2 years ago
Harrison Chase	a2eeaf3d43	strip whitespace (#680 )	2 years ago
Harrison Chase	54d7f1c933	fix caching (#658 )	2 years ago
Harrison Chase	1ac3319e45	simplify parsing of the final answer (#621 )	2 years ago
Harrison Chase	1511606799	Harrison/fix splitting (#563 ) fix issue where text splitting could possibly create empty docs	2 years ago
Harrison Chase	1192cc0767	smart text splitter (#530 ) smart text splitter that iteratively tries different separators until it works!	2 years ago
Harrison Chase	9833fcfe32	fix caching (#555 )	2 years ago
Harrison Chase	330a5b42d4	fix map reduce chain (#550 )	2 years ago
Harrison Chase	4974f49bb7	add return_direct flag to tool (#537 ) adds a return_direct flag to tools, which just returns the tool output as the final output	2 years ago
Harrison Chase	1631981f84	Harrison/fix and test caching (#538 )	2 years ago
Harrison Chase	9e04c34e20	Add BaseCallbackHandler and CallbackManager (#478 ) Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>	2 years ago
Harrison Chase	0db05b6725	Harrison/add human prefix (#520 ) Co-authored-by: Andrew Huang <jhuang16888@gmail.com>	2 years ago
Harrison Chase	985496f4be	Docs refactor (#480 ) Big docs refactor! Motivation is to make it easier for people to find resources they are looking for. To accomplish this, there are now three main sections: - Getting Started: steps for getting started, walking through most core functionality - Modules: these are different modules of functionality that langchain provides. Each part here has a "getting started", "how to", "key concepts" and "reference" section (except in a few select cases where it didnt easily fit). - Use Cases: this is to separate use cases (like summarization, question answering, evaluation, etc) from the modules, and provide a different entry point to the code base. There is also a full reference section, as well as extra resources (glossary, gallery, etc) Co-authored-by: Shreya Rajpal <ShreyaR@users.noreply.github.com>	2 years ago
Harrison Chase	d0f194de73	add logic for agent stopping (#420 )	2 years ago
Harrison Chase	95157d0aad	Add schema property to sql database utility class (#448 ) (#462 ) Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com> Co-authored-by: Nuno Campos <nuno@boringbits.io> Co-authored-by: Diwank Singh Tomer <diwank.singh@gmail.com>	2 years ago
Harrison Chase	0c5d3fd894	version 0.0.49 (#436 )	2 years ago
Harrison Chase	f8b605293f	Harrison/improve memory (#432 ) add AI prefix add new type of memory Co-authored-by: Jason <chisanch@usc.edu>	2 years ago
Harrison Chase	ee3b8e89b3	better parsing of agent output (#418 )	2 years ago
Harrison Chase	20959d8c36	check memory variables (#411 ) can have multiple input keys, if some come from memory	2 years ago
Harrison Chase	6b60c509ac	(WIP) add HyDE (#393 ) Co-authored-by: cameronccohen <cameron.c.cohen@gmail.com> Co-authored-by: Cameron Cohen <cameron.cohen@quantco.com>	2 years ago
Harrison Chase	c104d507bf	Harrison/improve data augmented generation docs (#390 ) Co-authored-by: cameronccohen <cameron.c.cohen@gmail.com> Co-authored-by: Cameron Cohen <cameron.cohen@quantco.com>	2 years ago
Harrison Chase	cf98f219f9	Harrison/tools exp (#372 )	2 years ago
Harrison Chase	e7b625fe03	fix text splitter (#375 )	2 years ago
Ankush Gola	8d0869c6d3	change run to use args and kwargs (#367 ) Before, `run` was not able to be called with multiple arguments. This expands the functionality.	2 years ago
Harrison Chase	c1b50b7b13	Harrison/map reduce merge (#344 ) Co-authored-by: John Nay <JohnNay@users.noreply.github.com>	2 years ago
Harrison Chase	78b31e5966	Harrison/cache (#343 )	2 years ago
Harrison Chase	8cf62ce06e	Harrison/single input (#347 ) allow passing of single input into chain Co-authored-by: thepok <richterthepok@yahoo.de>	2 years ago
Harrison Chase	9bb7195085	Harrison/llm saving (#331 ) Co-authored-by: Akash Samant <70665700+asamant21@users.noreply.github.com>	2 years ago
Hunter Gerlach	482611f426	unit test / code coverage improvements (#322 ) This PR has two contributions: 1. Add test for when stop token is found in middle of text 2. Add code coverage tooling and instructions - Add pytest-cov via poetry - Add necessary config files - Add new make instruction for `coverage` - Update README with coverage guidance - Update minor README formatting/spelling Co-authored-by: Hunter Gerlach <hunter@huntergerlach.com>	2 years ago
Shobith Alva	19a9fa16a9	Add `clear()` method for `Memory` (#305 ) a simple helper to clear the buffer in `Conversation*Memory` classes	2 years ago
Harrison Chase	e02d6b2288	beta: logger (#307 )	2 years ago
andersenchen	5267ebce2d	Add LLMCheckerChain (#281 ) Implementation of https://github.com/jagilley/fact-checker. Works pretty well. <img width="993" alt="Screenshot 2022-12-07 at 4 41 47 PM" src="https://user-images.githubusercontent.com/101075607/206302751-356a19ff-d000-4798-9aee-9c38b7f532b9.png"> Verifying this manually: 1. "Only two kinds of egg-laying mammals are left on the planet today—the duck-billed platypus and the echidna, or spiny anteater." https://www.scientificamerican.com/article/extreme-monotremes/ 2. "An [Echidna] egg weighs 1.5 to 2 grams (0.05 to 0.07 oz)[[19]](https://en.wikipedia.org/wiki/Echidna#cite_note-19) and is about 1.4 centimetres (0.55 in) long." https://en.wikipedia.org/wiki/Echidna#:~:text=sleep%20is%20suppressed.-,Reproduction,a%20reptile%2Dlike%20egg%20tooth. 3. "A [platypus] lays one to three (usually two) small, leathery eggs (similar to those of reptiles), about 11 mm (7⁄16 in) in diameter and slightly rounder than bird eggs." https://en.wikipedia.org/wiki/Platypus#:~:text=It%20lays%20one%20to%20three,slightly%20rounder%20than%20bird%20eggs. 4. Therefore, an Echidna is the mammal that lays the biggest eggs. cc @hwchase17	2 years ago
Harrison Chase	3c1c7ba672	update branch name in gha (#274 )	2 years ago
Akash Samant	48b093823e	Add a Transformation Chain (#257 ) Arbitrary transformation chains that can be used to add dictionary extractions from llms/other chains	2 years ago
coyotespike	b7bef36ee1	BashChain (#260 ) Love the project, a ton of fun! I think the PR is pretty self-explanatory, happy to make any changes! I am working on using it in an `LLMBashChain` and may update as that progresses. Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	2 years ago
Harrison Chase	28be37f470	LLMRequestsChain (#267 )	2 years ago
John McDonnell	68666d6a22	Gracefully degrade when model asks for nonexistent tool (#268 ) Not yet tested, but very simple change, assumption is that we're cool with just producing a generic output when tool is not found	2 years ago
Harrison Chase	f5c665a544	combine python files (#256 )	2 years ago
Harrison Chase	db58032973	introduce output parser (#250 )	2 years ago
Harrison Chase	a9ce04201f	Harrison/improve usability of api chain (#247 ) improve usability of api chain	2 years ago
Harrison Chase	c897bd6cbd	api chain (#246 ) Co-authored-by: Subhash Ramesh <33400216+thecooltechguy@users.noreply.github.com>	2 years ago
Xupeng (Tony) Tong	bb4bf9d6d0	chore: minor clean up / formatting (#233 ) to get familiarize with the project	2 years ago
Andrew Gleave	ea67c049f0	Support SQL statements that return no results (#222 ) Adds support for statements such as insert, update etc which do not return any rows. `engine.execute` is deprecated and so execution has been updated to use `connection.exec_driver_sql` as-per: https://docs.sqlalchemy.org/en/14/core/connections.html#sqlalchemy.engine.Engine.execute	2 years ago
Akash Samant	d368c43648	Bug Fix (#221 ) Quick bug fix for semantic similarity vector injection	2 years ago
Harrison Chase	b94244eb12	nits (#210 ) use json.dump move test to integration tests (since it requires huggingface_hub)	2 years ago
Akash Samant	ae72cf84b8	Save Prompts (#194 )	2 years ago
Bagatur	b90e25f786	Add HuggingFace Hub Embeddings (#125 ) Add support for calling HuggingFace embedding models using the HuggingFaceHub Inference API. New class mirrors the existing HuggingFaceHub LLM implementation. Currently only supports 'sentence-transformers' models. Closes #86	2 years ago
Harrison Chase	6eab5254e5	add docs for custom agents (#196 )	2 years ago
Harrison Chase	08deed9002	Harrison/memory docs (#195 ) update memory docs and change variables	2 years ago
Harrison Chase	b913df3774	make attrs public (#187 ) since they are used outside of the class, should be public	2 years ago
Samantha Whitmore	a408ed3ea3	Samantha/add conversation chain (#166 ) Add MemoryChain and ConversationChain as chains that take a docstore in addition to the prompt, and use the docstore to stuff context into the prompt. This can be used to have an ongoing conversation with a chatbot. Probably needs a bit of refactoring for code quality Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	2 years ago
Harrison Chase	4334ffa6f9	Harrison/clean up language (#179 ) dynamic prompts are no longer a thing	2 years ago
Samantha Whitmore	09f301cd38	Add add_example method to all ExampleSelector classes, with tests (#178 ) Also updated docs, and noticed an issue with the add_texts method on VectorStores that I had missed before -- the metadatas arg should be required to match the classmethod which initializes the VectorStores (the add_example methods break otherwise in the ExampleSelectors)	2 years ago
Harrison Chase	d3a7429f61	(WIP) agents (#171 )	2 years ago
Harrison Chase	4a4dfbfbed	Harrison/sequential chains (#168 ) add support for basic sequential chains	2 years ago
Samantha Whitmore	315b0c09c6	wip: add method for both docstore and embeddings (#119 ) this will break atm but wanted to get thoughts on implementation. 1. should add() be on docstore interface? 2. should InMemoryDocstore change to take a list of documents as init? (makes this slightly easier to implement in FAISS -- if we think it is less clean then could expose a method to get the number of documents currently in the dict, and perform the logic of creating the necessary dictionary in the FAISS.add_texts method. Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	2 years ago
Harrison Chase	c02eb199b6	add few shot example (#148 )	2 years ago
Nicholas Larus-Stone	0c3ae78ec1	chore: update ascii colors to work with dark mode (#152 )	2 years ago
Nicholas Larus-Stone	ca4b10bb74	feat: add option to ignore or restrict to SQL tables (#151 ) `SQLDatabase` now accepts two `init` arguments: 1. `ignore_tables` to pass in a list of tables to not search over 2. `include_tables` to restrict to a list of tables to consider	2 years ago
Harrison Chase	1835e8a681	prompt nit (#141 ) doing some cleanup, and i think this just simplifies things...	2 years ago
Harrison Chase	bbb405a492	update colors (#140 )	2 years ago
Harrison Chase	f23b3ceb49	consolidate run functions (#126 ) consolidating logic for when a chain is able to run with single input text, single output text open to feedback on naming, logic, usefulness	2 years ago
Edmar Ferreira	8a5ec894e7	Prompt from file proof of concept using plain text (#127 ) This is a simple proof of concept of using external files as templates. I'm still feeling my way around the codebase. As a user, I want to use files as prompts, so it will be easier to manage and test prompts. The future direction is to use a template engine, most likely Mako.	2 years ago
Harrison Chase	db37bd089f	model laboratory (#95 )	2 years ago
Harrison Chase	eb36317f9a	Harrison/fix imports (#72 ) fix imports and add section to notebook	2 years ago
Samantha Whitmore	a5b61d59e1	Refactor prompts into module, add example generation utils (#64 )	2 years ago

1 2 3 4

163 Commits (46c963601210d0f1793dd80bb7926c190e6cae13)