langchain

Commit Graph

Author	SHA1	Message	Date
Harrison Chase	db968284f8	tools refactor (#2961 ) Co-authored-by: vowelparrot <130414180+vowelparrot@users.noreply.github.com>	1 year ago
Sebastian	7a8c935b90	Edited for better readability (#3059 ) It looks like some dropdown functionality was intended, but it caused the markdown code to glitch which hurt readability.	1 year ago
Matthieu	822cdb161b	Adding shared chromaDB client option (#2886 ) This pull request addresses the need to share a single `chromadb.Client` instance across multiple instances of the `Chroma` class. By implementing a shared client, we can maintain consistency and reduce resource usage when multiple instances of the `Chroma` classes are created. This is especially relevant in a web app, where having multiple `Chroma` instances with a `persist_directory` leads to these clients not being synced. This PR implements this option while keeping the rest of the architecture unchanged. Changes: 1. Add a client attribute to the `Chroma` class to store the shared `chromadb.Client` instance. 2. Modify the `from_documents` method to accept an optional client parameter. 3. Update the `from_documents` method to use the shared client if provided or create a new client if not provided. Let me know if anything needs to be modified - thanks again for your work on this incredible repo	1 year ago
Harrison Chase	b140d366e3	Harrison/jira (#3055 ) Co-authored-by: William Li <32046231+zywilliamli@users.noreply.github.com> Co-authored-by: William Li <twelvehertz@Williams-MacBook-Air.local>	1 year ago
Amir Karimi	ae7ed31386	Fix redundancy check about config_type in AGENT_TO_CLASS (#2934 ) Fix of issue #2874	1 year ago
J Wynia	b40f90ea04	Spelling to correct conservation to conservation (#3049 ) Issue #3048 corrected spelling	1 year ago
leo-gan	c33883a40e	fixed the Cohere example title (#3053 ) - fixed the Cohere example title (bug in #3041, sorry for it) - fixed the runhouse.ipynb file name inconsistency	1 year ago
Harrison Chase	5107fac656	Harrison/rec gd (#3054 ) Co-authored-by: Benjamin Scholtz <BenSchZA@users.noreply.github.com>	1 year ago
Harrison Chase	eee2f23a79	Harrison/qa eg (#3052 ) Co-authored-by: Sukhpal Saini <bdcorps@users.noreply.github.com>	1 year ago
Harrison Chase	db7106cb79	Harrison/image caption loader (#3051 ) Co-authored-by: Sean Saito <saitosean@ymail.com>	1 year ago
Benjamin Scholtz	36138f28c8	Add GoogleSQL prompt (#2992 ) This PR extends upon @jzluo 's PR #2748 which addressed dialect-specific issues with SQL prompts, and adds a prompt that uses backticks for column names when querying BigQuery. See [GoogleSQL quoted identifiers](https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#quoted_identifiers). Additionally, the SQL agent currently uses a generic prompt. Not sure how best to adopt the same optional dialect-specific prompts as above, but will consider making an issue and PR for that too. See [langchain/agents/agent_toolkits/sql/prompt.py](langchain/agents/agent_toolkits/sql/prompt.py).	1 year ago
Naveen Tatikonda	bb619cd535	Pass kwargs to get OpenSearch client from_texts (#2993 ) ### Description Pass kwargs to get OpenSearch client from `from_texts` function ### Issues Resolved https://github.com/hwchase17/langchain/issues/2819 Signed-off-by: Naveen Tatikonda <navtat@amazon.com>	1 year ago
Harutaka Kawamura	ba9cc230fa	Stringify `AgentType` before saving to yaml (#2998 ) Code to reproduce the issue (with `langchain==0.0.141`): ```python from langchain.agents import initialize_agent, load_tools from langchain.llms import OpenAI llm = OpenAI(temperature=0.9, verbose=True) tools = load_tools(["llm-math"], llm=llm) agent = initialize_agent(tools, llm, agent="zero-shot-react-description", verbose=True) agent.save_agent("agent.yaml") with open("agent.yaml") as f: print(f.read()) ``` Output: ``` _type: !!python/object/apply:langchain.agents.agent_types.AgentType - zero-shot-react-description allowed_tools: - Calculator ... ``` I expected `_type` to be `zero-shot-react-description` but it's actually not. This PR fixes it by stringifying `AgentType` (`Enum`). Signed-off-by: harupy <hkawamura0130@gmail.com>	1 year ago
Nuno Campos	e25528c4f0	Fix incorrect value of outputKeys on AnalyzeDocumentsChain (#3010 )	1 year ago
engkheng	19febc77d6	Support inference of `input_variables` from `jinja2` template (#3013 ) `langchain.prompts.PromptTemplate` is unable to infer `input_variables` from jinja2 template. ```python # Using langchain v0.0.141 template_string = """\ Hello world Your variable: {{ var }} {# This will not get rendered #} {% if verbose %} Congrats! You just turned on verbose mode and got extra messages! {% endif %} """ template = PromptTemplate.from_template(template_string, template_format="jinja2") print(template.input_variables) # Output ['# This will not get rendered #', '% endif %', '% if verbose %'] ``` --------- Co-authored-by: engkheng <ongengkheng929@example.com>	1 year ago
Nuno Campos	dac32c59e5	Nc/combining output parser (#3014 ) Co-authored-by: vowelparrot <130414180+vowelparrot@users.noreply.github.com>	1 year ago
Nuno Campos	79bb5c4f95	Port format instructions fix from js (#3015 )	1 year ago
Harrison Chase	e3cf00b88b	redis from url (#3024 )	1 year ago
Davis Chase	19c85aa990	Factor out doc formatting and add validation (#3026 ) @cnhhoang850 slightly more generic fix for #2944, works for whatever the expected metadata keys are not just `source`	1 year ago
Naveen Tatikonda	3453b7457c	OpenSearch: Add Support for Boolean Filter with ANN search (#3038 ) ### Description Add Support for Boolean Filter with ANN search Documentation - https://opensearch.org/docs/latest/search-plugins/knn/filter-search-knn/#boolean-filter-with-ann-search ### Issues Resolved https://github.com/hwchase17/langchain/issues/2924 Signed-off-by: Naveen Tatikonda <navtat@amazon.com>	1 year ago
leo-gan	5420a0e404	updated langchain/docs/modules/models/llms/integrations/ notebooks (#3041 ) - Updated `langchain/docs/modules/models/llms/integrations/` notebooks: added links to the original sites, the install information, etc. - Added the `nlpcloud` notebook. - Removed "Example" from Titles of some notebooks, so all notebook titles are consistent.	1 year ago
Azam Iftikhar	471ef84835	Examples fixed (#3042 ) ### https://github.com/hwchase17/langchain/issues/2997 Replaced `conversation.memory.store` to `conversation.memory.entity_store.store` As conversation.memory.store doesn't exist and re-ran the whole file.	1 year ago
Tim Asp	dcdcd3f636	bugfix: throw exception if structured output parser doesn't get what it wants (#3044 ) allows the user to catch the issue and handle it rather than failing hard. This happens more than you'd expect when using output parsers with chatgpt, especially if the temp is anything but 0. Sometimes it doesn't want to listen and just does its own thing.	1 year ago
Harrison Chase	afd3e70ae5	Harrison/confluent loader (#2994 ) Co-authored-by: Justin Flick <Justinjayflick@gmail.com>	1 year ago
Altay Sansal	95d578d246	Fix type hint regression (#3033 ) Not sure what happened here but some of the file got overwritten by #2859 which broke filtering logic. Here is it fixed back to normal. @hwchase17 can we expedite this if possible :-) --------- Co-authored-by: Altay Sansal <altay.sansal@tgs.com>	1 year ago
Noah Gundotra	577ec92f16	Include testing instructions for getting setup in CONTRIBUTING.md (#3020 ) Running tests is good sanity check for new users to ensure their development environment is setup correctly.	1 year ago
Harrison Chase	98c70bc190	bump version to 142 (#3021 )	1 year ago
vowelparrot	2356447323	Update Characters notebook (#3019 ) - Most important - fixes the relevance_fn name in the notebook to align with the docs - Updates comments for the summary: <img width="787" alt="image" src="https://user-images.githubusercontent.com/130414180/232520616-2a99e8c3-a821-40c2-a0d5-3f3ea196c9bb.png"> - The new conversation is a bit better, still unfortunate they try to schedule a followup. - Rm the max dialogue turns argument to the conversation function	1 year ago
Harrison Chase	f1d15b4a75	update nb	1 year ago
Harrison Chase	e54f1b69ca	add notebook	1 year ago
vowelparrot	99c0382209	Generative Characters (#2859 ) Add a time-weighted memory retriever and a notebook that approximates a Generative Agent from https://arxiv.org/pdf/2304.03442.pdf The "daily plan" components are removed for now since they are less useful without a virtual world, but the memory is an interesting component to build off. --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Jan Backes	a9310a3e8b	Add Annoy as VectorStore (#2939 ) Adds Annoy (https://github.com/spotify/annoy) as vector Store. RESOLVES hwchase17/langchain#2842 discord ref: https://discord.com/channels/1038097195422978059/1051632794427723827/1096089994168377354 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: vowelparrot <130414180+vowelparrot@users.noreply.github.com>	1 year ago
Harrison Chase	e12e00df12	use output parsers in agents (#2987 )	1 year ago
cs0lar	8b9e02da9d	Fix/issue 1213 (#2932 ) ### Background Continuing to implement all the interface methods defined by the `VectorStore` class. This PR pertains to implementation of the `max_marginal_relevance_search` method. ### Changes - a `max_marginal_relevance_search` method implementation has been added in `weaviate.py` - tests have been added to the the new method - vcr cassettes have been added for the weaviate tests ### Test Plan Added tests for the `max_marginal_relevance_search` implementation ### Change Safety - [x] I have added tests to cover my changes	1 year ago
Harrison Chase	4c02f4bc30	Fix bug in svm.LinearSVC, add support for a relevancy_threshold (#2959 ) (#2981 ) - Modify SVMRetriever class to add an optional relevancy_threshold - Modify SVMRetriever.get_relevant_documents method to filter out documents with similarity scores below the relevancy threshold - Normalized the similarities to be between 0 and 1 so the relevancy_threshold makes more sense - The number of results are limited to the top k documents or the maximum number of relevant documents above the threshold, whichever is smaller This code will now return the top self.k results (or less, if there are not enough results that meet the self.relevancy_threshold criteria). The svm.LinearSVC implementation in scikit-learn is non-deterministic, which means SVMRetriever.from_texts(["bar", "world", "foo", "hello", "foo bar"]) could return [3 0 5 4 2 1] instead of [0 3 5 4 2 1] with a query of "foo". If you pass in multiple "foo" texts, the order could be different each time. Here, we only care if the 0 is the first element, otherwise it will offset the text and similarities. Example: ```python retriever = SVMRetriever.from_texts( ["foo", "bar", "world", "hello", "foo bar"], OpenAIEmbeddings(), k=4, relevancy_threshold=.25 ) result = retriever.get_relevant_documents("foo") ``` yields ```python [Document(page_content='foo', metadata={}), Document(page_content='foo bar', metadata={})] ``` --------- Co-authored-by: Brandon Sandoval <52767641+account00001@users.noreply.github.com>	1 year ago
Mauricio Scheffer	7302787a7b	Fix docs for parse_with_prompt (#2986 )	1 year ago
Paul Garner	69698be3e6	consistently use getLogger(__name__), no root logger (#2989 ) re https://github.com/hwchase17/langchain/issues/439#issuecomment-1510442791 I think it's not polite for a library to use the root logger both of these forms are also used: ``` logger = logging.getLogger(__name__) logger = logging.getLogger(__file__) ``` I am not sure if there is any reason behind one vs the other? (...I am guessing maybe just contributed by different people) it seems to me it'd be better to consistently use `logging.getLogger(__name__)` this makes it easier for consumers of the library to set up log handlers, e.g. for everything with `langchain.` prefix	1 year ago
Harrison Chase	32db2a2c2f	fix lint	1 year ago
Azam Iftikhar	1e655d5ffd	Fixed Regular expression (#2933 ) ### https://github.com/hwchase17/langchain/issues/2898 Instead of `"Action" and "Action Input"` keywords, we are getting `"Action 1" and "Action 1 Input" or "Action Input 1" ` from gpt-3.5-turbo Updated the Regular expression to handle all these cases Attaching the screenshot of the result from the updated Regular expression. <img width="1036" alt="Screenshot 2023-04-16 at 1 39 00 AM" src="https://user-images.githubusercontent.com/55012400/232251184-23ca6cc2-7229-411a-b6e1-53b2f5ec18a5.png">	1 year ago
Harrison Chase	88d3ce12b8	Harrison/diffbot (#2984 ) Co-authored-by: Manuel Saelices <msaelices@gmail.com>	1 year ago
vowelparrot	5ca7ce77cd	Remove pythonrepl from LLM-MathChain (#2943 ) Use numexpr evaluate instead of the python REPL to avoid malicious code injection. Tested against the (limited) math dataset and got the same score as before. For more permissive tools (like the REPL tool itself), other approaches ought to be provided (some combination of Sanitizer + Restricted python + unprivileged-docker + ...), but for a calculator tool, only mathematical expressions should be permitted. See https://github.com/hwchase17/langchain/issues/814	1 year ago
Daniel Nouri	2a0f65f7af	tiktoken: Relax Python version check (#2966 ) tiktoken supports Python >= 3.8, see here: `e1c661edf3/pyproject.toml (L10)` Also works fine when trying locally!	1 year ago
Chetanya Rastogi	aead062a70	Add an example tutorial for using PDFMinerPDFasHTMLLoader (#2960 ) Last week I added the `PDFMinerPDFasHTMLLoader`. I am adding some example code in the notebook to serve as a tutorial for how that loader can be used to create snippets of a pdf that are structured within sections. All the other loaders only provide the `Document` objects segmented by pages but that's pretty loose given the amount of other metadata that can be extracted. With the new loader, one can leverage font-size of the text to decide when a new sections starts and can segment the text more semantically as shown in the tutorial notebook. The cell shows that we are able to find the content of entire section under Related Work for the example pdf which is spread across 2 pages and hence is stored as two separate documents by other loaders	1 year ago
Tim Asp	51894ddd98	allow tokentextsplitters to use model name to select encoder (#2963 ) Fixes a bug I was seeing when the `TokenTextSplitter` was correctly splitting text under the gpt3.5-turbo token limit, but when firing the prompt off too openai, it'd come back with an error that we were over the context limit. gpt3.5-turbo and gpt-4 use `cl100k_base` tokenizer, and so the counts are just always off with the default `gpt-2` encoder. It's possible to pass along the encoding to the `TokenTextSplitter`, but it's much simpler to pass the model name of the LLM. No more concern about keeping the tokenizer and llm model in sync :)	1 year ago
Alex Iribarren	706ebd8f9c	Enforce maximum Wikipedia query length (#2969 ) I got the following stacktrace when the agent was trying to search Wikipedia with a huge query: ``` Thought:{ "action": "Wikipedia", "action_input": "Outstanding is a song originally performed by the Gap Band and written by member Raymond Calhoun. The song originally appeared on the group's platinum-selling 1982 album Gap Band IV. It is one of their signature songs and biggest hits, reaching the number one spot on the U.S. R&B Singles Chart in February 1983. \"Outstanding\" peaked at number 51 on the Billboard Hot 100." } Traceback (most recent call last): File "/usr/src/app/tests/chat.py", line 121, in <module> answer = agent_chain.run(input=question) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain/chains/base.py", line 216, in run return self(kwargs)[self.output_keys[0]] ^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain/chains/base.py", line 116, in __call__ raise e File "/usr/local/lib/python3.11/site-packages/langchain/chains/base.py", line 113, in __call__ outputs = self._call(inputs) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain/agents/agent.py", line 828, in _call next_step_output = self._take_next_step( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain/agents/agent.py", line 725, in _take_next_step observation = tool.run( ^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain/tools/base.py", line 73, in run raise e File "/usr/local/lib/python3.11/site-packages/langchain/tools/base.py", line 70, in run observation = self._run(tool_input) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain/agents/tools.py", line 17, in _run return self.func(tool_input) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain/utilities/wikipedia.py", line 40, in run search_results = self.wiki_client.search(query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/wikipedia/util.py", line 28, in __call__ ret = self._cache[key] = self.fn(args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/wikipedia/wikipedia.py", line 109, in search raise WikipediaException(raw_results['error']['info']) wikipedia.exceptions.WikipediaException: An unknown error occured: "Search request is longer than the maximum allowed length. (Actual: 373; allowed: 300)". Please report it on GitHub! ``` This commit limits the maximum size of the query passed to Wikipedia to avoid this issue.	1 year ago
Nahin Khan	9a03f00e6c	Fix typos (#2977 )	1 year ago
Altay Sansal	9d8ab28837	Add `top_k` and `filter` fields to `ChatGPTPluginRetriever` (#2852 ) This allows to adjust the number of results to retrieve and filter documents based on metadata. --------- Co-authored-by: Altay Sansal <altay.sansal@tgs.com>	1 year ago
vowelparrot	4ffc58e07b	Add similarity_search_with_normalized_similarities (#2916 ) Add a method that exposes a similarity search with corresponding normalized similarity scores. Implement only for FAISS now. ### Motivation: Some memory definitions combine `relevance` with other scores, like recency , importance, etc. While many (but not all) of the `VectorStore`'s expose a `similarity_search_with_score` method, they don't all interpret the units of that score (depends on the distance metric and whether or not the the embeddings are normalized). This PR proposes a `similarity_search_with_normalized_similarities` method that lets consumers of the vector store not have to worry about the metric and embedding scale. Most providers default to euclidean distance, with Pinecone being one exception (defaults to cosine _similarity_). --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Tim Asp	b9db20481f	Fix wrong token counts from `get_num_tokens` from openai llms (#2952 ) The encoding fetch was out of date. Luckily OpenAI has a nice[ `encoding_for_model`](`46287bfa49/tiktoken/model.py`) function in `tiktoken` we can use now.	1 year ago
Tim Asp	fea5619ce9	Add title, lang, description to Web loader document metadata (#2955 ) Title, lang and description are on almost every web page, and are incredibly useful pieces of information that currently isn't captured with the current web base loader I thought about adding the title and description to the content of the document, as that content could be useful in search, but I left it out for right now. If you think it'd be worth adding, happy to add it. I've found it's nice to have the title/description in the metadata to have some structured data when retrieving rows from vectordbs for use with summary and source citation, so if we do want to add it to the `page_content`, i'd advocate for it to also be included in metadata.	1 year ago

1 2 3 4 5 ...

1371 Commits (db968284f8f3964630f119c95cca923f112ad47b) All Branches Search

1371 Commits (db968284f8f3964630f119c95cca923f112ad47b)

All Branches