langchain

Commit Graph

Author	SHA1	Message	Date
Zander Chase	9b9b231e10	Update some Tools Docs (#3913 ) Haven't gotten to all of them, but this: - Updates some of the tools notebooks to actually instantiate a tool (many just show a 'utility' rather than a tool. More changes to come in separate PR) - Move the `Tool` and decorator definitions to `langchain/tools/base.py` (but still export from `langchain.agents`) - Add scene explain to the load_tools() function - Add unit tests for public apis for the langchain.tools and langchain.agents modules	1 year ago
Zander Chase	84ea17b786	Move Tool Validation (#3923 ) Move tool validation to each implementation of the Agent. Another alternative would be to adjust the `_validate_tools()` signature to accept the output parser (and format instructions) and add logic there. Something like `parser.outputs_structured_actions(format_instructions)` But don't think that's needed right now.	1 year ago
Eugene Yurtsev	7cce68a051	Add minimal file system blob loader (#3669 ) This adds a minimal file system blob loader. If looks good, this PR will be merged and a few additional enhancements will be made.	1 year ago
Bank Natchapol	487d4aeebd	Motorhead Memory messages come in reversed order. (#3835 ) History from Motorhead memory return in reversed order It should be Human: 1, AI:..., Human: 2, Ai... ``` You are a chatbot having a conversation with a human. AI: I'm sorry, I'm still not sure what you're trying to communicate. Can you please provide more context or information? Human: 3 AI: I'm sorry, I'm not sure what you mean by "1" and "2". Could you please clarify your request or question? Human: 2 AI: Hello, how can I assist you today? Human: 1 Human: 4 AI: ``` So, i `reversed` the messages before putting in chat_memory.	1 year ago
Davis Chase	900ad106d3	Update google palm model signatures (#3920 ) Signatures out of date after callback refactors	1 year ago
sherylZhaoCode	145ff23fb1	correct the llm type of AzureOpenAI (#3721 ) The llm type of AzureOpenAI was previously set to default, which is openai. But since AzureOpenAI has different API from openai, it creates problems when doing chain saving and loading. This PR corrected the llm type of AzureOpenAI to "azure"	1 year ago
engkheng	21335d43b2	Minor `LLMChain` docs correction (#3791 ) `LLMChain` run method can take multiple input variables.	1 year ago
Rafal Wojdyla	039b672f46	Fixup OpenAI Embeddings - fix the weighted mean (#3778 ) Re: https://github.com/hwchase17/langchain/issues/3777 Copy pasting from the issue: While working on https://github.com/hwchase17/langchain/issues/3722 I have noticed that there might be a bug in the current implementation of the OpenAI length safe embeddings in `_get_len_safe_embeddings`, which before https://github.com/hwchase17/langchain/issues/3722 was actually the default implementation regardless of the length of the context (via https://github.com/hwchase17/langchain/pull/2330). It appears the weights used are constant and the length of the embedding vector (1536) and NOT the number of tokens in the batch, as in the reference implementation at https://github.com/openai/openai-cookbook/blob/main/examples/Embedding_long_inputs.ipynb <hr> Here's some debug info: <img width="1094" alt="image" src="https://user-images.githubusercontent.com/1419010/235286595-a8b55298-7830-45df-b9f7-d2a2ad0356e0.png"> <hr> We can also validate this against the reference implementation: <details> <summary>Reference implementation (click to unroll)</summary> This implementation is copy pasted from https://github.com/openai/openai-cookbook/blob/main/examples/Embedding_long_inputs.ipynb ```py import openai from itertools import islice import numpy as np from tenacity import retry, wait_random_exponential, stop_after_attempt, retry_if_not_exception_type EMBEDDING_MODEL = 'text-embedding-ada-002' EMBEDDING_CTX_LENGTH = 8191 EMBEDDING_ENCODING = 'cl100k_base' # let's make sure to not retry on an invalid request, because that is what we want to demonstrate @retry(wait=wait_random_exponential(min=1, max=20), stop=stop_after_attempt(6), retry=retry_if_not_exception_type(openai.InvalidRequestError)) def get_embedding(text_or_tokens, model=EMBEDDING_MODEL): return openai.Embedding.create(input=text_or_tokens, model=model)["data"][0]["embedding"] def batched(iterable, n): """Batch data into tuples of length n. The last batch may be shorter.""" # batched('ABCDEFG', 3) --> ABC DEF G if n < 1: raise ValueError('n must be at least one') it = iter(iterable) while (batch := tuple(islice(it, n))): yield batch def chunked_tokens(text, encoding_name, chunk_length): encoding = tiktoken.get_encoding(encoding_name) tokens = encoding.encode(text) chunks_iterator = batched(tokens, chunk_length) yield from chunks_iterator def reference_safe_get_embedding(text, model=EMBEDDING_MODEL, max_tokens=EMBEDDING_CTX_LENGTH, encoding_name=EMBEDDING_ENCODING, average=True): chunk_embeddings = [] chunk_lens = [] for chunk in chunked_tokens(text, encoding_name=encoding_name, chunk_length=max_tokens): chunk_embeddings.append(get_embedding(chunk, model=model)) chunk_lens.append(len(chunk)) if average: chunk_embeddings = np.average(chunk_embeddings, axis=0, weights=chunk_lens) chunk_embeddings = chunk_embeddings / np.linalg.norm(chunk_embeddings) # normalizes length to 1 chunk_embeddings = chunk_embeddings.tolist() return chunk_embeddings ``` </details> ```py long_text = 'foo bar' * 5000 reference_safe_get_embedding(long_text, average=True)[:10] # Here's the first 10 floats from the reference embeddings: [0.004407593824276758, 0.0017611146161865465, -0.019824815970984996, -0.02177626039794025, -0.012060967454897886, 0.0017955296329155309, -0.015609168983609643, -0.012059823076681351, -0.016990468527792825, -0.004970484452089445] # and now langchain implementation from langchain.embeddings.openai import OpenAIEmbeddings OpenAIEmbeddings().embed_query(long_text)[:10] [0.003791506184693747, 0.0025310066579390025, -0.019282322699514628, -0.021492679249899803, -0.012598522213242891, 0.0022181168611315662, -0.015858940621301307, -0.011754004130791204, -0.016402944319627515, -0.004125287485127554] # clearly they are different ^ ```	1 year ago
Younis Shah	22a1896c30	[docs]: updates connecting_to_a_feature_store.ipynb (#3776 ) * fixes `FeastPromptTemplate.format` example to use `driver_id`	1 year ago
Harrison Chase	e28c6403aa	Harrison/cohere reranker (#3904 )	1 year ago
Zura Isakadze	647bbf61c1	Add SQLiteChatMessageHistory (#3534 ) It's based on already existing `PostgresChatMessageHistory` Use case somewhere in between multiple files and Postgres storage.	1 year ago
James Brotchie	921894960b	Add ChatModel, LLM, and Embeddings for Google's PaLM APIs (#3575 ) - Add langchain.llms.GooglePalm for text completion, - Add langchain.chat_models.ChatGooglePalm for chat completion, - Add langchain.embeddings.GooglePalmEmbeddings for sentence embeddings, - Add example field to HumanMessage and AIMessage so that users can feed in examples into the PaLM Chat API, - Add system and unit tests. Note async completion for the Text API is not yet supported and will be included in a future PR. Happy for feedback on any aspect of this PR, especially our choice of adding an example field to Human and AI Message objects to enable passing example messages to the API.	1 year ago
Roma	d15f481352	Add unit test to output parsers (#3911 ) This pull request adds unit tests for various output parsers (BooleanOutputParser, CommaSeparatedListOutputParser, and StructuredOutputParser) to ensure their correct functionality and to increase code reliability and maintainability. The tests cover both valid and invalid input cases. Changes: Added unit tests for BooleanOutputParser. Added unit tests for CommaSeparatedListOutputParser. Added unit tests for StructuredOutputParser. Testing: All new unit tests have been executed, and they pass successfully. The overall test suite has been run, and all tests pass. Notes: These tests cover both successful parsing scenarios and error handling for invalid inputs. If any new output parsers are added in the future, corresponding unit tests should also be created to maintain coverage.	1 year ago
Tim Asp	9c89ff8bd9	Increase `request_timeout` on ChatOpenAI (#3910 ) With longer context and completions, gpt-3.5-turbo and, especially, gpt-4, will more times than not take > 60seconds to respond. Based on some other discussions, it seems like this is an increasingly common problem, especially with summarization tasks. - https://github.com/hwchase17/langchain/issues/3512 - https://github.com/hwchase17/langchain/issues/3005 OpenAI's max 600s timeout seems excessive, so I settled on 120, but I do run into generations that take >240 seconds when using large prompts and completions with GPT-4, so maybe 240 would be a better compromise?	1 year ago
Davis Chase	2451310975	Chroma fix mmr (#3897 ) Fixes #3628, thanks @derekmoeller for the issue!	1 year ago
mbchang	3e1cb31f63	fix: add import for gymnasium (#3899 )	1 year ago
Zander Chase	484707ad29	Add incremental messages token count (#3890 )	1 year ago
Davis Chase	52e4fba897	Fix self query pinecone translation (#3892 ) Enum to string conversion handled differently between python 3.9 and 3.11, currently breaking in 3.11 (see #3788). Thanks @peter-brady for catching this!	1 year ago
Jef Packer	47a685adcf	count tokens instead of chars in autogpt prompt (#3841 ) This looks like a bug. Overall by using len instead of token_counter the prompt thinks it has less context window than it actually does. Because of this it adds fewer messages. The reduced previous message context makes the agent repetitive when selecting tasks.	1 year ago
Nikolas Garske	c4d3d74148	Fix typos in arxiv.ipynb (#3887 ) Several minor typos in the doc for the arxiv document loaders were fixed.	1 year ago
Zander Chase	f7cb2af5f4	Export StructuredTool at `/tools` (#3858 )	1 year ago
Ankush Gola	e87f81b3ec	add more color to callbacks docs (#3856 )	1 year ago
Zander Chase	19912d755e	Vwp/arxiv (#3855 ) Co-authored-by: Mike Wang <62768671+skcoirz@users.noreply.github.com>	1 year ago
Zander Chase	e17858470c	Vwp/multi line input (#3854 ) Co-authored-by: Paolo Rechia <paolorechia@gmail.com>	1 year ago
Harrison Chase	c896657d28	bump version to 154 (#3846 )	1 year ago
Zander Chase	d7e17fc8fe	Deprecate StdInquireTool (#3850 ) - Deprecate StdInInquire tool (dup of HumanInputRun) - Expose missing tools from `langchain.tools`	1 year ago
Zander Chase	b1d69d3e7a	Vwp/fix vectorstore typing (#3851 ) Co-authored-by: Jay Stakelon <stakes@users.noreply.github.com>	1 year ago
Zander Chase	fbbdf161cd	Lambda Tool (#3842 ) Co-authored-by: Jason Holtkamp <holtkam2@gmail.com>	1 year ago
Ankush Gola	d3ec00b566	Callbacks Refactor [base] (#3256 ) Co-authored-by: Nuno Campos <nuno@boringbits.io> Co-authored-by: Davis Chase <130488702+dev2049@users.noreply.github.com> Co-authored-by: Zander Chase <130414180+vowelparrot@users.noreply.github.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Zander Chase	18ec22fe56	Remove multi-input tool section (#3810 ) Moving to new notebook. Will re-intro w/ new agent	1 year ago
mbchang	adcad98bee	fix: fix filepath error in agent simulations docs (#3795 )	1 year ago
Harrison Chase	20aad0bed1	stripe docs	1 year ago
Harrison Chase	378f0889eb	bump version to 153 (#3774 )	1 year ago
Sheldon	399065e858	update zilliz example (#3578 ) 1. Now the Zilliz example can't connect to Zilliz Cloud, fixed Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	bd7e0a534c	Harrison/csv loader (#3771 ) Co-authored-by: mrT23 <tal.r@codium.ai>	1 year ago
Harrison Chase	c494ca3ad2	Harrison/doc2txt (#3772 ) Co-authored-by: rishni ratnam <rishniratnam@gmail.com>	1 year ago
Mike Wang	ce4fea983b	[simple] added test case and improve self class return type annotation (#3773 ) a simple follow up of https://github.com/hwchase17/langchain/pull/3748 - added test case - improve annotation when function return type is class itself.	1 year ago
Harrison Chase	0c0f14407c	Harrison/tair (#3770 ) Co-authored-by: Seth Huang <848849+seth-hg@users.noreply.github.com>	1 year ago
Aurélien SCHILTZ	502ba6a0be	Fix type annotation for SQLDatabaseToolkit.llm (#3581 ) Currently `langchain.agents.agent_toolkits.SQLDatabaseToolkit` has a field `llm` with type `BaseLLM`. This breaks initialization for some LLMs. For example, trying to use it with GPT4: ``` from langchain.sql_database import SQLDatabase from langchain.chat_models import ChatOpenAI from langchain.agents.agent_toolkits import SQLDatabaseToolkit db = SQLDatabase.from_uri("some_db_uri") llm = ChatOpenAI(model_name="gpt-4") toolkit = SQLDatabaseToolkit(db=db, llm=llm) # pydantic.error_wrappers.ValidationError: 1 validation error for SQLDatabaseToolkit # llm # Can't instantiate abstract class BaseLLM with abstract methods _agenerate, _generate, _llm_type (type=type_error) ``` Seems like much of the rest of the codebase has switched from BaseLLM to BaseLanguageModel. This PR makes the change for SQLDatabaseToolkit as well	1 year ago
uyhcire	0a7a2b99b5	Fix Chroma integration failing when there are less than 4 items in the collection (#3674 ) The code was failing to decrement the `n_results` kwarg passed to `query(...)`	1 year ago
Rafal Wojdyla	57e028549a	Expose kwargs in `LLMChainExtractor.from_llm` (#3748 ) Re: https://github.com/hwchase17/langchain/issues/3747	1 year ago
Mike Wang	512c24fc9c	[annotation improvement] Make AgentType->Class Conversion More Scalable (#3749 ) In the current solution, AgentType and AGENT_TO_CLASS are placed in two separate files and both manually maintained. This might cause inconsistency when we update either of them. — latest — based on the discussion with hwchase17, we don’t know how to further use the newly introduced AgentTypeConfig type, so it doesn’t make sense yet to add it. Instead, it’s better to move the dictionary to another file to keep the loading.py file clear. The consistency is a good point. Instead of asserting the consistency during linting, we added a unittest for consistency check. I think it works as auto unittest is triggered every time with clear failure notice. (well, force push is possible, but we all know what we are doing, so let’s show trust. :>) ~~This PR includes~~ - ~~Introduced AgentTypeConfig as the source of truth of all AgentType related meta data.~~ - ~~Each AgentTypeConfig is a annotated class type which can be used for annotation in other places.~~ - ~~Each AgentTypeConfig can be easily extended when we have more meta data needs.~~ - ~~Strong assertion to ensure AgentType and AGENT_TO_CLASS are always consistent.~~ - ~~Made AGENT_TO_CLASS automatically generated.~~ ~~Test Plan:~~ - ~~since this change is focusing on annotation, lint is the major test focus.~~ - ~~lint, format and test passed on local.~~	1 year ago
Harrison Chase	b7ae9f715d	Langchain with reddit (#3661 ) (#3768 ) I have added a reddit document loader which fetches the text from the Posts of Subreddits or Reddit users, using the `praw` Python package. I have also added an example notebook reddit.ipynb in order to guide users to use this dataloader. This code was made in format similar to twiiter document loader. I have run code formating, linting and also checked the code myself for different scenarios. This is my first contribution to an open source project and I am really excited about this. If you want to suggest some improvements in my code, I will be happy to do it. :) Co-authored-by: Taaha Bajwa <taaha.s.bajwa@gmail.com>	1 year ago
Kohei Kumazaki	fa4c35e9e5	Fix encoding issue in WebBaseLoader (#3602 ) The character code mismatches occurred when character information was not included in the response header (In my case, a Japanese web page). I solved this issue by changing the encoding setting to apparent_encoding.	1 year ago
Harrison Chase	be7a8e0824	Harrison/redis cache (#3766 ) Co-authored-by: Tyler Hutcherson <tyler.hutcherson@redis.com>	1 year ago
Mike Wang	b588446bf9	[simple][test] Added test case for schema.py (#3692 ) - added unittest for schema.py covering utility functions and token counting. - fixed a nit. based on huggingface doc, the tokenizer model is gpt-2. [link](https://huggingface.co/transformers/v4.8.2/_modules/transformers/models/gpt2/tokenization_gpt2_fast.html) - make lint && make format, passed on local - screenshot of new test running result <img width="1283" alt="Screenshot 2023-04-27 at 9 51 55 PM" src="https://user-images.githubusercontent.com/62768671/235057441-c0ac3406-9541-453f-ba14-3ebb08656114.png">	1 year ago
Harrison Chase	15b92d361d	Harrison/confluence stuff (#3765 ) Co-authored-by: Jelmer Borst <japborst@gmail.com>	1 year ago
SimFG	5998b53596	Use the GPTCache api interface (#3693 ) Use the GPTCache api interface to reduce the possibility of compatibility issues	1 year ago
engkheng	f37a932b24	Improve chat prompt template docs (#3719 ) Add a few more explanations and examples.	1 year ago
Robert Perrotta	22770f5202	Make StuffDocumentsChain doc separator configurable (#3718 ) This PR makes the `"\n\n"` string with which `StuffDocumentsChain` joins formatted documents a property so it can be configured. The new `document_separator` property defaults to `"\n\n"` so the change is backwards compatible.	1 year ago

... 2 3 4 5 6 ...

1821 Commits (ea83eed9ba3d20d24ee103320aa08e7369e2f19a) All Branches Search

1821 Commits (ea83eed9ba3d20d24ee103320aa08e7369e2f19a)

All Branches