langchain

Commit Graph

Author	SHA1	Message	Date
Paul-Emile Brotons	d2cf0d16b3	adding max_marginal_relevance_search method to MongoDBAtlasVectorSearch (#7310 ) Adding a maximal_marginal_relevance method to the MongoDBAtlasVectorSearch vectorstore enhances the user experience by providing more diverse search results Issue: #7304	1 year ago
Bagatur	04cddfba0d	Add lark import error (#7465 )	1 year ago
Matt Robinson	bcab894f4e	feat: Add `UnstructuredTSVLoader` (#7367 ) ### Summary Adds an `UnstructuredTSVLoader` for TSV files. Also updates the doc strings for `UnstructuredCSV` and `UnstructuredExcel` loaders. ### Testing ```python from langchain.document_loaders.tsv import UnstructuredTSVLoader loader = UnstructuredTSVLoader( file_path="example_data/mlb_teams_2012.csv", mode="elements" ) docs = loader.load() ```	1 year ago
Ronald Li	490f4a9ff0	Fixes KeyError in AmazonKendraRetriever initializer (#7464 ) ### Description argument variable client is marked as required in commit `81e5b1ad36` which breaks the default way of initialization providing only index_id. This commit avoid KeyError exception when it is initialized without a client variable ### Dependencies no dependency required	1 year ago
Jona Sassenhagen	7ffc431b3a	Add spacy sentencizer (#7442 ) `SpacyTextSplitter` currently uses spacy's statistics-based `en_core_web_sm` model for sentence splitting. This is a good splitter, but it's also pretty slow, and in this case it's doing a lot of work that's not needed given that the spacy parse is then just thrown away. However, there is also a simple rules-based spacy sentencizer. Using this is at least an order of magnitude faster than using `en_core_web_sm` according to my local tests. Also, spacy sentence tokenization based on `en_core_web_sm` can be sped up in this case by not doing the NER stage. This shaves some cycles too, both when loading the model and when parsing the text. Consequently, this PR adds the option to use the basic spacy sentencizer, and it disables the NER stage for the current approach, which is kept as the default. Lastly, when extracting the tokenized sentences, the `text` attribute is called directly instead of doing the string conversion, which is IMO a bit more idiomatic.	1 year ago
charosen	50a9fcccb0	feat(module): add param ids to ElasticVectorSearch.from_texts method (#7425 ) # add param ids to ElasticVectorSearch.from_texts method. - Description: add param ids to ElasticVectorSearch.from_texts method. - Issue: NA. It seems `add_texts` already supports passing in document ids, but param `ids` is omitted in `from_texts` classmethod, - Dependencies: None, - Tag maintainer: @rlancemartin, @eyurtsev please have a look, thanks ``` # ElasticVectorSearch add_texts def add_texts( self, texts: Iterable[str], metadatas: Optional[List[dict]] = None, refresh_indices: bool = True, ids: Optional[List[str]] = None, kwargs: Any, ) -> List[str]: ... ``` ``` # ElasticVectorSearch from_texts @classmethod def from_texts( cls, texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, elasticsearch_url: Optional[str] = None, index_name: Optional[str] = None, refresh_indices: bool = True, kwargs: Any, ) -> ElasticVectorSearch: ``` Co-authored-by: charosen <charosen@bupt.cn>	1 year ago
James Yin	a5fd8873b1	fix: type hint of get_chat_history in BaseConversationalRetrievalChain (#7461 ) The type hint of `get_chat_history` property in `BaseConversationalRetrievalChain` is incorrect. @baskaryan	1 year ago
nikkie	dfc3f83b0f	docs(vectorstores/integrations/chroma): Fix loading and saving (#7437 ) - Description: Fix loading and saving code about Chroma - Issue: the issue #7436 - Dependencies: - - Twitter handle: https://twitter.com/ftnext	1 year ago
Daniel Chalef	c7f7788d0b	Add ZepMemory; improve ZepChatMessageHistory handling of metadata; Fix bugs (#7444 ) Hey @hwchase17 - This PR adds a `ZepMemory` class, improves handling of Zep's message metadata, and makes it easier for folks building custom chains to persist metadata alongside their chat history. We've had plenty confused users unfamiliar with ChatMessageHistory classes and how to wrap the `ZepChatMessageHistory` in a `ConversationBufferMemory`. So we've created the `ZepMemory` class as a light wrapper for `ZepChatMessageHistory`. Details: - add ZepMemory, modify notebook to demo use of ZepMemory - Modify summary to be SystemMessage - add metadata argument to add_message; add Zep metadata to Message.additional_kwargs - support passing in metadata	1 year ago
Saurabh Chaturvedi	8f8e8d701e	Fix info about YouTube (#7447 ) (Unintentionally mean 😅) nit: YouTube wasn't created by Google, this PR fixes the mention in docs.	1 year ago
Leonid Ganeline	560c4dfc98	docstrings: `docstore` and `client` (#6783 ) updated docstrings in `docstore/` and `client/` @baskaryan	1 year ago
Jeroen Van Goey	f5bd88757e	Fix typo (#7416 ) `quesitons` -> `questions`.	1 year ago
Alejandro Garrido Mota	ea9c3cc9c9	Fix syntax erros in documentation (#7409 ) - Description: Tiny documentation fix. In Python, when defining function parameters or providing arguments to a function or class constructor, we do not use the `:` character. - Issue: N/A - Dependencies: N/A, - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: @mogaal	1 year ago
Nolan	5da9f9abcb	docs(agents/toolkits): Fix error in document_comparison_toolkit.ipynb (#7417 ) Replace this comment with: - Description: Removes unneeded output warning in documentation at https://python.langchain.com/docs/modules/agents/toolkits/document_comparison_toolkit - Issue: - - Dependencies: - - Tag maintainer: @baskaryan - Twitter handle: @finnless	1 year ago
nikkie	2eb4a2ceea	docs(retrievers/get-started): Fix broken state_of_the_union.txt link (#7399 ) Thank you for this awesome library. - Description: Fix broken link in documentation - Issue: - https://python.langchain.com/docs/modules/data_connection/retrievers/#get-started - the URL: https://github.com/hwchase17/langchain/blob/master/docs/modules/state_of_the_union.txt - I think the right one is https://github.com/hwchase17/langchain/blob/master/docs/extras/modules/state_of_the_union.txt - Dependencies: - - Tag maintainer: @baskaryan - Twitter handle: -	1 year ago
Delgermurun	e7420789e4	improve description of JinaChat (#7397 ) very small doc string change in the `JinaChat` class.	1 year ago
Bagatur	26c86a197c	bump 228 (#7393 )	1 year ago
SvMax	1d649b127e	Added param to return only a structured json from the get_format_instructions method (#5848 ) I just added a parameter to the method get_format_instructions, to return directly the JSON instructions without the leading instruction sentence. I'm planning to use it to define the structure of a JSON object passed in input, the get_format_instructions(). --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	362bc301df	fix jina (#7392 )	1 year ago
Delgermurun	a1603fccfb	integrate JinaChat (#6927 ) Integration with https://chat.jina.ai/api. It is OpenAI compatible API. - Twitter handle: [https://twitter.com/JinaAI_](https://twitter.com/JinaAI_) --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
William FH	4ba7396f96	Add single run eval loader (#7390 ) Plus - add evaluation name to make string and embedding validators work with the run evaluator loader. - Rm unused root validator	1 year ago
Roger Yu	633b673b85	Update pinecone.ipynb (#7382 ) Fix typo	1 year ago
Oleg Zabluda	4d697d3f24	Allow passing custom prompts to GraphIndexCreator (#7381 ) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
William FH	612a74eb7e	Make Ref Example Threadsafe (#7383 ) Have noticed transient ref example misalignment. I believe this is caused by the logic of assigning an example within the thread executor rather than before.	1 year ago
William FH	4789c99bc2	Add String Distance and Embedding Evaluators (#7123 ) Add a string evaluator and pairwise string evaluator implementation for: - Embedding distance - String distance Update docs	1 year ago
ljeagle	fb6e63dc36	Upgrade the AwaDB from 0.3.5 to 0.3.6 (#7363 )	1 year ago
William FH	c5edbea34a	Load Run Evaluator (#7101 ) Current problems: 1. Evaluating LLMs or Chat models isn't smooth. Even specifying 'generations' as the output inserts a redundant list into the eval template 2. Configuring input / prediction / reference keys in the `get_qa_evaluator` function is confusing. Unless you are using a chain with the default keys, you have to specify all the variables and need to reason about whether the key corresponds to the traced run's inputs, outputs or the examples inputs or outputs. Proposal: - Configure the run evaluator according to a model. Use the model type and input/output keys to assert compatibility where possible. Only need to specify a reference_key for certain evaluators (which is less confusing than specifying input keys) When does this work: - If you have your langchain model available (assumed always for run_on_dataset flow) - If you are evaluating an LLM, Chat model, or chain - If the LLM or chat models are traced by langchain (wouldn't work if you add an incompatible schema via the REST API) When would this fail: - Currently if you directly create an example from an LLM run, the outputs are generations with all the extra metadata present. A simple `example_key` and dumping all to the template could make the evaluations unreliable - Doesn't help if you're not using the low level API - If you want to instantiate the evaluator without instantiating your chain or LLM (maybe common for monitoring, for instance) -> could also load from run or run type though What's ugly: - Personally think it's better to load evaluators one by one since passing a config down is pretty confusing. - Lots of testing needs to be added - Inconsistent in that it makes a separate run and example input mapper instead of the original `RunEvaluatorInputMapper`, which maps a run and example to a single input. Example usage running the for an LLM, Chat Model, and Agent. ``` # Test running for the string evaluators evaluator_names = ["qa", "criteria"] model = ChatOpenAI() configured_evaluators = load_run_evaluators_for_model(evaluator_names, model=model, reference_key="answer") run_on_dataset(ds_name, model, run_evaluators=configured_evaluators) ``` <details> <summary>Full code with dataset upload</summary> ``` ## Create dataset from langchain.evaluation.run_evaluators.loading import load_run_evaluators_for_model from langchain.evaluation import load_dataset import pandas as pd lcds = load_dataset("llm-math") df = pd.DataFrame(lcds) from uuid import uuid4 from langsmith import Client client = Client() ds_name = "llm-math - " + str(uuid4())[0:8] ds = client.upload_dataframe(df, name=ds_name, input_keys=["question"], output_keys=["answer"]) ## Define the models we'll test over from langchain.llms import OpenAI from langchain.chat_models import ChatOpenAI from langchain.agents import initialize_agent, AgentType from langchain.tools import tool llm = OpenAI(temperature=0) chat_model = ChatOpenAI(temperature=0) @tool def sum(a: float, b: float) -> float: """Add two numbers""" return a + b def construct_agent(): return initialize_agent( llm=chat_model, tools=[sum], agent=AgentType.OPENAI_MULTI_FUNCTIONS, ) agent = construct_agent() # Test running for the string evaluators evaluator_names = ["qa", "criteria"] models = [llm, chat_model, agent] run_evaluators = [] for model in models: run_evaluators.append(load_run_evaluators_for_model(evaluator_names, model=model, reference_key="answer")) # Run on LLM, Chat Model, and Agent from langchain.client.runner_utils import run_on_dataset to_test = [llm, chat_model, construct_agent] for model, configured_evaluators in zip(to_test, run_evaluators): run_on_dataset(ds_name, model, run_evaluators=configured_evaluators, verbose=True) ``` </details> --------- Co-authored-by: Nuno Campos <nuno@boringbits.io>	1 year ago
Bagatur	1ac347b4e3	update databerry-chaindesk redirect (#7378 )	1 year ago
Joshua Carroll	705d2f5b92	Update the API Reference link in Streamlit integration docs (#7377 ) This page: https://python.langchain.com/docs/modules/callbacks/integrations/streamlit Has a bad API Reference link currently. This PR fixes it to the correct link. Also updates the embedded app link to https://langchain-mrkl.streamlit.app/ (better name) which is hosted in langchain-ai/streamlit-agent repo	1 year ago
Georges Petrov	ec033ae277	Rename Databerry to Chaindesk (#7022 ) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Philip Meier	da5b0723d2	update MosaicML inputs and outputs (#7348 ) As of today (July 7, 2023), the [MosaicML API](https://docs.mosaicml.com/en/latest/inference.html#text-completion-requests) uses `"inputs"` for the prompt This PR adds support for this new format. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bearnardd	184ede4e48	Fix buggy output from GraphQAChain (#7372 ) fixes https://github.com/hwchase17/langchain/issues/7289 A simple fix of the buggy output of `graph_qa`. If we have several entities with triplets then the last entry of `triplets` for a given entity merges with the first entry of the `triplets` of the next entity.	1 year ago
Harrison Chase	7cdf97ba9b	Harrison/add to imports (#7370 ) pgvector cleanup	1 year ago
Bagatur	4d427b2397	Base language model docstrings (#7104 )	1 year ago
ॐ shivam mamgain	2179d4eef8	Fix for KeyError in MlflowCallbackHandler (#7051 ) - Description: `MlflowCallbackHandler` fails with `KeyError: "['name'] not in index"`. See https://github.com/hwchase17/langchain/issues/5770 for more details. Root cause is that LangChain does not pass "name" as a part of `serialized` argument to `on_llm_start()` callback method. The commit where this change was made is probably this: `18af149e91`. My bug fix derives "name" from "id" field. - Issue: https://github.com/hwchase17/langchain/issues/5770 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Alex Gamble	df746ad821	Add a callback handler for Context (https://getcontext.ai ) (#7151 ) ### Description Adding a callback handler for Context. Context is a product analytics platform for AI chat experiences to help you understand how users are interacting with your product. I've added the callback library + an example notebook showing its use. ### Dependencies Requires the user to install the `context-python` library. The library is lazily-loaded when the callback is instantiated. ### Announcing the feature We spoke with Harrison a few weeks ago about also doing a blog post announcing our integration, so will coordinate this with him. Our Twitter handle for the company is @getcontextai, and the founders are @_agamble and @HenrySG. Thanks in advance!	1 year ago
Austin	c9a0f24646	Add verbose parameter for llamacpp (#7253 ) Title: Add verbose parameter for llamacpp Description: This pull request adds a 'verbose' parameter to the llamacpp module. The 'verbose' parameter, when set to True, will enable the output of detailed logs during the execution of the Llama model. This added parameter can aid in debugging and understanding the internal processes of the module. The verbose parameter is a boolean that prints verbose output to stderr when set to True. By default, the verbose parameter is set to True but can be toggled off if less output is desired. This new parameter has been added to the `validate_environment` method of the `LlamaCpp` class which initializes the `llama_cpp.Llama` API: ```python class LlamaCpp(LLM): ... @root_validator() def validate_environment(cls, values: Dict) -> Dict: ... model_param_names = [ ... "verbose", # New verbose parameter added ] ... values["client"] = Llama(model_path, **model_params) ... ``` --------- Signed-off-by: teleprint-me <77757836+teleprint-me@users.noreply.github.com>	1 year ago
Kenny	34a2755a54	Allow passing api key into OpenAIWhisperParser (#7281 ) This just allows the user to pass in an api_key directly into OpenAIWhisperParser. Very simple addition.	1 year ago
mrkhalil6	4e7d0c115b	Add support for filters and namespaces in similarity search in Pinecone similarity_score_threshold (#7301 ) At the moment, pinecone vectorStore does not support filters and namespaces when using similarity_score_threshold search type. In this PR, I've implemented that. It passes all the kwargs except "score_threshold" as that is not a supported argument for method "similarity_search_with_score". --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Manuel Saelices	01dca1e438	Add context to an output parsing error on Pydantic schema to improve exception handling (#7344 ) ## Changes - [X] Fill the `llm_output` param when there is an output parsing error in a Pydantic schema so that we can get the original text that failed to parse when handling the exception ## Background With this change, we could do something like this: ``` output_parser = PydanticOutputParser(pydantic_object=pydantic_obj) chain = ConversationChain(..., output_parser=output_parser) try: response: PydanticSchema = chain.predict(input=input) except OutputParserException as exc: logger.error( 'OutputParserException while parsing chatbot response: %s', exc.llm_output, ) ``` --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Raouf Chebri	1ac6deda89	update extension name (#7359 ) hi @rlancemartin , We had a new deployment and the `pg_extension` creation command was updated from `CREATE EXTENSION pg_embedding` to `CREATE EXTENSION embedding`. https://github.com/neondatabase/neon/pull/4646 The extension not made public yet. No users will be affected by this. Will be public next week. Please let me know if you have any questions. Thank you in advance 🙏	1 year ago
William FH	4e180dc54e	Unset Cache in Tests (#7362 ) This is impacting other unit tests that use callbacks since the cache is still set (just empty)	1 year ago
German Martin	3ce4e46c8c	The Fellowship of the Vectors: New Embeddings Filter using clustering. (#7015 ) Continuing with Tolkien inspired series of langchain tools. I bring to you: The Fellowship of the Vectors, AKA EmbeddingsClusteringFilter. This document filter uses embeddings to group vectors together into clusters, then allows you to pick an arbitrary number of documents vector based on proximity to the cluster centers. That's a representative sample of the cluster. The original idea is from [Greg Kamradt](https://github.com/gkamradt) from this video (Level4): https://www.youtube.com/watch?v=qaPMdcCqtWk&t=365s I added few tricks to make it a bit more versatile, so you can parametrize what to do with duplicate documents in case of cluster overlap: replace the duplicates with the next closest document or remove it. This allow you to use it as an special kind of redundant filter too. Additionally you can choose 2 diff orders: grouped by cluster or respecting the original retriever scores. In my use case I was using the docs grouped by cluster to run refine chains per cluster to generate summarization over a large corpus of documents. Let me know if you want to change anything! @rlancemartin, @eyurtsev, @hwchase17, --------- Co-authored-by: rlm <pexpresss31@gmail.com>	1 year ago
Leonid Ganeline	b489466488	docs: `dependents` update 4 (#7360 ) Updated links and counters of the `dependents` page.	1 year ago
William FH	38ca5c84cb	Explicitly list requires_reference in function (#7357 )	1 year ago
Harrison Chase	49b2b0e3c0	change embedding to None (#7355 )	1 year ago
imaprogrammer	a2830e3056	Update chroma.py: Persist directory from client_settings if provided there (#7087 ) Change details: - Description: When calling db.persist(), a check prevents from it proceeding as the constructor only sets member `_persist_directory` from parameters. But the ChromaDB client settings also has this parameter, and if the client_settings parameter is used without passing the persist_directory (which is optional), the `persist` method raises `ValueError` for not setting `_persist_directory`. This change fixes it by setting the member `_persist_directory` variable from client_settings if it is set, else uses the constructor parameter. - Issue: I didn't find any github issue of this, but I discovered it after calling the persist method - Dependencies: None - Tag maintainer: vectorstore related change - @rlancemartin, @eyurtsev - Twitter handle: Don't have one :( Additional discussion: We may need to discuss the way I implemented the fallback using `or`. --------- Co-authored-by: rlm <pexpresss31@gmail.com>	1 year ago
Bagatur	cb4e88e4fb	bump 227 (#7354 )	1 year ago
Bagatur	d1c7237034	openai fn update nb (#7352 )	1 year ago
Bagatur	0ed2da7020	bump 226 (#7335 )	1 year ago

1 2 3 4 5 ...

3015 Commits (d2cf0d16b3e814e2bc86472e309c7ad81bf36cdb) All Branches Search

3015 Commits (d2cf0d16b3e814e2bc86472e309c7ad81bf36cdb)

All Branches