langchain

Commit Graph

Author	SHA1	Message	Date
Eugene Yurtsev	0470198fb5	Remove packages for pydantic compatibility (#9217 ) # Poetry updates This PR updates LangChains poetry file to remove any dependencies that aren't pydantic v2 compatible yet. All packages remain usable under pydantic v1, and can be installed separately. ## Bumping the following packages: * langsmith ## Removing the following packages not used in extended unit-tests: * zep-python, anthropic, jina, spacy, steamship, betabageldb not used at all: * octoai-sdk Cleaning up extras w/ for removed packages. ## Snapshots updated Some snapshots had to be updated due to a change in the data model in langsmith. RunType used to be Union of Enum and string and was changed to be string only.	1 year ago
Bagatur	e986afa13a	bump 265 (#9253 )	1 year ago
Hech	4b505060bd	fix: max_marginal_relevance_search and docs in Dingo (#9244 )	1 year ago
axiangcoding	664ff28cba	feat(llms): support ernie chat (#9114 ) Description: support ernie (文心一言) chat model Related issue: #7990 Dependencies: None Tag maintainer: @baskaryan	1 year ago
Bharat Ramanathan	08a8363fc6	feat(integration): Add support to serialize protobufs in WandbTracer (#8914 ) This PR adds serialization support for protocol bufferes in `WandbTracer`. This allows code generation chains to be visualized. Additionally, it also fixes a minor bug where the settings are not honored when a run is initialized before using the `WandbTracer` @agola11 --------- Co-authored-by: Bharat Ramanathan <ramanathan.parameshwaran@gohuddl.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Joshua Sundance Bailey	ef0664728e	ArcGISLoader update (#9240 ) Small bug fixes and added metadata based on user feedback. This PR is from the author of https://github.com/langchain-ai/langchain/pull/8873 .	1 year ago
Joseph McElroy	eac4ddb4bb	Elasticsearch Store Improvements (#8636 ) Todo: - [x] Connection options (cloud, localhost url, es_connection) support - [x] Logging support - [x] Customisable field support - [x] Distance Similarity support - [x] Metadata support - [x] Metadata Filter support - [x] Retrieval Strategies - [x] Approx - [x] Approx with Hybrid - [x] Exact - [x] Custom - [x] ELSER (excluding hybrid as we are working on RRF support) - [x] integration tests - [x] Documentation 👋 this is a contribution to improve Elasticsearch integration with Langchain. Its based loosely on the changes that are in master but with some notable changes: ## Package name & design improvements The import name is now `ElasticsearchStore`, to aid discoverability of the VectorStore. ```py ## Before from langchain.vectorstores.elastic_vector_search import ElasticVectorSearch, ElasticKnnSearch ## Now from langchain.vectorstores.elasticsearch import ElasticsearchStore ``` ## Retrieval Strategy support Before we had a number of classes, depending on the strategy you wanted. `ElasticKnnSearch` for approx, `ElasticVectorSearch` for exact / brute force. With `ElasticsearchStore` we have retrieval strategies: ### Approx Example Default strategy for the vast majority of developers who use Elasticsearch will be inferring the embeddings from outside of Elasticsearch. Uses KNN functionality of _search. ```py texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), es_url="http://localhost:9200", index_name="sample-index" ) output = docsearch.similarity_search("foo", k=1) ``` ### Approx, with hybrid Developers who want to search, using both the embedding and the text bm25 match. Its simple to enable. ```py texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), es_url="http://localhost:9200", index_name="sample-index", strategy=ElasticsearchStore.ApproxRetrievalStrategy(hybrid=True) ) output = docsearch.similarity_search("foo", k=1) ``` ### Approx, with `query_model_id` Developers who want to infer within Elasticsearch, using the model loaded in the ml node. This relies on the developer to setup the pipeline and index if they wish to embed the text in Elasticsearch. Example of this in the test. ```py texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), es_url="http://localhost:9200", index_name="sample-index", strategy=ElasticsearchStore.ApproxRetrievalStrategy( query_model_id="sentence-transformers__all-minilm-l6-v2" ), ) output = docsearch.similarity_search("foo", k=1) ``` ### I want to provide my own custom Elasticsearch Query You might want to have more control over the query, to perform multi-phase retrieval such as LTR, linearly boosting on document parameters like recently updated or geo-distance. You can do this with `custom_query_fn` ```py def my_custom_query(query_body: dict, query: str) -> dict: return {"query": {"match": {"text": {"query": "bar"}}}} texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), **elasticsearch_connection, index_name=index_name ) docsearch.similarity_search("foo", k=1, custom_query=my_custom_query) ``` ### Exact Example Developers who have a small dataset in Elasticsearch, dont want the cost of indexing the dims vs tradeoff on cost at query time. Uses script_score. ```py texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), es_url="http://localhost:9200", index_name="sample-index", strategy=ElasticsearchStore.ExactRetrievalStrategy(), ) output = docsearch.similarity_search("foo", k=1) ``` ### ELSER Example Elastic provides its own sparse vector model called ELSER. With these changes, its really easy to use. The vector store creates a pipeline and index thats setup for ELSER. All the developer needs to do is configure, ingest and query via langchain tooling. ```py texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), es_url="http://localhost:9200", index_name="sample-index", strategy=ElasticsearchStore.SparseVectorStrategy(), ) output = docsearch.similarity_search("foo", k=1) ``` ## Architecture In future, we can introduce new strategies and allow us to not break bwc as we evolve the index / query strategy. ## Credit On release, could you credit @elastic and @phoey1 please? Thank you! --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Divyansh Garg	9529483c2a	Improve MultiOn client toolkit prompts (#9222 ) - Updated prompts for the MultiOn toolkit for better functionality - Non-blocking but good to have it merged to improve the overall performance for the toolkit @hinthornw @hwchase17 --------- Co-authored-by: Naman Garg <ngarg3@binghamton.edu>	1 year ago
William FH	c478fc208e	Default On Retry (#9230 ) Base callbacks don't have a default on retry event Fix #8542 --------- Co-authored-by: landonsilla <landon.silla@stepstone.com>	1 year ago
Leonid Ganeline	93dd499997	docstrings: `document_loaders` consistency 3 (#9216 ) Updated docstrings into the consistent format (probably, the last update for the `document_loaders`.	1 year ago
Kshitij Wadhwa	a69cb95850	track langchain usage for Rockset (#9229 ) Add ability to track langchain usage for Rockset. Rockset's new python client allows setting this. To prevent old clients from failing, it ignore if setting throws exception (we can't track old versions) Tested locally with old and new Rockset python client cc @baskaryan	1 year ago
Leonid Ganeline	7810ea5812	docstrings: `chat_models` consistency (#9227 ) Updated docstrings into the consistent format.	1 year ago
William FH	b0896210c7	Return feedback with failed response if there's an error (#9223 ) In Evals	1 year ago
William FH	7124f2ebfa	Parent Doc Retriever (#9214 ) 2 things: - Implement the private method rather than the public one so callbacks are handled properly - Add search_kwargs (Open to not adding this if we are trying to deprecate this UX but seems like as a user i'd assume similar args to the vector store retriever. In fact some may assume this implements the same interface but I'm not dealing with that here) -	1 year ago
Harrison Chase	3f601b5809	add async method in (#9204 )	1 year ago
Clark	03ea0762a1	fix(jinachat): related to #9197 (#9200 ) related to: https://github.com/langchain-ai/langchain/issues/9197 --------- Co-authored-by: qianjun.wqj <qianjun.wqj@alibaba-inc.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Eugene Yurtsev	4f1feaca83	Wrap OpenAPI features in conditionals for pydantic v2 compatibility (#9205 ) Wrap OpenAPI in conditionals for pydantic v2 compatibility.	1 year ago
Glauco Custódio	89be10f6b4	add ttl to RedisCache (#9068 ) Add `ttl` (time to live) to `RedisCache`	1 year ago
Eugene Yurtsev	04bc5f3b18	Conditionally add pydantic v1 to namespace (#9202 ) Conditionally add pydantic_v1 to namespace.	1 year ago
shibuiwilliam	feec422bf7	fix logging to logger (#9192 ) # What - fix logging to logger	1 year ago
Bagatur	5935767056	bump lc 246, lce 9 (#9207 )	1 year ago
Bagatur	b5a57acf6c	lite llm lint (#9208 )	1 year ago
Krish Dholakia	49f1d8477c	Adding ChatLiteLLM model (#9020 ) Description: Adding a langchain integration for the LiteLLM library Tag maintainer: @hwchase17, @baskaryan Twitter handle: @krrish_dh / @Berri_AI --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Eugene Yurtsev	72f9150a50	Update 2 more pydantic imports (#9203 ) Update two more pydantic imports to use v1 explicitly	1 year ago
Eugene Yurtsev	c172f972ea	Create pydantic v1 namespace, add partial compatibility for pydantic v2 (#9123 ) First of a few PRs to add full compatibility to both pydantic v1 and v2. This PR creates pydantic v1 namespace and adds it to sys.modules. Upcoming changes: 1. Handle `openapi-schema-pydantic = "^1.2"` and dependent chains/tools 2. bump dependencies to versions that are cross compatible for pydantic or remove them (see below) 3. Add tests to github workflows to test with pydantic v1 and v2 Dependencies From a quick look (could be wrong since was done manually) dependencies pinning pydantic below 2 (some of these can be bumped to newer versions are provide cross-compatible code) anthropic bentoml confection fastapi langsmith octoai-sdk openapi-schema-pydantic qdrant-client spacy steamship thinc zep-python Unpinned marqo () nomic () xinference(*)	1 year ago
Evan Schultz	8189dea0d8	Fixes typing issues in BaseOpenAI (#9183 ) ## Description: Sets default values for `client` and `model` attributes in the BaseOpenAI class to fix Pylance Typing issue. - Issue: #9182. - Twitter handle: @evanmschultz	1 year ago
Massimiliano Pronesti	d95eeaedbe	feat(llms): support vLLM's OpenAI-compatible server (#9179 ) This PR aims at supporting [vLLM's OpenAI-compatible server feature](https://vllm.readthedocs.io/en/latest/getting_started/quickstart.html#openai-compatible-server), i.e. allowing to call vLLM's LLMs like if they were OpenAI's. I've also udpated the related notebook providing an example usage. At the moment, vLLM only supports the `Completion` API.	1 year ago
Michael Goin	621da3c164	Adds DeepSparse as an LLM (#9184 ) Adds [DeepSparse](https://github.com/neuralmagic/deepsparse) as an LLM backend. DeepSparse supports running various open-source sparsified models hosted on [SparseZoo](https://sparsezoo.neuralmagic.com/) for performance gains on CPUs. Twitter handles: @mgoin_ @neuralmagic --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	0fa69d8988	Bagatur/zep python 1.0 (#9186 ) Co-authored-by: Daniel Chalef <131175+danielchalef@users.noreply.github.com>	1 year ago
Eugene Yurtsev	9b24f0b067	Enhance deprecation decorator to modify docs with sphinx directives (#9069 ) Enhance deprecation decorator	1 year ago
Bagatur	cdfe2c96c5	bump 263 (#9156 )	1 year ago
Leonid Ganeline	19f504790e	docstrings: document_loaders consitency 2 (#9148 ) This is Part 2. See #9139 (Part 1).	1 year ago
Harrison Chase	1b58460fe3	update keys for chain (#5164 ) Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
胡亮	7edf4ca396	Support multi gpu inference for HuggingFaceEmbeddings (#4732 ) Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
UmerHA	8aab39e3ce	Added SmartGPT workflow (issue #4463 ) (#4816 ) # Added SmartGPT workflow by providing SmartLLM wrapper around LLMs Edit: As @hwchase17 suggested, this should be a chain, not an LLM. I have adapted the PR. It is used like this: ``` from langchain.prompts import PromptTemplate from langchain.chains import SmartLLMChain from langchain.chat_models import ChatOpenAI hard_question = "I have a 12 liter jug and a 6 liter jug. I want to measure 6 liters. How do I do it?" hard_question_prompt = PromptTemplate.from_template(hard_question) llm = ChatOpenAI(model_name="gpt-4") prompt = PromptTemplate.from_template(hard_question) chain = SmartLLMChain(llm=llm, prompt=prompt, verbose=True) chain.run({}) ``` Original text: Added SmartLLM wrapper around LLMs to allow for SmartGPT workflow (as in https://youtu.be/wVzuvf9D9BU). SmartLLM can be used wherever LLM can be used. E.g: ``` smart_llm = SmartLLM(llm=OpenAI()) smart_llm("What would be a good company name for a company that makes colorful socks?") ``` or ``` smart_llm = SmartLLM(llm=OpenAI()) prompt = PromptTemplate( input_variables=["product"], template="What is a good name for a company that makes {product}?", ) chain = LLMChain(llm=smart_llm, prompt=prompt) chain.run("colorful socks") ``` SmartGPT consists of 3 steps: 1. Ideate - generate n possible solutions ("ideas") to user prompt 2. Critique - find flaws in every idea & select best one 3. Resolve - improve upon best idea & return it Fixes #4463 ## Who can review? Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested: - @hwchase17 - @agola11 Twitter: [@UmerHAdil](https://twitter.com/@UmerHAdil) \| Discord: RicChilligerDude#7589 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Lucas Pickup	1d3735a84c	Ensure deployment_id is set to provided deployment, required for Azure OpenAI. (#5002 ) # Ensure deployment_id is set to provided deployment, required for Azure OpenAI. --------- Co-authored-by: Lucas Pickup <lupickup@microsoft.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	45741bcc1b	Bagatur/vectara nit (#9140 ) Co-authored-by: Ofer Mendelevitch <ofer@vectara.com>	1 year ago
Dominick DEV	9b64932e55	Add LangChain utility for real-time crypto exchange prices (#4501 ) This commit adds the LangChain utility which allows for the real-time retrieval of cryptocurrency exchange prices. With LangChain, users can easily access up-to-date pricing information by running the command ".run(from_currency, to_currency)". This new feature provides a convenient way to stay informed on the latest exchange rates and make informed decisions when trading crypto. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Joshua Sundance Bailey	eaa505fb09	Create ArcGISLoader & example notebook (#8873 ) - Description: Adds the ArcGISLoader class to `langchain.document_loaders` - Allows users to load data from ArcGIS Online, Portal, and similar - Users can authenticate with `arcgis.gis.GIS` or retrieve public data anonymously - Uses the `arcgis.features.FeatureLayer` class to retrieve the data - Defines the most relevant keywords arguments and accepts `**kwargs` - Dependencies: Using this class requires `arcgis` and, optionally, `bs4.BeautifulSoup`. Tagging maintainers: - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	e21152358a	fix (#9145 )	1 year ago
Leonid Ganeline	edb585228d	docstrings: document_loaders consitency (#9139 ) Formatted docstrings from different formats to consistent format, lile: >Loads processed docs from Docugami. "Load from `Docugami`." >Loader that uses Unstructured to load HTML files. "Load `HTML` files using `Unstructured`." >Load documents from a directory. "Load from a directory." - `Load` - no `Loads` - DocumentLoader always loads Documents, so no more "documents/docs/texts/ etc" - integrated systems and APIs enclosed in backticks,	1 year ago
Markus Schiffer	00bf472265	Fix for SVM retriever discarding document metadata (#9141 ) As stated in the title the SVM retriever discarded the metadata of passed in docs. This code fixes that. I also added one unit test that should test that. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	bace17e0aa	rm integration deps (#9142 )	1 year ago
Eugene Yurtsev	44bc89b7bf	Support a few list like operations on ChatPromptTemplate (#9077 ) Make it easier to work with chat prompt template	1 year ago
Hai The Dude	e4418d1b7e	Added new use case docs for Web Scraping, Chromium loader, BS4 transformer (#8732 ) - Description: Added a new use case category called "Web Scraping", and a tutorial to scrape websites using OpenAI Functions Extraction chain to the docs. - Tag maintainer:@baskaryan @hwchase17 , - Twitter handle: https://www.linkedin.com/in/haiphunghiem/ (I'm on LinkedIn mostly) --------- Co-authored-by: Lance Martin <lance@langchain.dev>	1 year ago
sseide	6cb763507c	add basic support for redis cluster server (#9128 ) This change updates the central utility class to recognize a Redis cluster server after connection and returns an new cluster aware Redis client. The "normal" Redis client would not be able to talk to a cluster node because keys might be stored on other shards of the Redis cluster and therefor not readable or writable. With this patch clients do not need to know what Redis server it is, they just connect though the same API calls for standalone and cluster server. There are no dependencies added due to this MR. Remark - with current redis-py client library (4.6.0) a cluster cannot be used as VectorStore. It can be used for other use-cases. There is a bug / missing feature(?) in the Redis client breaking the VectorStore implementation. I opened an issue at the client library too (redis/redis-py#2888) to fix this. As soon as this is fixed in `redis-py` library it should be usable there too. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
David Duong	6d03f8b5d8	Add serialisable support for Replicate (#8525 )	1 year ago
niklub	16af5f8690	Add LabelStudio integration (#8880 ) This PR introduces [Label Studio](https://labelstud.io/) integration with LangChain via `LabelStudioCallbackHandler`: - sending data to the Label Studio instance - labeling dataset for supervised LLM finetuning - rating model responses - tracking and displaying chat history - support for custom data labeling workflow ### Example ``` chat_llm = ChatOpenAI(callbacks=[LabelStudioCallbackHandler(mode="chat")]) chat_llm([ SystemMessage(content="Always use emojis in your responses."), HumanMessage(content="Hey AI, how's your day going?"), AIMessage(content="🤖 I don't have feelings, but I'm running smoothly! How can I help you today?"), HumanMessage(content="I'm feeling a bit down. Any advice?"), AIMessage(content="🤗 I'm sorry to hear that. Remember, it's okay to seek help or talk to someone if you need to. 💬"), HumanMessage(content="Can you tell me a joke to lighten the mood?"), AIMessage(content="Of course! 🎭 Why did the scarecrow win an award? Because he was outstanding in his field! 🌾"), HumanMessage(content="Haha, that was a good one! Thanks for cheering me up."), AIMessage(content="Always here to help! 😊 If you need anything else, just let me know."), HumanMessage(content="Will do! By the way, can you recommend a good movie?"), ]) ``` <img width="906" alt="image" src="https://github.com/langchain-ai/langchain/assets/6087484/0a1cf559-0bd3-4250-ad96-6e71dbb1d2f3"> ### Dependencies - [label-studio](https://pypi.org/project/label-studio/) - [label-studio-sdk](https://pypi.org/project/label-studio-sdk/) https://twitter.com/labelstudiohq --------- Co-authored-by: nik <nik@heartex.net>	1 year ago
Bagatur	8cb2594562	Bagatur/dingo (#9079 ) Co-authored-by: gary <1625721671@qq.com>	1 year ago
Jacques Arnoux	926c64da60	Fix web research retriever for unknown links in results (#9115 ) Fixes an issue with web research retriever for unknown links in results. This is currently making the retrieve crash sometimes. @rlancemartin	1 year ago

1 2 3 4 5 ...

394 Commits (0a04e63811f3e531956cc6de85291333b3eaf5b7)