langchain

Commit Graph

Author	SHA1	Message	Date
Bagatur	9abf60acb6	Bagatur/vectara regression (#9276 ) Co-authored-by: Ofer Mendelevitch <ofer@vectara.com> Co-authored-by: Ofer Mendelevitch <ofermend@gmail.com>	1 year ago
Xiaoyu Xee	b30f449dae	Add dashvector vectorstore (#9163 ) ## Description Add `Dashvector` vectorstore for langchain - [dashvector quick start](https://help.aliyun.com/document_detail/2510223.html) - [dashvector package description](https://pypi.org/project/dashvector/) ## How to use ```python from langchain.vectorstores.dashvector import DashVector dashvector = DashVector.from_documents(docs, embeddings) ``` --------- Co-authored-by: smallrain.xuxy <smallrain.xuxy@alibaba-inc.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Kunj-2206	1b3942ba74	Added BittensorLLM (#9250 ) Description: Adding NIBittensorLLM via Validator Endpoint to langchain llms Tag maintainer: @Kunj-2206 Maintainer responsibilities: Models / Prompts: @hwchase17, @baskaryan --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Toshish Jawale	852722ea45	Improvements in Nebula LLM (#9226 ) - Description: Added improvements in Nebula LLM to perform auto-retry; more generation parameters supported. Conversation is no longer required to be passed in the LLM object. Examples are updated. - Issue: N/A - Dependencies: N/A - Tag maintainer: @baskaryan - Twitter handle: symbldotai --------- Co-authored-by: toshishjawale <toshish@symbl.ai>	1 year ago
Anthony Mahanna	0a04e63811	docs: Update ArangoDB Links (#9251 ) ready for review - mdx link update - colab link update	1 year ago
Joseph McElroy	eac4ddb4bb	Elasticsearch Store Improvements (#8636 ) Todo: - [x] Connection options (cloud, localhost url, es_connection) support - [x] Logging support - [x] Customisable field support - [x] Distance Similarity support - [x] Metadata support - [x] Metadata Filter support - [x] Retrieval Strategies - [x] Approx - [x] Approx with Hybrid - [x] Exact - [x] Custom - [x] ELSER (excluding hybrid as we are working on RRF support) - [x] integration tests - [x] Documentation 👋 this is a contribution to improve Elasticsearch integration with Langchain. Its based loosely on the changes that are in master but with some notable changes: ## Package name & design improvements The import name is now `ElasticsearchStore`, to aid discoverability of the VectorStore. ```py ## Before from langchain.vectorstores.elastic_vector_search import ElasticVectorSearch, ElasticKnnSearch ## Now from langchain.vectorstores.elasticsearch import ElasticsearchStore ``` ## Retrieval Strategy support Before we had a number of classes, depending on the strategy you wanted. `ElasticKnnSearch` for approx, `ElasticVectorSearch` for exact / brute force. With `ElasticsearchStore` we have retrieval strategies: ### Approx Example Default strategy for the vast majority of developers who use Elasticsearch will be inferring the embeddings from outside of Elasticsearch. Uses KNN functionality of _search. ```py texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), es_url="http://localhost:9200", index_name="sample-index" ) output = docsearch.similarity_search("foo", k=1) ``` ### Approx, with hybrid Developers who want to search, using both the embedding and the text bm25 match. Its simple to enable. ```py texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), es_url="http://localhost:9200", index_name="sample-index", strategy=ElasticsearchStore.ApproxRetrievalStrategy(hybrid=True) ) output = docsearch.similarity_search("foo", k=1) ``` ### Approx, with `query_model_id` Developers who want to infer within Elasticsearch, using the model loaded in the ml node. This relies on the developer to setup the pipeline and index if they wish to embed the text in Elasticsearch. Example of this in the test. ```py texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), es_url="http://localhost:9200", index_name="sample-index", strategy=ElasticsearchStore.ApproxRetrievalStrategy( query_model_id="sentence-transformers__all-minilm-l6-v2" ), ) output = docsearch.similarity_search("foo", k=1) ``` ### I want to provide my own custom Elasticsearch Query You might want to have more control over the query, to perform multi-phase retrieval such as LTR, linearly boosting on document parameters like recently updated or geo-distance. You can do this with `custom_query_fn` ```py def my_custom_query(query_body: dict, query: str) -> dict: return {"query": {"match": {"text": {"query": "bar"}}}} texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), **elasticsearch_connection, index_name=index_name ) docsearch.similarity_search("foo", k=1, custom_query=my_custom_query) ``` ### Exact Example Developers who have a small dataset in Elasticsearch, dont want the cost of indexing the dims vs tradeoff on cost at query time. Uses script_score. ```py texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), es_url="http://localhost:9200", index_name="sample-index", strategy=ElasticsearchStore.ExactRetrievalStrategy(), ) output = docsearch.similarity_search("foo", k=1) ``` ### ELSER Example Elastic provides its own sparse vector model called ELSER. With these changes, its really easy to use. The vector store creates a pipeline and index thats setup for ELSER. All the developer needs to do is configure, ingest and query via langchain tooling. ```py texts = ["foo", "bar", "baz"] docsearch = ElasticsearchStore.from_texts( texts, FakeEmbeddings(), es_url="http://localhost:9200", index_name="sample-index", strategy=ElasticsearchStore.SparseVectorStrategy(), ) output = docsearch.similarity_search("foo", k=1) ``` ## Architecture In future, we can introduce new strategies and allow us to not break bwc as we evolve the index / query strategy. ## Credit On release, could you credit @elastic and @phoey1 please? Thank you! --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Michael Goin	621da3c164	Adds DeepSparse as an LLM (#9184 ) Adds [DeepSparse](https://github.com/neuralmagic/deepsparse) as an LLM backend. DeepSparse supports running various open-source sparsified models hosted on [SparseZoo](https://sparsezoo.neuralmagic.com/) for performance gains on CPUs. Twitter handles: @mgoin_ @neuralmagic --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	45741bcc1b	Bagatur/vectara nit (#9140 ) Co-authored-by: Ofer Mendelevitch <ofer@vectara.com>	1 year ago
Bagatur	8cb2594562	Bagatur/dingo (#9079 ) Co-authored-by: gary <1625721671@qq.com>	1 year ago
Chenyu Zhao	c0acbdca1b	Update Fireworks model names (#9085 )	1 year ago
Bidhan Roy	02430e25b6	BagelDB (bageldb.ai), VectorStore integration. (#8971 ) - Description: [BagelDB](bageldb.ai) a collaborative vector database. Integrated the bageldb PyPi package with langchain with related tests and code. - Issue: Not applicable. - Dependencies: `betabageldb` PyPi package. - Tag maintainer: @rlancemartin, @eyurtsev, @baskaryan - Twitter handle: bageldb_ai (https://twitter.com/BagelDB_ai) We ran `make format`, `make lint` and `make test` locally. Followed the contribution guideline thoroughly https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --------- Co-authored-by: Towhid1 <nurulaktertowhid@gmail.com>	1 year ago
Luca Foppiano	dfb93dd2b5	Improved grobid documentation (#9025 ) - Description: Improvement in the Grobid loader documentation, typos and suggesting to use the docker image instead of installing Grobid in local (the documentation was also limited to Mac, while docker allow running in any platform) - Tag maintainer: @rlancemartin, @eyurtsev - Twitter handle: @whitenoise	1 year ago
arjunbansal	a2681f950d	add instructions on integrating Log10 (#8938 ) - Description: Instruction for integration with Log10: an [open source](https://github.com/log10-io/log10) proxiless LLM data management and application development platform that lets you log, debug and tag your Langchain calls - Tag maintainer: @baskaryan - Twitter handle: @log10io @coffeephoenix Several examples showing the integration included [here](https://github.com/log10-io/log10/tree/main/examples/logging) and in the PR	1 year ago
Aarav Borthakur	3f64b8a761	Integrate Rockset as a chat history store (#8940 ) Description: Adds Rockset as a chat history store Dependencies: no changes Tag maintainer: @hwchase17 This PR passes linting and testing. I added a test for the integration and an example notebook showing its use.	1 year ago
Leonid Ganeline	33a2f58fbf	`tensoflow_datasets` document loader (#8721 ) This PR adds `tensoflow_datasets` document loader	1 year ago
Leonid Ganeline	2d078c7767	`PubMed` document loader (#8893 ) - added `PubMed Document Loader` artifacts; ut-s; examples - fixed `PubMed utility`; ut-s @hwchase17	1 year ago
Maurits de Groot	61c2d918c6	Fixed inaccurate import in integrations:providers:bedrock documentation (#8915 ) Description: Fixed inaccurate import in integrations:providers:bedrock documentation In the current version of the bedrock documentation, page https://python.langchain.com/docs/integrations/providers/bedrock it states that the import is from langchain import Bedrock This has been changed to from langchain.llms.bedrock import Bedrock as stated in https://python.langchain.com/docs/integrations/llms/bedrock Issue: Not applicable Dependencies No dependencies required Tag maintainer @baskaryan Twitter handle: Not applicable	1 year ago
David vonThenen	40079d4936	Introduce Nebula LLM to LangChain (#8876 ) ## Description This PR adds Nebula to the available LLMs in LangChain. Nebula is an LLM focused on conversation understanding and enables users to extract conversation insights from video, audio, text, and chat-based conversations. These conversations can occur between any mix of human or AI participants. Examples of some questions you could ask Nebula from a given conversation are: - What could be the customer’s pain points based on the conversation? - What sales opportunities can be identified from this conversation? - What best practices can be derived from this conversation for future customer interactions? You can read more about Nebula here: https://symbl.ai/blog/extract-insights-symbl-ai-generative-ai-recall-ai-meetings/ #### Integration Test An integration test is added, but it requires network access. Since Nebula is fully managed like OpenAI, network access is required to exercise the integration test. #### Linting - [x] make lint - [x] make test (TODO: there seems to be a failure in another non-related test??? Need to check on this.) - [x] make format ### Dependencies No new dependencies were introduced. ### Twitter handle [@symbldotai](https://twitter.com/symbldotai) [@dvonthenen](https://twitter.com/dvonthenen) If you have any questions, please let me know. cc: @hwchase17, @baskaryan --------- Co-authored-by: dvonthenen <david.vonthenen@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
rjanardhan3	affaaea87b	Updates fireworks (#8765 ) <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: Updates to Fireworks Documentation, - Issue: N/A, - Dependencies: N/A, - Tag maintainer: @rlancemartin, --------- Co-authored-by: Raj Janardhan <rajjanardhan@Rajs-Laptop.attlocal.net>	1 year ago
Leonid Ganeline	1335f2b9f8	`MLflow` examples (#8642 ) Updated `MLflow` examples with links to the examples from MLflow @baskaryan	1 year ago
rjanardhan3	68113348cc	Fireworks integration (#8322 ) Description - Integrates Fireworks within Langchain LLMs to allow users to use Fireworks models with Langchain, mainly for summarization. Issue - Not applicable Dependencies - None Tag maintainer - @rlancemartin --------- Co-authored-by: Raj Janardhan <rajjanardhan@Rajs-Laptop.attlocal.net>	1 year ago
Matt Robinson	8961c720b8	docs: update `unstructured` install instructions (#8596 ) ### Summary Updates the `unstructured` install instructions. For `unstructured>=0.9.0`, dependencies are broken out by document type and the base `unstructured` package includes fewer dependencies. `pip install "unstructured[local-inference]"` has been replace by `pip install "unstructured[all-docs]"`, though the `local-inference` extra is still supported for the time being. ### Reviewers - @rlancemartin - @eyurtsev - @hwchase17	1 year ago
Bagatur	73072d3db8	mv (#8595 )	1 year ago
Tesfagabir Meharizghi	a7000ee89e	Callback handler for Amazon SageMaker Experiments (#8587 ) ## Description This PR implements a callback handler for SageMaker Experiments which is similar to that of mlflow. * When creating the callback handler, it takes the experiment's run object as an argument. All the callback outputs are then logged to the run object. * The output of each callback action (e.g., `on_llm_start`) is saved to S3 bucket as json file. * Optionally, you can also log additional information such as the LLM hyper-parameters to the same run object. * Once the callback object is no more needed, you will need to call the `flush_tracker()` method. This makes sure that any intermediate files are deleted. * A separate notebook example is provided to show how the callback is used. @3coins @agola11 --------- Co-authored-by: Tesfagabir Meharizghi <mehariz@amazon.com>	1 year ago
William FH	b7c0eb9ecb	Wfh/ref links (#8454 )	1 year ago
HeTaoPKU	d5884017a9	Add Minimax llm model to langchain (#7645 ) - Description: Minimax is a great AI startup from China, recently they released their latest model and chat API, and the API is widely-spread in China. As a result, I'd like to add the Minimax llm model to Langchain. - Tag maintainer: @hwchase17, @baskaryan --------- Co-authored-by: the <tao.he@hulu.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Jiayi Ni	1efb9bae5f	FEAT: Integrate Xinference LLMs and Embeddings (#8171 ) - [Xorbits Inference(Xinference)](https://github.com/xorbitsai/inference) is a powerful and versatile library designed to serve language, speech recognition, and multimodal models. Xinference supports a variety of GGML-compatible models including chatglm, whisper, and vicuna, and utilizes heterogeneous hardware and a distributed architecture for seamless cross-device and cross-server model deployment. - This PR integrates Xinference models and Xinference embeddings into LangChain. - Dependencies: To install the depenedencies for this integration, run `pip install "xinference[all]"` - Example Usage: To start a local instance of Xinference, run `xinference`. To deploy Xinference in a distributed cluster, first start an Xinference supervisor using `xinference-supervisor`: `xinference-supervisor -H "${supervisor_host}"` Then, start the Xinference workers using `xinference-worker` on each server you want to run them on. `xinference-worker -e "http://${supervisor_host}:9997"` To use Xinference with LangChain, you also need to launch a model. You can use command line interface (CLI) to do so. Fo example: `xinference launch -n vicuna-v1.3 -f ggmlv3 -q q4_0`. This launches a model named vicuna-v1.3 with `model_format="ggmlv3"` and `quantization="q4_0"`. A model UID is returned for you to use. Now you can use Xinference with LangChain: ```python from langchain.llms import Xinference llm = Xinference( server_url="http://0.0.0.0:9997", # suppose the supervisor_host is "0.0.0.0" model_uid = {model_uid} # model UID returned from launching a model ) llm( prompt="Q: where can we visit in the capital of France? A:", generate_config={"max_tokens": 1024}, ) ``` You can also use RESTful client to launch a model: ```python from xinference.client import RESTfulClient client = RESTfulClient("http://0.0.0.0:9997") model_uid = client.launch_model(model_name="vicuna-v1.3", model_size_in_billions=7, quantization="q4_0") ``` The following code block demonstrates how to use Xinference embeddings with LangChain: ```python from langchain.embeddings import XinferenceEmbeddings xinference = XinferenceEmbeddings( server_url="http://0.0.0.0:9997", model_uid = model_uid ) ``` ```python query_result = xinference.embed_query("This is a test query") ``` ```python doc_result = xinference.embed_documents(["text A", "text B"]) ``` Xinference is still under rapid development. Feel free to [join our Slack community](https://xorbitsio.slack.com/join/shared_invite/zt-1z3zsm9ep-87yI9YZ_B79HLB2ccTq4WA) to get the latest updates! - Request for review: @hwchase17, @baskaryan - Twitter handle: https://twitter.com/Xorbitsio --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	68763bd25f	mv popular and additional chains to use cases (#8242 )	1 year ago
Aarav Borthakur	8ce661d5a1	Docs: Fix Rockset links (#8214 ) Fix broken Rockset links. Right now links at https://python.langchain.com/docs/integrations/providers/rockset are broken.	1 year ago
Emory Petermann	7734a2b5ab	update golden-query notebook and fix typo in golden docs (#8253 ) updating the documentation to be consistent for Golden query tool and have a better introduction to the tool	1 year ago
William FH	0a16b3d84b	Update Integrations links (#8206 )	1 year ago
Anthony Mahanna	76102971c0	ArangoDB/AQL support for Graph QA Chain (#7880 ) Description: Serves as an introduction to LangChain's support for [ArangoDB](https://github.com/arangodb/arangodb), similar to https://github.com/hwchase17/langchain/pull/7165 and https://github.com/hwchase17/langchain/pull/4881 Issue: No issue has been created for this feature Dependencies: `python-arango` has been added as an optional dependency via the `CONTRIBUTING.md` guidelines Twitter handle: [at]arangodb - Integration test has been added - Notebook has been added: [graph_arangodb_qa.ipynb](https://github.com/amahanna/langchain/blob/master/docs/extras/modules/chains/additional/graph_arangodb_qa.ipynb) [![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/amahanna/langchain/blob/master/docs/extras/modules/chains/additional/graph_arangodb_qa.ipynb) ``` docker run -p 8529:8529 -e ARANGO_ROOT_PASSWORD= arangodb/arangodb ``` ``` pip install git+https://github.com/amahanna/langchain.git ``` ```python from arango import ArangoClient from langchain.chat_models import ChatOpenAI from langchain.graphs import ArangoGraph from langchain.chains import ArangoGraphQAChain db = ArangoClient(hosts="localhost:8529").db(name="_system", username="root", password="", verify=True) graph = ArangoGraph(db) chain = ArangoGraphQAChain.from_llm(ChatOpenAI(temperature=0), graph=graph) chain.run("Is Ned Stark alive?") ``` --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	1a7d8667c8	Bagatur/gateway chat (#8198 ) Signed-off-by: dbczumar <corey.zumar@databricks.com> Co-authored-by: dbczumar <corey.zumar@databricks.com>	1 year ago
Bagatur	c8c8635dc9	mv module integrations docs (#8101 )	1 year ago

34 Commits (9417961b171f772caa965d773f866002daa60266)