10 Commits (52d6b91c18ca0c9374f9a465bc9af72547a55c38)

Author	SHA1	Message	Date
manmax31	40096c73cd	Add BGE embeddings support (#8848) ... - Description: [BGE-large](https://huggingface.co/BAAI/bge-large-en) embeddings from BAAI are at the top of [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard). Hence adding support for it. - Tag maintainer: @baskaryan - Twitter handle: @ManabChetia3 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	6c3573e7f6	Harrison/aleph alpha (#8735) ... Co-authored-by: PiotrMazurek <piotr.mazurek@aleph-alpha.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	b2b71b0d35	Bagatur/eden llm (#8670) ... Co-authored-by: RedhaWassim <rwasssim@gmail.com> Co-authored-by: KyrianC <ckyrian@protonmail.com> Co-authored-by: sam <melaine.samy@gmail.com>	1 year ago
Leonid Kuligin	b4a126ae71	Updated docs on Vertex AI going GA (#8531) ... #8074 Co-authored-by: Leonid Kuligin <kuligin@google.com>	1 year ago
William FH	b7c0eb9ecb	Wfh/ref links (#8454)	1 year ago
Jiayi Ni	1efb9bae5f	FEAT: Integrate Xinference LLMs and Embeddings (#8171) ... - [Xorbits Inference(Xinference)](https://github.com/xorbitsai/inference) is a powerful and versatile library designed to serve language, speech recognition, and multimodal models. Xinference supports a variety of GGML-compatible models including chatglm, whisper, and vicuna, and utilizes heterogeneous hardware and a distributed architecture for seamless cross-device and cross-server model deployment. - This PR integrates Xinference models and Xinference embeddings into LangChain. - Dependencies: To install the depenedencies for this integration, run `pip install "xinference[all]"` - Example Usage: To start a local instance of Xinference, run `xinference`. To deploy Xinference in a distributed cluster, first start an Xinference supervisor using `xinference-supervisor`: `xinference-supervisor -H "${supervisor_host}"` Then, start the Xinference workers using `xinference-worker` on each server you want to run them on. `xinference-worker -e "http://${supervisor_host}:9997"` To use Xinference with LangChain, you also need to launch a model. You can use command line interface (CLI) to do so. Fo example: `xinference launch -n vicuna-v1.3 -f ggmlv3 -q q4_0`. This launches a model named vicuna-v1.3 with `model_format="ggmlv3"` and `quantization="q4_0"`. A model UID is returned for you to use. Now you can use Xinference with LangChain: ```python from langchain.llms import Xinference llm = Xinference( server_url="http://0.0.0.0:9997", # suppose the supervisor_host is "0.0.0.0" model_uid = {model_uid} # model UID returned from launching a model ) llm( prompt="Q: where can we visit in the capital of France? A:", generate_config={"max_tokens": 1024}, ) ``` You can also use RESTful client to launch a model: ```python from xinference.client import RESTfulClient client = RESTfulClient("http://0.0.0.0:9997") model_uid = client.launch_model(model_name="vicuna-v1.3", model_size_in_billions=7, quantization="q4_0") ``` The following code block demonstrates how to use Xinference embeddings with LangChain: ```python from langchain.embeddings import XinferenceEmbeddings xinference = XinferenceEmbeddings( server_url="http://0.0.0.0:9997", model_uid = model_uid ) ``` ```python query_result = xinference.embed_query("This is a test query") ``` ```python doc_result = xinference.embed_documents(["text A", "text B"]) ``` Xinference is still under rapid development. Feel free to [join our Slack community](https://xorbitsio.slack.com/join/shared_invite/zt-1z3zsm9ep-87yI9YZ_B79HLB2ccTq4WA) to get the latest updates! - Request for review: @hwchase17, @baskaryan - Twitter handle: https://twitter.com/Xorbitsio --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Taozhi Wang	594f195e54	Add embeddings for AwaEmbedding (#8353) ... - Description: Adds AwaEmbeddings class for embeddings, which provides users with a convenient way to do fine-tuning, as well as the potential need for multimodality - Tag maintainer: @baskaryan Create `Awa.ipynb`: an example notebook for AwaEmbeddings class Modify `embeddings/__init__.py`: Import the class Create `embeddings/awa.py`: The embedding class Create `embeddings/test_awa.py`: The test file. --------- Co-authored-by: taozhiwang <taozhiwa@gmail.com>	1 year ago
Juan José Torres	1cc7d4c9eb	Update SageMaker Endpoint Embeddings docs to be up to date with current requirements (#8103) ... - **Description:** Simple change of the Class that ContentHandler inherits from. To create an object of type SagemakerEndpointEmbeddings, the property content_handler must be of type EmbeddingsContentHandler not ContentHandlerBase anymore, - **Twitter handle:** @Juanjo_Torres11 Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Ettore Di Giacinto	ae28568e2a	Add embeddings for LocalAI (#8134) ... Description: This PR adds embeddings for LocalAI ( https://github.com/go-skynet/LocalAI ), a self-hosted OpenAI drop-in replacement. As LocalAI can re-use OpenAI clients it is mostly following the lines of the OpenAI embeddings, however when embedding documents, it just uses string instead of sending tokens as sending tokens is best-effort depending on the model being used in LocalAI. Sending tokens is also tricky as token id's can mismatch with the model - so it's safer to just send strings in this case. Partly related to: https://github.com/hwchase17/langchain/issues/5256 Dependencies: No new dependencies Twitter: @mudler_it --------- Signed-off-by: mudler <mudler@localai.io> Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Bagatur	c8c8635dc9	mv module integrations docs (#8101)	1 year ago