From f6a05e964bdb141ca28846eaeac77d47b99c5b9a Mon Sep 17 00:00:00 2001 From: Leonid Ganeline Date: Wed, 24 Jan 2024 16:59:00 -0800 Subject: [PATCH] docs: `Hugging Face` update (#16490) - added missed integrations to the platform page - updated integration examples: added links and fixed formats --- docs/docs/integrations/chat/huggingface.ipynb | 14 ++-- .../integrations/platforms/huggingface.mdx | 65 ++++++++++++++----- .../text_embeddings_inference.ipynb | 12 ++-- 3 files changed, 64 insertions(+), 27 deletions(-) diff --git a/docs/docs/integrations/chat/huggingface.ipynb b/docs/docs/integrations/chat/huggingface.ipynb index c1608b035a..6a93ebf4ad 100644 --- a/docs/docs/integrations/chat/huggingface.ipynb +++ b/docs/docs/integrations/chat/huggingface.ipynb @@ -4,9 +4,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Hugging Face Chat Wrapper\n", + "# Hugging Face\n", "\n", - "This notebook shows how to get started using Hugging Face LLM's as chat models.\n", + "This notebook shows how to get started using `Hugging Face` LLM's as chat models.\n", "\n", "In particular, we will:\n", "1. Utilize the [HuggingFaceTextGenInference](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/llms/huggingface_text_gen_inference.py), [HuggingFaceEndpoint](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/llms/huggingface_endpoint.py), or [HuggingFaceHub](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/llms/huggingface_hub.py) integrations to instantiate an `LLM`.\n", @@ -49,7 +49,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "#### `HuggingFaceTextGenInference`" + "### `HuggingFaceTextGenInference`" ] }, { @@ -93,7 +93,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "#### `HuggingFaceEndpoint`" + "### `HuggingFaceEndpoint`" ] }, { @@ -121,7 +121,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "#### `HuggingFaceHub`" + "### `HuggingFaceHub`" ] }, { @@ -291,7 +291,7 @@ "source": [ "## 3. Take it for a spin as an agent!\n", "\n", - "Here we'll test out `Zephyr-7B-beta` as a zero-shot ReAct Agent. The example below is taken from [here](https://python.langchain.com/docs/modules/agents/agent_types/react#using-chat-models).\n", + "Here we'll test out `Zephyr-7B-beta` as a zero-shot `ReAct` Agent. The example below is taken from [here](https://python.langchain.com/docs/modules/agents/agent_types/react#using-chat-models).\n", "\n", "> Note: To run this section, you'll need to have a [SerpAPI Token](https://serpapi.com/) saved as an environment variable: `SERPAPI_API_KEY`" ] @@ -448,7 +448,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.5" + "version": "3.10.12" } }, "nbformat": 4, diff --git a/docs/docs/integrations/platforms/huggingface.mdx b/docs/docs/integrations/platforms/huggingface.mdx index 341d8f6cbc..927464a5e9 100644 --- a/docs/docs/integrations/platforms/huggingface.mdx +++ b/docs/docs/integrations/platforms/huggingface.mdx @@ -58,31 +58,24 @@ See a [usage example](/docs/integrations/llms/huggingface_textgen_inference). from langchain_community.llms import HuggingFaceTextGenInference ``` +## Chat models +### Models from Hugging Face -## Document Loaders - -### Hugging Face dataset +We can use the `Hugging Face` LLM classes or directly use the `ChatHuggingFace` class. ->[Hugging Face Hub](https://huggingface.co/docs/hub/index) is home to over 75,000 -> [datasets](https://huggingface.co/docs/hub/index#datasets) in more than 100 languages -> that can be used for a broad range of tasks across NLP, Computer Vision, and Audio. -> They used for a diverse range of tasks such as translation, automatic speech -> recognition, and image classification. - -We need to install `datasets` python package. +We need to install several python packages. ```bash -pip install datasets +pip install huggingface_hub +pip install transformers ``` - -See a [usage example](/docs/integrations/document_loaders/hugging_face_dataset). +See a [usage example](/docs/integrations/chat/huggingface). ```python -from langchain_community.document_loaders.hugging_face_dataset import HuggingFaceDatasetLoader +from langchain_community.chat_models.huggingface import ChatHuggingFace ``` - ## Embedding Models ### Hugging Face Hub @@ -126,6 +119,48 @@ See a [usage example](/docs/integrations/text_embedding/bge_huggingface). from langchain_community.embeddings import HuggingFaceBgeEmbeddings ``` +### Hugging Face Text Embeddings Inference (TEI) + +>[Hugging Face Text Embeddings Inference (TEI)](https://huggingface.co/docs/text-generation-inference/index) is a toolkit for deploying and serving open-source +> text embeddings and sequence classification models. `TEI` enables high-performance extraction for the most popular models, +>including `FlagEmbedding`, `Ember`, `GTE` and `E5`. + +We need to install `huggingface-hub` python package. + +```bash +pip install huggingface-hub +``` + +See a [usage example](/docs/integrations/text_embedding/text_embeddings_inference). + +```python +from langchain_community.embeddings import HuggingFaceHubEmbeddings +``` + + +## Document Loaders + +### Hugging Face dataset + +>[Hugging Face Hub](https://huggingface.co/docs/hub/index) is home to over 75,000 +> [datasets](https://huggingface.co/docs/hub/index#datasets) in more than 100 languages +> that can be used for a broad range of tasks across NLP, Computer Vision, and Audio. +> They used for a diverse range of tasks such as translation, automatic speech +> recognition, and image classification. + +We need to install `datasets` python package. + +```bash +pip install datasets +``` + +See a [usage example](/docs/integrations/document_loaders/hugging_face_dataset). + +```python +from langchain_community.document_loaders.hugging_face_dataset import HuggingFaceDatasetLoader +``` + + ## Tools diff --git a/docs/docs/integrations/text_embedding/text_embeddings_inference.ipynb b/docs/docs/integrations/text_embedding/text_embeddings_inference.ipynb index 328da81e75..6c5a9e1a22 100644 --- a/docs/docs/integrations/text_embedding/text_embeddings_inference.ipynb +++ b/docs/docs/integrations/text_embedding/text_embeddings_inference.ipynb @@ -7,7 +7,9 @@ "source": [ "# Text Embeddings Inference\n", "\n", - "Text Embeddings Inference (TEI) is a toolkit for deploying and serving open source text embeddings and sequence classification models. TEI enables high-performance extraction for the most popular models, including FlagEmbedding, Ember, GTE and E5.\n", + ">[Hugging Face Text Embeddings Inference (TEI)](https://huggingface.co/docs/text-generation-inference/index) is a toolkit for deploying and serving open-source\n", + "> text embeddings and sequence classification models. `TEI` enables high-performance extraction for the most popular models,\n", + ">including `FlagEmbedding`, `Ember`, `GTE` and `E5`.\n", "\n", "To use it within langchain, first install `huggingface-hub`." ] @@ -21,7 +23,7 @@ }, "outputs": [], "source": [ - "%pip install --upgrade --quiet huggingface-hub -q" + "%pip install --upgrade huggingface-hub" ] }, { @@ -146,9 +148,9 @@ ], "metadata": { "kernelspec": { - "display_name": "conda_python3", + "display_name": "Python 3 (ipykernel)", "language": "python", - "name": "conda_python3" + "name": "python3" }, "language_info": { "codemirror_mode": { @@ -160,7 +162,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.13" + "version": "3.10.12" } }, "nbformat": 4,