From 9fa172bc26260fd9d9d641471480ae78f6da8eda Mon Sep 17 00:00:00 2001 From: rbrugaro Date: Wed, 28 Aug 2024 14:33:25 -0700 Subject: [PATCH] add links in example nb with tei/tgi references (#25821) I have validated langchain interface with tei/tgi works as expected when TEI and TGI running on Intel Gaudi2. Adding some references to notebooks to help users find relevant info. --------- Co-authored-by: Rita Brugarolas Co-authored-by: Chester Curme --- docs/docs/integrations/llms/huggingface_endpoint.ipynb | 7 +++++++ .../text_embedding/text_embeddings_inference.ipynb | 4 +++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/docs/docs/integrations/llms/huggingface_endpoint.ipynb b/docs/docs/integrations/llms/huggingface_endpoint.ipynb index dc177d908b..933a4d1d52 100644 --- a/docs/docs/integrations/llms/huggingface_endpoint.ipynb +++ b/docs/docs/integrations/llms/huggingface_endpoint.ipynb @@ -210,6 +210,13 @@ ")\n", "llm(\"What did foo say about bar?\", callbacks=[StreamingStdOutCallbackHandler()])" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This same `HuggingFaceEndpoint` class can be used with a local [HuggingFace TGI instance](https://github.com/huggingface/text-generation-inference/blob/main/docs/source/index.md) serving the LLM. Check out the TGI [repository](https://github.com/huggingface/text-generation-inference/tree/main) for details on various hardware (GPU, TPU, Gaudi...) support." + ] } ], "metadata": { diff --git a/docs/docs/integrations/text_embedding/text_embeddings_inference.ipynb b/docs/docs/integrations/text_embedding/text_embeddings_inference.ipynb index cc054282f2..730a4f025f 100644 --- a/docs/docs/integrations/text_embedding/text_embeddings_inference.ipynb +++ b/docs/docs/integrations/text_embedding/text_embeddings_inference.ipynb @@ -39,7 +39,9 @@ "volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run\n", "\n", "docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:0.6 --model-id $model --revision $revision\n", - "```" + "```\n", + "\n", + "Specifics on Docker usage might vary with the underlying hardware. For example, to serve the model on Intel Gaudi/Gaudi2 hardware, refer to the [tei-gaudi repository](https://github.com/huggingface/tei-gaudi) for the relevant docker run command." ] }, {