mirror of
https://github.com/hwchase17/langchain
synced 2024-11-10 01:10:59 +00:00
add links in example nb with tei/tgi references (#25821)
I have validated langchain interface with tei/tgi works as expected when TEI and TGI running on Intel Gaudi2. Adding some references to notebooks to help users find relevant info. --------- Co-authored-by: Rita Brugarolas <rbrugaro@idc708053.jf.intel.com> Co-authored-by: Chester Curme <chester.curme@gmail.com>
This commit is contained in:
parent
8fb594fd2a
commit
9fa172bc26
@ -210,6 +210,13 @@
|
|||||||
")\n",
|
")\n",
|
||||||
"llm(\"What did foo say about bar?\", callbacks=[StreamingStdOutCallbackHandler()])"
|
"llm(\"What did foo say about bar?\", callbacks=[StreamingStdOutCallbackHandler()])"
|
||||||
]
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"This same `HuggingFaceEndpoint` class can be used with a local [HuggingFace TGI instance](https://github.com/huggingface/text-generation-inference/blob/main/docs/source/index.md) serving the LLM. Check out the TGI [repository](https://github.com/huggingface/text-generation-inference/tree/main) for details on various hardware (GPU, TPU, Gaudi...) support."
|
||||||
|
]
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
@ -39,7 +39,9 @@
|
|||||||
"volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run\n",
|
"volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run\n",
|
||||||
"\n",
|
"\n",
|
||||||
"docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:0.6 --model-id $model --revision $revision\n",
|
"docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:0.6 --model-id $model --revision $revision\n",
|
||||||
"```"
|
"```\n",
|
||||||
|
"\n",
|
||||||
|
"Specifics on Docker usage might vary with the underlying hardware. For example, to serve the model on Intel Gaudi/Gaudi2 hardware, refer to the [tei-gaudi repository](https://github.com/huggingface/tei-gaudi) for the relevant docker run command."
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
Loading…
Reference in New Issue
Block a user