From 6d0af4e805a7617bfc433b74ed136dbf44046eb2 Mon Sep 17 00:00:00 2001 From: Leonid Ganeline Date: Fri, 1 Mar 2024 10:00:42 -0800 Subject: [PATCH] docs: nvidia: provider page update (#18054) Nvidia provider page is missing a Triton Inference Server package reference. Changes: - added the Triton Inference Server reference - copied the example notebook from the package into the doc files. - added the Triton Inference Server description and links, the link to the above example notebook - formatted page to the consistent format NOTE: It seems that the [example notebook](https://github.com/langchain-ai/langchain/blob/master/libs/partners/nvidia-trt/docs/llms.ipynb) was originally created in wrong place. It should be in the LangChain docs [here](https://github.com/langchain-ai/langchain/tree/master/docs/docs/integrations/llms). So, I've created a copy of this example. The original example is still in the nvidia-trt package. --- docs/docs/integrations/providers/nvidia.mdx | 50 ++++++++++++++------- 1 file changed, 34 insertions(+), 16 deletions(-) diff --git a/docs/docs/integrations/providers/nvidia.mdx b/docs/docs/integrations/providers/nvidia.mdx index c00eea6416..0be21e38f7 100644 --- a/docs/docs/integrations/providers/nvidia.mdx +++ b/docs/docs/integrations/providers/nvidia.mdx @@ -1,18 +1,28 @@ # NVIDIA -> [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) give users easy access to NVIDIA hosted API endpoints for NVIDIA AI Foundation Models like Mixtral 8x7B, Llama 2, Stable Diffusion, etc. These models, hosted on the [NVIDIA NGC catalog](https://catalog.ngc.nvidia.com/ai-foundation-models), are optimized, tested, and hosted on the NVIDIA AI platform, making them fast and easy to evaluate, further customize, and seamlessly run at peak performance on any accelerated stack. -> -> With [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/), you can get quick results from a fully accelerated stack running on [NVIDIA DGX Cloud](https://www.nvidia.com/en-us/data-center/dgx-cloud/). Once customized, these models can be deployed anywhere with enterprise-grade security, stability, and support using [NVIDIA AI Enterprise](https://www.nvidia.com/en-us/data-center/products/ai-enterprise/). +>NVIDIA provides an integration package for LangChain: `langchain-nvidia-ai-endpoints`. + +## NVIDIA AI Foundation Endpoints + +> [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) give users easy access to NVIDIA hosted API endpoints for +> NVIDIA AI Foundation Models like `Mixtral 8x7B`, `Llama 2`, `Stable Diffusion`, etc. These models, +> hosted on the [NVIDIA NGC catalog](https://catalog.ngc.nvidia.com/ai-foundation-models), are optimized, tested, and hosted on +> the NVIDIA AI platform, making them fast and easy to evaluate, further customize, +> and seamlessly run at peak performance on any accelerated stack. > -> These models can be easily accessed via the [`langchain-nvidia-ai-endpoints`](https://pypi.org/project/langchain-nvidia-ai-endpoints/) package, as shown below. +> With [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/), you can get quick results from a fully +> accelerated stack running on [NVIDIA DGX Cloud](https://www.nvidia.com/en-us/data-center/dgx-cloud/). Once customized, these +> models can be deployed anywhere with enterprise-grade security, stability, +> and support using [NVIDIA AI Enterprise](https://www.nvidia.com/en-us/data-center/products/ai-enterprise/). -## Installation +A selection of NVIDIA AI Foundation models is supported directly in LangChain with familiar APIs. -```bash -pip install -U langchain-nvidia-ai-endpoints -``` +The supported models can be found [in NGC](https://catalog.ngc.nvidia.com/ai-foundation-models). + +These models can be accessed via the [`langchain-nvidia-ai-endpoints`](https://pypi.org/project/langchain-nvidia-ai-endpoints/) +package, as shown below. -## Setup and Authentication +### Setting up - Create a free [NVIDIA NGC](https://catalog.ngc.nvidia.com/) account. - Navigate to `Catalog > AI Foundation Models > (Model with API endpoint)`. @@ -22,6 +32,16 @@ pip install -U langchain-nvidia-ai-endpoints export NVIDIA_API_KEY=nvapi-XXXXXXXXXXXXXXXXXXXXXXXXXX ``` +- Install a package: + +```bash +pip install -U langchain-nvidia-ai-endpoints +``` + +### Chat models + +See a [usage example](/docs/integrations/chat/nvidia_ai_endpoints). + ```python from langchain_nvidia_ai_endpoints import ChatNVIDIA @@ -30,12 +50,10 @@ result = llm.invoke("Write a ballad about LangChain.") print(result.content) ``` -## Using NVIDIA AI Foundation Endpoints - -A selection of NVIDIA AI Foundation models are supported directly in LangChain with familiar APIs. +### Embedding models -The active models which are supported can be found [in NGC](https://catalog.ngc.nvidia.com/ai-foundation-models). +See a [usage example](/docs/integrations/text_embedding/nvidia_ai_endpoints). -**The following may be useful examples to help you get started:** -- **[`ChatNVIDIA` Model](/docs/integrations/chat/nvidia_ai_endpoints).** -- **[`NVIDIAEmbeddings` Model for RAG Workflows](/docs/integrations/text_embedding/nvidia_ai_endpoints).** +```python +from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings +```