mirror of https://github.com/hwchase17/langchain
You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
240 lines
13 KiB
Plaintext
240 lines
13 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "62bacc68-1976-44eb-9316-d5baf54bf595",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Javelin AI Gateway Tutorial\n",
|
|
"\n",
|
|
"This Jupyter Notebook will explore how to interact with the Javelin AI Gateway using the Python SDK. \n",
|
|
"The Javelin AI Gateway facilitates the utilization of large language models (LLMs) like OpenAI, Cohere, Anthropic, and others by \n",
|
|
"providing a secure and unified endpoint. The gateway itself provides a centralized mechanism to roll out models systematically, \n",
|
|
"provide access security, policy & cost guardrails for enterprises, etc., \n",
|
|
"\n",
|
|
"For a complete listing of all the features & benefits of Javelin, please visit www.getjavelin.io\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e52185f8-132b-4585-b73d-6fee928ac199",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 1: Introduction\n",
|
|
"[The Javelin AI Gateway](https://www.getjavelin.io) is an enterprise-grade API Gateway for AI applications. It integrates robust access security, ensuring secure interactions with large language models. Learn more in the [official documentation](https://docs.getjavelin.io).\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2e2acdb3-e3b8-422b-b077-7a0d63d18349",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 2: Installation\n",
|
|
"Before we begin, we must install the `javelin_sdk` and set up the Javelin API key as an environment variable. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "e91518a4-43ce-443e-b4c0-dbc652eb749f",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Requirement already satisfied: javelin_sdk in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (0.1.8)\n",
|
|
"Requirement already satisfied: httpx<0.25.0,>=0.24.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from javelin_sdk) (0.24.1)\n",
|
|
"Requirement already satisfied: pydantic<2.0.0,>=1.10.7 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from javelin_sdk) (1.10.12)\n",
|
|
"Requirement already satisfied: certifi in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (2023.5.7)\n",
|
|
"Requirement already satisfied: httpcore<0.18.0,>=0.15.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (0.17.3)\n",
|
|
"Requirement already satisfied: idna in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (3.4)\n",
|
|
"Requirement already satisfied: sniffio in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (1.3.0)\n",
|
|
"Requirement already satisfied: typing-extensions>=4.2.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from pydantic<2.0.0,>=1.10.7->javelin_sdk) (4.7.1)\n",
|
|
"Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpcore<0.18.0,>=0.15.0->httpx<0.25.0,>=0.24.0->javelin_sdk) (0.14.0)\n",
|
|
"Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpcore<0.18.0,>=0.15.0->httpx<0.25.0,>=0.24.0->javelin_sdk) (3.7.1)\n",
|
|
"Note: you may need to restart the kernel to use updated packages.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"pip install 'javelin_sdk'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "53b546dc-9ca3-4602-9a7b-d733d99e8e2f",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 3: Completions Example\n",
|
|
"This section will demonstrate how to interact with the Javelin AI Gateway to get completions from a large language model. Here is a Python script that demonstrates this:\n",
|
|
"(note) assumes that you have setup a route in the gateway called 'eng_dept03'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "d36949f0-5354-44ca-9a31-70c769344319",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"ename": "ImportError",
|
|
"evalue": "cannot import name 'JavelinAIGateway' from 'langchain.llms' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/llms/__init__.py)",
|
|
"output_type": "error",
|
|
"traceback": [
|
|
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
|
"\u001b[0;31mImportError\u001b[0m Traceback (most recent call last)",
|
|
"Cell \u001b[0;32mIn[6], line 2\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mchains\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m LLMChain\n\u001b[0;32m----> 2\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mllms\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m JavelinAIGateway\n\u001b[1;32m 3\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mprompts\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m PromptTemplate\n\u001b[1;32m 5\u001b[0m route_completions \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124meng_dept03\u001b[39m\u001b[38;5;124m\"\u001b[39m\n",
|
|
"\u001b[0;31mImportError\u001b[0m: cannot import name 'JavelinAIGateway' from 'langchain.llms' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/llms/__init__.py)"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain.chains import LLMChain\n",
|
|
"from langchain.prompts import PromptTemplate\n",
|
|
"from langchain_community.llms import JavelinAIGateway\n",
|
|
"\n",
|
|
"route_completions = \"eng_dept03\"\n",
|
|
"\n",
|
|
"gateway = JavelinAIGateway(\n",
|
|
" gateway_uri=\"http://localhost:8000\", # replace with service URL or host/port of Javelin\n",
|
|
" route=route_completions,\n",
|
|
" model_name=\"gpt-3.5-turbo-instruct\",\n",
|
|
")\n",
|
|
"\n",
|
|
"prompt = PromptTemplate(\"Translate the following English text to French: {text}\")\n",
|
|
"\n",
|
|
"llmchain = LLMChain(llm=gateway, prompt=prompt)\n",
|
|
"result = llmchain.run(\"podcast player\")\n",
|
|
"\n",
|
|
"print(result)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6b63fe93-2e77-4ea9-b8e7-dec2b96b8e95",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Step 4: Embeddings Example\n",
|
|
"This section demonstrates how to use the Javelin AI Gateway to obtain embeddings for text queries and documents. Here is a Python script that illustrates this:\n",
|
|
"(note) assumes that you have setup a route in the gateway called 'embeddings'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"id": "878e6c1d-be7f-49de-825c-43c266c8714e",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"ename": "ImportError",
|
|
"evalue": "cannot import name 'JavelinAIGatewayEmbeddings' from 'langchain.embeddings' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/embeddings/__init__.py)",
|
|
"output_type": "error",
|
|
"traceback": [
|
|
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
|
"\u001b[0;31mImportError\u001b[0m Traceback (most recent call last)",
|
|
"Cell \u001b[0;32mIn[9], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01membeddings\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m JavelinAIGatewayEmbeddings\n\u001b[1;32m 2\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01membeddings\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mopenai\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m OpenAIEmbeddings\n\u001b[1;32m 4\u001b[0m embeddings \u001b[38;5;241m=\u001b[39m JavelinAIGatewayEmbeddings(\n\u001b[1;32m 5\u001b[0m gateway_uri\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mhttp://localhost:8000\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;66;03m# replace with service URL or host/port of Javelin\u001b[39;00m\n\u001b[1;32m 6\u001b[0m route\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124membeddings\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m 7\u001b[0m )\n",
|
|
"\u001b[0;31mImportError\u001b[0m: cannot import name 'JavelinAIGatewayEmbeddings' from 'langchain.embeddings' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/embeddings/__init__.py)"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain_community.embeddings import JavelinAIGatewayEmbeddings\n",
|
|
"\n",
|
|
"embeddings = JavelinAIGatewayEmbeddings(\n",
|
|
" gateway_uri=\"http://localhost:8000\", # replace with service URL or host/port of Javelin\n",
|
|
" route=\"embeddings\",\n",
|
|
")\n",
|
|
"\n",
|
|
"print(embeddings.embed_query(\"hello\"))\n",
|
|
"print(embeddings.embed_documents([\"hello\"]))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "07c6691b-d333-4598-b2b7-c0933ed75937",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Step 5: Chat Example\n",
|
|
"This section illustrates how to interact with the Javelin AI Gateway to facilitate a chat with a large language model. Here is a Python script that demonstrates this:\n",
|
|
"(note) assumes that you have setup a route in the gateway called 'mychatbot_route'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"id": "653ef88c-36cd-4730-9c12-43c246b551f1",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"ename": "ImportError",
|
|
"evalue": "cannot import name 'ChatJavelinAIGateway' from 'langchain.chat_models' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/chat_models/__init__.py)",
|
|
"output_type": "error",
|
|
"traceback": [
|
|
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
|
"\u001b[0;31mImportError\u001b[0m Traceback (most recent call last)",
|
|
"Cell \u001b[0;32mIn[8], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mchat_models\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m ChatJavelinAIGateway\n\u001b[1;32m 2\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mschema\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m HumanMessage, SystemMessage\n\u001b[1;32m 4\u001b[0m messages \u001b[38;5;241m=\u001b[39m [\n\u001b[1;32m 5\u001b[0m SystemMessage(\n\u001b[1;32m 6\u001b[0m content\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mYou are a helpful assistant that translates English to French.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 10\u001b[0m ),\n\u001b[1;32m 11\u001b[0m ]\n",
|
|
"\u001b[0;31mImportError\u001b[0m: cannot import name 'ChatJavelinAIGateway' from 'langchain.chat_models' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/chat_models/__init__.py)"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain_community.chat_models import ChatJavelinAIGateway\n",
|
|
"from langchain_core.messages import HumanMessage, SystemMessage\n",
|
|
"\n",
|
|
"messages = [\n",
|
|
" SystemMessage(\n",
|
|
" content=\"You are a helpful assistant that translates English to French.\"\n",
|
|
" ),\n",
|
|
" HumanMessage(\n",
|
|
" content=\"Artificial Intelligence has the power to transform humanity and make the world a better place\"\n",
|
|
" ),\n",
|
|
"]\n",
|
|
"\n",
|
|
"chat = ChatJavelinAIGateway(\n",
|
|
" gateway_uri=\"http://localhost:8000\", # replace with service URL or host/port of Javelin\n",
|
|
" route=\"mychatbot_route\",\n",
|
|
" model_name=\"gpt-3.5-turbo\",\n",
|
|
" params={\"temperature\": 0.1},\n",
|
|
")\n",
|
|
"\n",
|
|
"print(chat(messages))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6eb9cf33-6505-4e05-808b-645856463a8e",
|
|
"metadata": {},
|
|
"source": [
|
|
"Step 6: Conclusion\n",
|
|
"This tutorial introduced the Javelin AI Gateway and demonstrated how to interact with it using the Python SDK. \n",
|
|
"Remember to check the Javelin [Python SDK](https://www.github.com/getjavelin.io/javelin-python) for more examples and to explore the official documentation for additional details.\n",
|
|
"\n",
|
|
"Happy coding!"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.4"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|