mirror of
https://github.com/hwchase17/langchain
synced 2024-10-31 15:20:26 +00:00
a7000ee89e
## Description This PR implements a callback handler for SageMaker Experiments which is similar to that of mlflow. * When creating the callback handler, it takes the experiment's run object as an argument. All the callback outputs are then logged to the run object. * The output of each callback action (e.g., `on_llm_start`) is saved to S3 bucket as json file. * Optionally, you can also log additional information such as the LLM hyper-parameters to the same run object. * Once the callback object is no more needed, you will need to call the `flush_tracker()` method. This makes sure that any intermediate files are deleted. * A separate notebook example is provided to show how the callback is used. @3coins @agola11 --------- Co-authored-by: Tesfagabir Meharizghi <mehariz@amazon.com>
917 lines
23 KiB
Plaintext
917 lines
23 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ef3909cf-72ca-4841-85c6-ef4e0eae3aaf",
|
|
"metadata": {},
|
|
"source": [
|
|
"# SageMaker Tracking\n",
|
|
"\n",
|
|
"This notebook shows how LangChain Callback can be used to log and track prompts and other LLM hyperparameters into SageMaker Experiments. Here, we use different scenarios to showcase the capability:\n",
|
|
"* **Scenario 1**: *Single LLM* - A case where a single LLM model is used to generate output based on a given prompt.\n",
|
|
"* **Scenario 2**: *Sequential Chain* - A case where a sequential chain of two LLM models is used.\n",
|
|
"* **Scenario 3**: *Agent with Tools (Chain of Thought)* - A case where multiple tools (search and math) are used in addition to an LLM.\n",
|
|
"\n",
|
|
"[Amazon SageMaker](https://aws.amazon.com/sagemaker/) is a fully managed service that is used to quickly and easily build, train and deploy machine learning (ML) models. \n",
|
|
"\n",
|
|
"[Amazon SageMaker Experiments](https://docs.aws.amazon.com/sagemaker/latest/dg/experiments.html) is a capability of Amazon SageMaker that lets you organize, track, compare and evaluate ML experiments and model versions.\n",
|
|
"\n",
|
|
"In this notebook, we will create a single experiment to log the prompts from each scenario."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "94c22cb4-3b1c-432b-b3be-0235eec79c5c",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Installation and Setup"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "2353436d-17fe-4f58-a2f9-c299d56393fd",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"!pip install sagemaker\n",
|
|
"!pip install openai\n",
|
|
"!pip install google-search-results"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "65dcf62e-7a38-4119-adb9-d9e884e82499",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"First, setup the required API keys\n",
|
|
"* OpenAI: https://platform.openai.com/account/api-keys (For OpenAI LLM model)\n",
|
|
"* Google SERP API: https://serpapi.com/manage-api-key (For Google Search Tool)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "5ec2b898-0cfc-4308-8e86-569cd7b7cf41",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"\n",
|
|
"## Add your API keys below\n",
|
|
"os.environ[\"OPENAI_API_KEY\"] = \"<ADD-KEY-HERE>\"\n",
|
|
"os.environ[\"SERPAPI_API_KEY\"] = \"<ADD-KEY-HERE>\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "80968ebf-519f-46de-8703-97532ac39e3e",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain.llms import OpenAI\n",
|
|
"from langchain.prompts import PromptTemplate\n",
|
|
"from langchain.chains import LLMChain, SimpleSequentialChain\n",
|
|
"from langchain.agents import initialize_agent, load_tools\n",
|
|
"from langchain.agents import Tool\n",
|
|
"from langchain.callbacks import SageMakerCallbackHandler\n",
|
|
"\n",
|
|
"from sagemaker.analytics import ExperimentAnalytics\n",
|
|
"from sagemaker.session import Session\n",
|
|
"from sagemaker.experiments.run import Run"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b67d031f-a01f-4009-ad29-c80ab8ad50ea",
|
|
"metadata": {},
|
|
"source": [
|
|
"## LLM Prompt Tracking"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "da2d70ee-173b-469d-a718-54c33d862844",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"#LLM Hyperparameters\n",
|
|
"HPARAMS = {\n",
|
|
" \"temperature\": 0.1,\n",
|
|
" \"model_name\": \"text-davinci-003\",\n",
|
|
"}\n",
|
|
"\n",
|
|
"#Bucket used to save prompt logs (Use `None` is used to save the default bucket or otherwise change it)\n",
|
|
"BUCKET_NAME = None\n",
|
|
"\n",
|
|
"#Experiment name\n",
|
|
"EXPERIMENT_NAME = \"langchain-sagemaker-tracker\"\n",
|
|
"\n",
|
|
"#Create SageMaker Session with the given bucket\n",
|
|
"session = Session(default_bucket=BUCKET_NAME)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7239a39a-08d8-43cb-8922-81abdd5d9ebf",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Scenario 1 - LLM"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "abc00335-50c8-4119-adb8-4c4ab8522e23",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"RUN_NAME = \"run-scenario-1\"\n",
|
|
"PROMPT_TEMPLATE = \"tell me a joke about {topic}\"\n",
|
|
"INPUT_VARIABLES = {\"topic\": \"fish\"}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "4a3a3cbe-db85-4255-8d8b-eaafdca8c6e2",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"with Run(experiment_name=EXPERIMENT_NAME, run_name=RUN_NAME, sagemaker_session=session) as run:\n",
|
|
"\n",
|
|
" # Create SageMaker Callback\n",
|
|
" sagemaker_callback = SageMakerCallbackHandler(run)\n",
|
|
"\n",
|
|
" # Define LLM model with callback\n",
|
|
" llm = OpenAI(callbacks=[sagemaker_callback], **HPARAMS)\n",
|
|
"\n",
|
|
" # Create prompt template\n",
|
|
" prompt = PromptTemplate.from_template(template=PROMPT_TEMPLATE)\n",
|
|
"\n",
|
|
" # Create LLM Chain\n",
|
|
" chain = LLMChain(llm=llm, prompt=prompt, callbacks=[sagemaker_callback])\n",
|
|
"\n",
|
|
" # Run chain\n",
|
|
" chain.run(**INPUT_VARIABLES)\n",
|
|
"\n",
|
|
" # Reset the callback\n",
|
|
" sagemaker_callback.flush_tracker()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7dc69934-9f42-40b7-9931-36a3371a38da",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Scenario 2 - Sequential Chain"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "50b75ef9-9825-4ccc-8414-4cd7525a1b68",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"RUN_NAME = \"run-scenario-2\"\n",
|
|
"\n",
|
|
"PROMPT_TEMPLATE_1 = \"\"\"You are a playwright. Given the title of play, it is your job to write a synopsis for that title.\n",
|
|
"Title: {title}\n",
|
|
"Playwright: This is a synopsis for the above play:\"\"\"\n",
|
|
"PROMPT_TEMPLATE_2 = \"\"\"You are a play critic from the New York Times. Given the synopsis of play, it is your job to write a review for that play.\n",
|
|
"Play Synopsis: {synopsis}\n",
|
|
"Review from a New York Times play critic of the above play:\"\"\"\n",
|
|
"\n",
|
|
"INPUT_VARIABLES = {\n",
|
|
" \"input\": \"documentary about good video games that push the boundary of game design\"\n",
|
|
"}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "fb7fff5f-e89f-40e2-96b4-3641a0b6e9b4",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"with Run(experiment_name=EXPERIMENT_NAME, run_name=RUN_NAME, sagemaker_session=session) as run:\n",
|
|
"\n",
|
|
" # Create SageMaker Callback\n",
|
|
" sagemaker_callback = SageMakerCallbackHandler(run)\n",
|
|
"\n",
|
|
" # Create prompt templates for the chain\n",
|
|
" prompt_template1 = PromptTemplate.from_template(template=PROMPT_TEMPLATE_1)\n",
|
|
" prompt_template2 = PromptTemplate.from_template(template=PROMPT_TEMPLATE_2)\n",
|
|
"\n",
|
|
" # Define LLM model with callback\n",
|
|
" llm = OpenAI(callbacks=[sagemaker_callback], **HPARAMS)\n",
|
|
"\n",
|
|
" # Create chain1\n",
|
|
" chain1 = LLMChain(llm=llm, prompt=prompt_template1, callbacks=[sagemaker_callback])\n",
|
|
"\n",
|
|
" # Create chain2\n",
|
|
" chain2 = LLMChain(llm=llm, prompt=prompt_template2, callbacks=[sagemaker_callback])\n",
|
|
"\n",
|
|
" # Create Sequential chain\n",
|
|
" overall_chain = SimpleSequentialChain(chains=[chain1, chain2], callbacks=[sagemaker_callback])\n",
|
|
"\n",
|
|
" # Run overall sequential chain\n",
|
|
" overall_chain.run(**INPUT_VARIABLES)\n",
|
|
"\n",
|
|
" # Reset the callback\n",
|
|
" sagemaker_callback.flush_tracker()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6b82bd0e-c626-4797-bb06-c1983f176315",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Scenario 3 - Agent with Tools"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "b5066f03-49dc-4868-be8e-d21ce22063fe",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"RUN_NAME = \"run-scenario-3\"\n",
|
|
"PROMPT_TEMPLATE = \"Who is the oldest person alive? And what is their current age raised to the power of 1.51?\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "98385c42-9e44-4b03-b76d-007cb4797864",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"with Run(experiment_name=EXPERIMENT_NAME, run_name=RUN_NAME, sagemaker_session=session) as run:\n",
|
|
"\n",
|
|
" # Create SageMaker Callback\n",
|
|
" sagemaker_callback = SageMakerCallbackHandler(run)\n",
|
|
"\n",
|
|
" # Define LLM model with callback\n",
|
|
" llm = OpenAI(callbacks=[sagemaker_callback], **HPARAMS)\n",
|
|
"\n",
|
|
" # Define tools\n",
|
|
" tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm, callbacks=[sagemaker_callback])\n",
|
|
"\n",
|
|
" # Initialize agent with all the tools\n",
|
|
" agent = initialize_agent(tools, llm, agent=\"zero-shot-react-description\", callbacks=[sagemaker_callback])\n",
|
|
"\n",
|
|
" # Run agent\n",
|
|
" agent.run(input=PROMPT_TEMPLATE)\n",
|
|
"\n",
|
|
" # Reset the callback\n",
|
|
" sagemaker_callback.flush_tracker()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c306a1c9-99f8-476d-96db-347746f5cfe0",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Load Log Data\n",
|
|
"\n",
|
|
"Once the prompts are logged, we can easily load and convert them to Pandas DataFrame as follows."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "ec7b4af2-e01d-4f6c-9de5-70d2b4acb9e6",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"#Load\n",
|
|
"logs = ExperimentAnalytics(experiment_name=EXPERIMENT_NAME)\n",
|
|
"\n",
|
|
"#Convert as pandas dataframe\n",
|
|
"df = logs.dataframe(force_refresh=True)\n",
|
|
"\n",
|
|
"print(df.shape)\n",
|
|
"df.head()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "29991c75-f9cf-4c36-abfd-903c09fb170d",
|
|
"metadata": {},
|
|
"source": [
|
|
"As can be seen above, there are three runs (rows) in the experiment corresponding to each scenario. Each run logs the prompts and related LLM settings/hyperparameters as json and are saved in s3 bucket. Feel free to load and explore the log data from each json path."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "61a695d6-0aef-4284-9e12-eea8bc143dbd",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"availableInstances": [
|
|
{
|
|
"_defaultOrder": 0,
|
|
"_isFastLaunch": true,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 4,
|
|
"name": "ml.t3.medium",
|
|
"vcpuNum": 2
|
|
},
|
|
{
|
|
"_defaultOrder": 1,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 8,
|
|
"name": "ml.t3.large",
|
|
"vcpuNum": 2
|
|
},
|
|
{
|
|
"_defaultOrder": 2,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 16,
|
|
"name": "ml.t3.xlarge",
|
|
"vcpuNum": 4
|
|
},
|
|
{
|
|
"_defaultOrder": 3,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 32,
|
|
"name": "ml.t3.2xlarge",
|
|
"vcpuNum": 8
|
|
},
|
|
{
|
|
"_defaultOrder": 4,
|
|
"_isFastLaunch": true,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 8,
|
|
"name": "ml.m5.large",
|
|
"vcpuNum": 2
|
|
},
|
|
{
|
|
"_defaultOrder": 5,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 16,
|
|
"name": "ml.m5.xlarge",
|
|
"vcpuNum": 4
|
|
},
|
|
{
|
|
"_defaultOrder": 6,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 32,
|
|
"name": "ml.m5.2xlarge",
|
|
"vcpuNum": 8
|
|
},
|
|
{
|
|
"_defaultOrder": 7,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 64,
|
|
"name": "ml.m5.4xlarge",
|
|
"vcpuNum": 16
|
|
},
|
|
{
|
|
"_defaultOrder": 8,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 128,
|
|
"name": "ml.m5.8xlarge",
|
|
"vcpuNum": 32
|
|
},
|
|
{
|
|
"_defaultOrder": 9,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 192,
|
|
"name": "ml.m5.12xlarge",
|
|
"vcpuNum": 48
|
|
},
|
|
{
|
|
"_defaultOrder": 10,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 256,
|
|
"name": "ml.m5.16xlarge",
|
|
"vcpuNum": 64
|
|
},
|
|
{
|
|
"_defaultOrder": 11,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 384,
|
|
"name": "ml.m5.24xlarge",
|
|
"vcpuNum": 96
|
|
},
|
|
{
|
|
"_defaultOrder": 12,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 8,
|
|
"name": "ml.m5d.large",
|
|
"vcpuNum": 2
|
|
},
|
|
{
|
|
"_defaultOrder": 13,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 16,
|
|
"name": "ml.m5d.xlarge",
|
|
"vcpuNum": 4
|
|
},
|
|
{
|
|
"_defaultOrder": 14,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 32,
|
|
"name": "ml.m5d.2xlarge",
|
|
"vcpuNum": 8
|
|
},
|
|
{
|
|
"_defaultOrder": 15,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 64,
|
|
"name": "ml.m5d.4xlarge",
|
|
"vcpuNum": 16
|
|
},
|
|
{
|
|
"_defaultOrder": 16,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 128,
|
|
"name": "ml.m5d.8xlarge",
|
|
"vcpuNum": 32
|
|
},
|
|
{
|
|
"_defaultOrder": 17,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 192,
|
|
"name": "ml.m5d.12xlarge",
|
|
"vcpuNum": 48
|
|
},
|
|
{
|
|
"_defaultOrder": 18,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 256,
|
|
"name": "ml.m5d.16xlarge",
|
|
"vcpuNum": 64
|
|
},
|
|
{
|
|
"_defaultOrder": 19,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 384,
|
|
"name": "ml.m5d.24xlarge",
|
|
"vcpuNum": 96
|
|
},
|
|
{
|
|
"_defaultOrder": 20,
|
|
"_isFastLaunch": false,
|
|
"category": "General purpose",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": true,
|
|
"memoryGiB": 0,
|
|
"name": "ml.geospatial.interactive",
|
|
"supportedImageNames": [
|
|
"sagemaker-geospatial-v1-0"
|
|
],
|
|
"vcpuNum": 0
|
|
},
|
|
{
|
|
"_defaultOrder": 21,
|
|
"_isFastLaunch": true,
|
|
"category": "Compute optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 4,
|
|
"name": "ml.c5.large",
|
|
"vcpuNum": 2
|
|
},
|
|
{
|
|
"_defaultOrder": 22,
|
|
"_isFastLaunch": false,
|
|
"category": "Compute optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 8,
|
|
"name": "ml.c5.xlarge",
|
|
"vcpuNum": 4
|
|
},
|
|
{
|
|
"_defaultOrder": 23,
|
|
"_isFastLaunch": false,
|
|
"category": "Compute optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 16,
|
|
"name": "ml.c5.2xlarge",
|
|
"vcpuNum": 8
|
|
},
|
|
{
|
|
"_defaultOrder": 24,
|
|
"_isFastLaunch": false,
|
|
"category": "Compute optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 32,
|
|
"name": "ml.c5.4xlarge",
|
|
"vcpuNum": 16
|
|
},
|
|
{
|
|
"_defaultOrder": 25,
|
|
"_isFastLaunch": false,
|
|
"category": "Compute optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 72,
|
|
"name": "ml.c5.9xlarge",
|
|
"vcpuNum": 36
|
|
},
|
|
{
|
|
"_defaultOrder": 26,
|
|
"_isFastLaunch": false,
|
|
"category": "Compute optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 96,
|
|
"name": "ml.c5.12xlarge",
|
|
"vcpuNum": 48
|
|
},
|
|
{
|
|
"_defaultOrder": 27,
|
|
"_isFastLaunch": false,
|
|
"category": "Compute optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 144,
|
|
"name": "ml.c5.18xlarge",
|
|
"vcpuNum": 72
|
|
},
|
|
{
|
|
"_defaultOrder": 28,
|
|
"_isFastLaunch": false,
|
|
"category": "Compute optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 192,
|
|
"name": "ml.c5.24xlarge",
|
|
"vcpuNum": 96
|
|
},
|
|
{
|
|
"_defaultOrder": 29,
|
|
"_isFastLaunch": true,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 1,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 16,
|
|
"name": "ml.g4dn.xlarge",
|
|
"vcpuNum": 4
|
|
},
|
|
{
|
|
"_defaultOrder": 30,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 1,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 32,
|
|
"name": "ml.g4dn.2xlarge",
|
|
"vcpuNum": 8
|
|
},
|
|
{
|
|
"_defaultOrder": 31,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 1,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 64,
|
|
"name": "ml.g4dn.4xlarge",
|
|
"vcpuNum": 16
|
|
},
|
|
{
|
|
"_defaultOrder": 32,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 1,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 128,
|
|
"name": "ml.g4dn.8xlarge",
|
|
"vcpuNum": 32
|
|
},
|
|
{
|
|
"_defaultOrder": 33,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 4,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 192,
|
|
"name": "ml.g4dn.12xlarge",
|
|
"vcpuNum": 48
|
|
},
|
|
{
|
|
"_defaultOrder": 34,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 1,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 256,
|
|
"name": "ml.g4dn.16xlarge",
|
|
"vcpuNum": 64
|
|
},
|
|
{
|
|
"_defaultOrder": 35,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 1,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 61,
|
|
"name": "ml.p3.2xlarge",
|
|
"vcpuNum": 8
|
|
},
|
|
{
|
|
"_defaultOrder": 36,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 4,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 244,
|
|
"name": "ml.p3.8xlarge",
|
|
"vcpuNum": 32
|
|
},
|
|
{
|
|
"_defaultOrder": 37,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 8,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 488,
|
|
"name": "ml.p3.16xlarge",
|
|
"vcpuNum": 64
|
|
},
|
|
{
|
|
"_defaultOrder": 38,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 8,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 768,
|
|
"name": "ml.p3dn.24xlarge",
|
|
"vcpuNum": 96
|
|
},
|
|
{
|
|
"_defaultOrder": 39,
|
|
"_isFastLaunch": false,
|
|
"category": "Memory Optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 16,
|
|
"name": "ml.r5.large",
|
|
"vcpuNum": 2
|
|
},
|
|
{
|
|
"_defaultOrder": 40,
|
|
"_isFastLaunch": false,
|
|
"category": "Memory Optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 32,
|
|
"name": "ml.r5.xlarge",
|
|
"vcpuNum": 4
|
|
},
|
|
{
|
|
"_defaultOrder": 41,
|
|
"_isFastLaunch": false,
|
|
"category": "Memory Optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 64,
|
|
"name": "ml.r5.2xlarge",
|
|
"vcpuNum": 8
|
|
},
|
|
{
|
|
"_defaultOrder": 42,
|
|
"_isFastLaunch": false,
|
|
"category": "Memory Optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 128,
|
|
"name": "ml.r5.4xlarge",
|
|
"vcpuNum": 16
|
|
},
|
|
{
|
|
"_defaultOrder": 43,
|
|
"_isFastLaunch": false,
|
|
"category": "Memory Optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 256,
|
|
"name": "ml.r5.8xlarge",
|
|
"vcpuNum": 32
|
|
},
|
|
{
|
|
"_defaultOrder": 44,
|
|
"_isFastLaunch": false,
|
|
"category": "Memory Optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 384,
|
|
"name": "ml.r5.12xlarge",
|
|
"vcpuNum": 48
|
|
},
|
|
{
|
|
"_defaultOrder": 45,
|
|
"_isFastLaunch": false,
|
|
"category": "Memory Optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 512,
|
|
"name": "ml.r5.16xlarge",
|
|
"vcpuNum": 64
|
|
},
|
|
{
|
|
"_defaultOrder": 46,
|
|
"_isFastLaunch": false,
|
|
"category": "Memory Optimized",
|
|
"gpuNum": 0,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 768,
|
|
"name": "ml.r5.24xlarge",
|
|
"vcpuNum": 96
|
|
},
|
|
{
|
|
"_defaultOrder": 47,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 1,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 16,
|
|
"name": "ml.g5.xlarge",
|
|
"vcpuNum": 4
|
|
},
|
|
{
|
|
"_defaultOrder": 48,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 1,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 32,
|
|
"name": "ml.g5.2xlarge",
|
|
"vcpuNum": 8
|
|
},
|
|
{
|
|
"_defaultOrder": 49,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 1,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 64,
|
|
"name": "ml.g5.4xlarge",
|
|
"vcpuNum": 16
|
|
},
|
|
{
|
|
"_defaultOrder": 50,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 1,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 128,
|
|
"name": "ml.g5.8xlarge",
|
|
"vcpuNum": 32
|
|
},
|
|
{
|
|
"_defaultOrder": 51,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 1,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 256,
|
|
"name": "ml.g5.16xlarge",
|
|
"vcpuNum": 64
|
|
},
|
|
{
|
|
"_defaultOrder": 52,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 4,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 192,
|
|
"name": "ml.g5.12xlarge",
|
|
"vcpuNum": 48
|
|
},
|
|
{
|
|
"_defaultOrder": 53,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 4,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 384,
|
|
"name": "ml.g5.24xlarge",
|
|
"vcpuNum": 96
|
|
},
|
|
{
|
|
"_defaultOrder": 54,
|
|
"_isFastLaunch": false,
|
|
"category": "Accelerated computing",
|
|
"gpuNum": 8,
|
|
"hideHardwareSpecs": false,
|
|
"memoryGiB": 768,
|
|
"name": "ml.g5.48xlarge",
|
|
"vcpuNum": 192
|
|
}
|
|
],
|
|
"instance_type": "ml.t3.large",
|
|
"kernelspec": {
|
|
"display_name": "conda_pytorch_p310",
|
|
"language": "python",
|
|
"name": "conda_pytorch_p310"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.10"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|