langchain/docs/docs/integrations/callbacks/promptlayer.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PromptLayer\n",
    "\n",
    ">[PromptLayer](https://docs.promptlayer.com/introduction) is a platform for prompt engineering. It also helps with the LLM observability to visualize requests, version prompts, and track usage.\n",
    ">\n",
    ">While `PromptLayer` does have LLMs that integrate directly with LangChain (e.g. [`PromptLayerOpenAI`](/docs/integrations/llms/promptlayer_openai)), using a callback is the recommended way to integrate `PromptLayer` with LangChain.\n",
    "\n",
    "In this guide, we will go over how to setup the `PromptLayerCallbackHandler`. \n",
    "\n",
    "See [PromptLayer docs](https://docs.promptlayer.com/languages/langchain) for more information."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Installation and Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet  promptlayer --upgrade"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting API Credentials\n",
    "\n",
    "If you do not have a PromptLayer account, create one on [promptlayer.com](https://www.promptlayer.com). Then get an API key by clicking on the settings cog in the navbar and\n",
    "set it as an environment variabled called `PROMPTLAYER_API_KEY`\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Usage\n",
    "\n",
    "Getting started with `PromptLayerCallbackHandler` is fairly simple, it takes two optional arguments:\n",
    "1. `pl_tags` - an optional list of strings that will be tracked as tags on PromptLayer.\n",
    "2. `pl_id_callback` - an optional function that will take `promptlayer_request_id` as an argument. This ID can be used with all of PromptLayer's tracking features to track, metadata, scores, and prompt usage."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple OpenAI Example\n",
    "\n",
    "In this simple example we use `PromptLayerCallbackHandler` with `ChatOpenAI`. We add a PromptLayer tag named `chatopenai`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-06T19:10:56.673622Z",
     "iopub.status.busy": "2024-03-06T19:10:56.673421Z",
     "iopub.status.idle": "2024-03-06T19:10:56.887519Z",
     "shell.execute_reply": "2024-03-06T19:10:56.886895Z",
     "shell.execute_reply.started": "2024-03-06T19:10:56.673608Z"
    }
   },
   "outputs": [],
   "source": [
    "import promptlayer  # Don't forget this 🍰\n",
    "from langchain_community.callbacks.promptlayer_callback import (\n",
    "    PromptLayerCallbackHandler,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.schema import (\n",
    "    HumanMessage,\n",
    ")\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "chat_llm = ChatOpenAI(\n",
    "    temperature=0,\n",
    "    callbacks=[PromptLayerCallbackHandler(pl_tags=[\"chatopenai\"])],\n",
    ")\n",
    "llm_results = chat_llm.invoke(\n",
    "    [\n",
    "        HumanMessage(content=\"What comes after 1,2,3 ?\"),\n",
    "        HumanMessage(content=\"Tell me another joke?\"),\n",
    "    ]\n",
    ")\n",
    "print(llm_results)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## GPT4All Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.llms import GPT4All\n",
    "\n",
    "model = GPT4All(model=\"./models/gpt4all-model.bin\", n_ctx=512, n_threads=8)\n",
    "callbacks = [PromptLayerCallbackHandler(pl_tags=[\"langchain\", \"gpt4all\"])]\n",
    "\n",
    "response = model.invoke(\n",
    "    \"Once upon a time, \",\n",
    "    config={\"callbacks\": callbacks},\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Full Featured Example\n",
    "\n",
    "In this example, we unlock more of the power of `PromptLayer`.\n",
    "\n",
    "PromptLayer allows you to visually create, version, and track prompt templates. Using the [Prompt Registry](https://docs.promptlayer.com/features/prompt-registry), we can programmatically fetch the prompt template called `example`.\n",
    "\n",
    "We also define a `pl_id_callback` function which takes in the `promptlayer_request_id` and logs a score, metadata and links the prompt template used. Read more about tracking on [our docs](https://docs.promptlayer.com/features/prompt-history/request-id)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import OpenAI\n",
    "\n",
    "\n",
    "def pl_id_callback(promptlayer_request_id):\n",
    "    print(\"prompt layer id \", promptlayer_request_id)\n",
    "    promptlayer.track.score(\n",
    "        request_id=promptlayer_request_id, score=100\n",
    "    )  # score is an integer 0-100\n",
    "    promptlayer.track.metadata(\n",
    "        request_id=promptlayer_request_id, metadata={\"foo\": \"bar\"}\n",
    "    )  # metadata is a dictionary of key value pairs that is tracked on PromptLayer\n",
    "    promptlayer.track.prompt(\n",
    "        request_id=promptlayer_request_id,\n",
    "        prompt_name=\"example\",\n",
    "        prompt_input_variables={\"product\": \"toasters\"},\n",
    "        version=1,\n",
    "    )  # link the request to a prompt template\n",
    "\n",
    "\n",
    "openai_llm = OpenAI(\n",
    "    model_name=\"gpt-3.5-turbo-instruct\",\n",
    "    callbacks=[PromptLayerCallbackHandler(pl_id_callback=pl_id_callback)],\n",
    ")\n",
    "\n",
    "example_prompt = promptlayer.prompts.get(\"example\", version=1, langchain=True)\n",
    "openai_llm.invoke(example_prompt.format(product=\"toasters\"))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is all it takes! After setup all your requests will show up on the PromptLayer dashboard.\n",
    "This callback also works with any LLM implemented on LangChain."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "c4fe2cd85a8d9e8baaec5340ce66faff1c77581a9f43e6c45e85e09b6fced008"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}