diff --git a/docs/extras/modules/callbacks/integrations/promptlayer.ipynb b/docs/extras/modules/callbacks/integrations/promptlayer.ipynb deleted file mode 100644 index 5527cb1980..0000000000 --- a/docs/extras/modules/callbacks/integrations/promptlayer.ipynb +++ /dev/null @@ -1,213 +0,0 @@ -{ - "cells": [ - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# PromptLayer\n", - "\n", - "\n" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "[PromptLayer](https://promptlayer.com) is a an observability platform for prompts and LLMs. In this guide we will go over how to setup the `PromptLayerCallbackHandler`. While PromptLayer does have LLMs that integrate directly with LangChain (eg [`PromptLayerOpenAI`](https://python.langchain.com/docs/modules/model_io/models/llms/integrations/promptlayer_openai)), this callback will be an easier and more feature rich way to integrate PromptLayer with any model on LangChain. \n", - "\n", - "This callback is also the recommended way to connect with PromptLayer when building Chains and Agents on LangChain." - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Installation and Setup" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!pip install promptlayer --upgrade" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Getting API Credentials\n", - "\n", - "If you have not already create an account on [PromptLayer](https://www.promptlayer.com) and get an API key by clicking on the settings cog in the navbar\n", - "Set it as an environment variabled called `PROMPTLAYER_API_KEY`\n" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Usage\n", - "\n", - "To get started with `PromptLayerCallbackHandler` is fairly simple, it takes two optional arguments:\n", - "1. `pl_tags` - an optional list of strings that will be tags tracked on PromptLayer\n", - "2. `pl_id_callback` - an optional function that will get a `promptlayer_request_id` as an argument. This id can be used with all of PromptLayers tracking features to track, metadata, scores, and prompt usage." - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Simple Example\n", - "\n", - "In this simple example we use `PromptLayerCallbackHandler` with `ChatOpenAI`. We add a PromptLayer tag named `chatopenai`" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "content=\"Sure, here's one:\\n\\nWhy did the tomato turn red?\\n\\nBecause it saw the salad dressing!\" additional_kwargs={} example=False\n" - ] - } - ], - "source": [ - "from langchain.chat_models import ChatOpenAI\n", - "from langchain.schema import (\n", - " AIMessage,\n", - " HumanMessage,\n", - " SystemMessage,\n", - ")\n", - "from langchain.callbacks import PromptLayerCallbackHandler\n", - "\n", - "chat_llm = ChatOpenAI(\n", - " temperature=0,\n", - " callbacks=[PromptLayerCallbackHandler(pl_tags=[\"chatopenai\"])],\n", - ")\n", - "llm_results = chat_llm(\n", - " [\n", - " HumanMessage(content=\"What comes after 1,2,3 ?\"),\n", - " HumanMessage(content=\"Tell me another joke?\"),\n", - " ]\n", - ")\n", - "print(llm_results)\n" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Full Featured Example\n", - "\n", - "In this example we unlock more of the power of PromptLayer.\n", - "\n", - "We are using the Prompt Registry and fetching the prompt called `example`.\n", - "\n", - "We also define a `pl_id_callback` function that tracks a score, metadata and the prompt used. Read more about tracking on [our docs](docs.promptlayer.com)." - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "prompt layer id 6050929\n" - ] - }, - { - "data": { - "text/plain": [ - "'\\nToasterCo.'" - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain.llms import OpenAI\n", - "from langchain.callbacks import PromptLayerCallbackHandler\n", - "import promptlayer\n", - "\n", - "def pl_id_callback(promptlayer_request_id):\n", - " print(\"prompt layer id \", promptlayer_request_id)\n", - " promptlayer.track.score(\n", - " request_id=promptlayer_request_id, score=100\n", - " ) # score is an integer 0-100\n", - " promptlayer.track.metadata(\n", - " request_id=promptlayer_request_id, metadata={\"foo\": \"bar\"}\n", - " ) # metadata is a dictionary of key value pairs that is tracked on PromptLayer\n", - " promptlayer.track.prompt(\n", - " request_id=promptlayer_request_id,\n", - " prompt_name=\"example\",\n", - " prompt_input_variables={\"product\": \"toasters\"},\n", - " version=1,\n", - " )\n", - "\n", - "\n", - "openai_llm = OpenAI(\n", - " model_name=\"text-davinci-002\",\n", - " callbacks=[PromptLayerCallbackHandler(pl_id_callback=pl_id_callback)],\n", - ")\n", - "\n", - "example_prompt = promptlayer.prompts.get(\"example\", version=1, langchain=True)\n", - "openai_llm(example_prompt.format(product=\"toasters\"))" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "That is all it takes! After setup all your requests will show up on the PromptLayer dasahboard.\n", - "This callback also works with any LLM implemented on LangChain." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "base", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.8" - }, - "vscode": { - "interpreter": { - "hash": "c4fe2cd85a8d9e8baaec5340ce66faff1c77581a9f43e6c45e85e09b6fced008" - } - } - }, - "nbformat": 4, - "nbformat_minor": 4 -}