mirror of
https://github.com/hwchase17/langchain
synced 2024-10-31 15:20:26 +00:00
c460c29a64
Here I am adding documentation for the `PromptLayerCallbackHandler`. When we created the initial PR for the callback handler the docs were causing issues, so we merged without the docs.
211 lines
6.7 KiB
Plaintext
211 lines
6.7 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# PromptLayer\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"[PromptLayer](https://promptlayer.com) is a an LLM observability platform that lets you visualize requests, version prompts, and track usage. In this guide we will go over how to setup the `PromptLayerCallbackHandler`. \n",
|
|
"\n",
|
|
"While PromptLayer does have LLMs that integrate directly with LangChain (eg [`PromptLayerOpenAI`](https://python.langchain.com/docs/modules/model_io/models/llms/integrations/promptlayer_openai)), this callback is the recommended way to integrate PromptLayer with LangChain.\n",
|
|
"\n",
|
|
"See [our docs](https://docs.promptlayer.com/languages/langchain) for more information."
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Installation and Setup"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!pip install promptlayer --upgrade"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Getting API Credentials\n",
|
|
"\n",
|
|
"If you do not have a PromptLayer account, create one on [promptlayer.com](https://www.promptlayer.com). Then get an API key by clicking on the settings cog in the navbar and\n",
|
|
"set it as an environment variabled called `PROMPTLAYER_API_KEY`\n"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Usage\n",
|
|
"\n",
|
|
"Getting started with `PromptLayerCallbackHandler` is fairly simple, it takes two optional arguments:\n",
|
|
"1. `pl_tags` - an optional list of strings that will be tracked as tags on PromptLayer.\n",
|
|
"2. `pl_id_callback` - an optional function that will take `promptlayer_request_id` as an argument. This ID can be used with all of PromptLayer's tracking features to track, metadata, scores, and prompt usage."
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Simple OpenAI Example\n",
|
|
"\n",
|
|
"In this simple example we use `PromptLayerCallbackHandler` with `ChatOpenAI`. We add a PromptLayer tag named `chatopenai`"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import promptlayer # Don't forget this 🍰\n",
|
|
"from langchain.callbacks import PromptLayerCallbackHandler\n",
|
|
"from langchain.chat_models import ChatOpenAI\n",
|
|
"from langchain.schema import (\n",
|
|
" HumanMessage,\n",
|
|
")\n",
|
|
"\n",
|
|
"chat_llm = ChatOpenAI(\n",
|
|
" temperature=0,\n",
|
|
" callbacks=[PromptLayerCallbackHandler(pl_tags=[\"chatopenai\"])],\n",
|
|
")\n",
|
|
"llm_results = chat_llm(\n",
|
|
" [\n",
|
|
" HumanMessage(content=\"What comes after 1,2,3 ?\"),\n",
|
|
" HumanMessage(content=\"Tell me another joke?\"),\n",
|
|
" ]\n",
|
|
")\n",
|
|
"print(llm_results)"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### GPT4All Example"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import promptlayer # Don't forget this 🍰\n",
|
|
"from langchain.callbacks import PromptLayerCallbackHandler\n",
|
|
"\n",
|
|
"from langchain.llms import GPT4All\n",
|
|
"\n",
|
|
"model = GPT4All(model=\"./models/gpt4all-model.bin\", n_ctx=512, n_threads=8)\n",
|
|
"\n",
|
|
"response = model(\n",
|
|
" \"Once upon a time, \",\n",
|
|
" callbacks=[PromptLayerCallbackHandler(pl_tags=[\"langchain\", \"gpt4all\"])],\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Full Featured Example\n",
|
|
"\n",
|
|
"In this example we unlock more of the power of PromptLayer.\n",
|
|
"\n",
|
|
"PromptLayer allows you to visually create, version, and track prompt templates. Using the [Prompt Registry](https://docs.promptlayer.com/features/prompt-registry), we can programatically fetch the prompt template called `example`.\n",
|
|
"\n",
|
|
"We also define a `pl_id_callback` function which takes in the `promptlayer_request_id` and logs a score, metadata and links the prompt template used. Read more about tracking on [our docs](https://docs.promptlayer.com/features/prompt-history/request-id)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import promptlayer # Don't forget this 🍰\n",
|
|
"from langchain.callbacks import PromptLayerCallbackHandler\n",
|
|
"from langchain.llms import OpenAI\n",
|
|
"\n",
|
|
"\n",
|
|
"def pl_id_callback(promptlayer_request_id):\n",
|
|
" print(\"prompt layer id \", promptlayer_request_id)\n",
|
|
" promptlayer.track.score(\n",
|
|
" request_id=promptlayer_request_id, score=100\n",
|
|
" ) # score is an integer 0-100\n",
|
|
" promptlayer.track.metadata(\n",
|
|
" request_id=promptlayer_request_id, metadata={\"foo\": \"bar\"}\n",
|
|
" ) # metadata is a dictionary of key value pairs that is tracked on PromptLayer\n",
|
|
" promptlayer.track.prompt(\n",
|
|
" request_id=promptlayer_request_id,\n",
|
|
" prompt_name=\"example\",\n",
|
|
" prompt_input_variables={\"product\": \"toasters\"},\n",
|
|
" version=1,\n",
|
|
" ) # link the request to a prompt template\n",
|
|
"\n",
|
|
"\n",
|
|
"openai_llm = OpenAI(\n",
|
|
" model_name=\"text-davinci-002\",\n",
|
|
" callbacks=[PromptLayerCallbackHandler(pl_id_callback=pl_id_callback)],\n",
|
|
")\n",
|
|
"\n",
|
|
"example_prompt = promptlayer.prompts.get(\"example\", version=1, langchain=True)\n",
|
|
"openai_llm(example_prompt.format(product=\"toasters\"))"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"That is all it takes! After setup all your requests will show up on the PromptLayer dashboard.\n",
|
|
"This callback also works with any LLM implemented on LangChain."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "base",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.8.8 (default, Apr 13 2021, 12:59:45) \n[Clang 10.0.0 ]"
|
|
},
|
|
"vscode": {
|
|
"interpreter": {
|
|
"hash": "c4fe2cd85a8d9e8baaec5340ce66faff1c77581a9f43e6c45e85e09b6fced008"
|
|
}
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|