mirror of https://github.com/hwchase17/langchain
Harrison/tracing docs (#804)
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>pull/806/head
parent
ae1b589f60
commit
f3da4dc6ba
@ -0,0 +1,53 @@
|
|||||||
|
# LangChain Tracing Instructions
|
||||||
|
|
||||||
|
First, you should install tracing and set up your environment properly.
|
||||||
|
You can use either a locally hosted version of this (uses Docker) or a cloud hosted version (in closed alpha).
|
||||||
|
|
||||||
|
- [Locally Hosted Setup](./tracing/local_installation.md)
|
||||||
|
- [Cloud Hosted Setup](./tracing/hosted_installation.md)
|
||||||
|
|
||||||
|
## Tracing Walkthrough
|
||||||
|
|
||||||
|
When you first access the UI, you should see a page with your tracing sessions.
|
||||||
|
An initial one "default" should already be created for you.
|
||||||
|
A session is just a way to group traces together.
|
||||||
|
If you click on a session, it will take you to a page with no recorded traces that says "No Runs."
|
||||||
|
You can create a new session with the new session form.
|
||||||
|
|
||||||
|
![](tracing/homepage.png)
|
||||||
|
|
||||||
|
If we click on the `default` session, we can see that to start we have no traces stored.
|
||||||
|
|
||||||
|
![](tracing/default_empty.png)
|
||||||
|
|
||||||
|
If we now start running chains and agents with tracing enabled, we will see data show up here.
|
||||||
|
To do so, we can run [this notebook](tracing/agent_with_tracing.ipynb) as an example.
|
||||||
|
After running it, we will see an initial trace show up.
|
||||||
|
|
||||||
|
![](tracing/first_trace.png)
|
||||||
|
|
||||||
|
From here we can explore the trace at a high level by clicking on the arrow to show nested runs.
|
||||||
|
We can keep on clicking further and further down to explore deeper and deeper.
|
||||||
|
|
||||||
|
![](tracing/explore.png)
|
||||||
|
|
||||||
|
We can also click on the "Explore" button of the top level run to dive even deeper.
|
||||||
|
Here, we can see the inputs and outputs in full, as well as all the nested traces.
|
||||||
|
|
||||||
|
![](tracing/explore_trace.png)
|
||||||
|
|
||||||
|
We can keep on exploring each of these nested traces in more detail.
|
||||||
|
For example, here is the lowest level trace with the exact inputs/outputs to the LLM.
|
||||||
|
|
||||||
|
![](tracing/explore_llm.png)
|
||||||
|
|
||||||
|
## Changing Sessions
|
||||||
|
1. To initially record traces to a session other than `"default"`, you can set the `LANGCHAIN_SESSION` environment variable to the name of the session you want to record to:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import os
|
||||||
|
os.environ["LANGCHAIN_HANDLER"] = "langchain"
|
||||||
|
os.environ["LANGCHAIN_SESSION"] = "my_session" # Make sure this session actually exists. You can create a new session in the UI.
|
||||||
|
```
|
||||||
|
|
||||||
|
2. To switch sessions mid-script or mid-notebook, do NOT set the `LANGCHAIN_SESSION` environment variable. Instead: `langchain.set_tracing_callback_manager(session_name="my_session")`
|
@ -0,0 +1,108 @@
|
|||||||
|
{
|
||||||
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 1,
|
||||||
|
"id": "17c04cc6-c93d-4b6c-a033-e897577f4ed1",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"import os\n",
|
||||||
|
"os.environ[\"LANGCHAIN_HANDLER\"] = \"langchain\"\n",
|
||||||
|
"\n",
|
||||||
|
"## Uncomment this if using hosted setup.\n",
|
||||||
|
"\n",
|
||||||
|
"# os.environ[\"LANGCHAIN_ENDPOINT\"] = \"https://langchain-api-gateway-57eoxz8z.uc.gateway.dev\" \n",
|
||||||
|
"\n",
|
||||||
|
"## Uncomment this if you want traces to be recorded to \"my_session\" instead of default.\n",
|
||||||
|
"\n",
|
||||||
|
"# os.environ[\"LANGCHAIN_SESSION\"] = \"my_session\" \n",
|
||||||
|
"\n",
|
||||||
|
"## Better to set this environment variable in the terminal\n",
|
||||||
|
"# Uncomment this if using hosted version. Replace \"my_api_key\" with your actual API Key.\n",
|
||||||
|
"\n",
|
||||||
|
"# os.environ[\"LANGCHAIN_API_KEY\"] = \"my_api_key\" \n",
|
||||||
|
"\n",
|
||||||
|
"import langchain\n",
|
||||||
|
"from langchain.agents import Tool, initialize_agent, load_tools\n",
|
||||||
|
"from langchain.llms import OpenAI"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 2,
|
||||||
|
"id": "bfa16b79-aa4b-4d41-a067-70d1f593f667",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
|
||||||
|
"\u001b[32;1m\u001b[1;3m I need to use a calculator to solve this.\n",
|
||||||
|
"Action: Calculator\n",
|
||||||
|
"Action Input: 2^.123243\u001b[0m\n",
|
||||||
|
"Observation: \u001b[36;1m\u001b[1;3mAnswer: 1.0891804557407723\n",
|
||||||
|
"\u001b[0m\n",
|
||||||
|
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n",
|
||||||
|
"Final Answer: 1.0891804557407723\u001b[0m\n",
|
||||||
|
"\n",
|
||||||
|
"\u001b[1m> Finished chain.\u001b[0m\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"text/plain": [
|
||||||
|
"'1.0891804557407723'"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"execution_count": 2,
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "execute_result"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"# Agent run with tracing. Ensure that OPENAI_API_KEY is set appropriately to run this example.\n",
|
||||||
|
"\n",
|
||||||
|
"llm = OpenAI(temperature=0)\n",
|
||||||
|
"tools = load_tools([\"llm-math\"], llm=llm)\n",
|
||||||
|
"agent = initialize_agent(\n",
|
||||||
|
" tools, llm, agent=\"zero-shot-react-description\", verbose=True\n",
|
||||||
|
")\n",
|
||||||
|
"\n",
|
||||||
|
"agent.run(\"What is 2 raised to .123243 power?\")"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": null,
|
||||||
|
"id": "0bf0304c",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [],
|
||||||
|
"source": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"metadata": {
|
||||||
|
"kernelspec": {
|
||||||
|
"display_name": "Python 3 (ipykernel)",
|
||||||
|
"language": "python",
|
||||||
|
"name": "python3"
|
||||||
|
},
|
||||||
|
"language_info": {
|
||||||
|
"codemirror_mode": {
|
||||||
|
"name": "ipython",
|
||||||
|
"version": 3
|
||||||
|
},
|
||||||
|
"file_extension": ".py",
|
||||||
|
"mimetype": "text/x-python",
|
||||||
|
"name": "python",
|
||||||
|
"nbconvert_exporter": "python",
|
||||||
|
"pygments_lexer": "ipython3",
|
||||||
|
"version": "3.10.9"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"nbformat": 4,
|
||||||
|
"nbformat_minor": 5
|
||||||
|
}
|
Binary file not shown.
After Width: | Height: | Size: 73 KiB |
Binary file not shown.
After Width: | Height: | Size: 348 KiB |
Binary file not shown.
After Width: | Height: | Size: 239 KiB |
Binary file not shown.
After Width: | Height: | Size: 253 KiB |
Binary file not shown.
After Width: | Height: | Size: 117 KiB |
Binary file not shown.
After Width: | Height: | Size: 94 KiB |
@ -0,0 +1,36 @@
|
|||||||
|
# Cloud Hosted Setup
|
||||||
|
|
||||||
|
We offer a hosted version of tracing at https://langchainplus.vercel.app/. You can use this to view traces from your run without having to run the server locally.
|
||||||
|
|
||||||
|
Note: we are currently only offering this to a limited number of users. The hosted platform is VERY alpha, in active development, and data might be dropped at any time. Don't depend on data being persisted in the system long term and don't log traces that may contain sensitive information. If you're interested in using the hosted platform, please fill out the form [here](https://forms.gle/tRCEMSeopZf6TE3b6).
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
1. Login to the system and click "API Key" in the top right corner. Generate a new key and keep it safe. You will need it to authenticate with the system.
|
||||||
|
|
||||||
|
## Environment Setup
|
||||||
|
|
||||||
|
After installation, you must now set up your environment to use tracing.
|
||||||
|
|
||||||
|
This can be done by setting an environment variable in your terminal by running `export LANGCHAIN_HANDLER=langchain`.
|
||||||
|
|
||||||
|
You can also do this by adding the below snippet to the top of every script. **IMPORTANT:** this must go at the VERY TOP of your script, before you import anything from `langchain`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
import os
|
||||||
|
os.environ["LANGCHAIN_HANDLER"] = "langchain"
|
||||||
|
```
|
||||||
|
|
||||||
|
You will also need to set an environment variable to specify the endpoint and your API key. This can be done with the following environment variables:
|
||||||
|
|
||||||
|
1. `LANGCHAIN_ENDPOINT` = "https://langchain-api-gateway-57eoxz8z.uc.gateway.dev"
|
||||||
|
2. `LANGCHAIN_API_KEY` - set this to the API key you generated during installation.
|
||||||
|
|
||||||
|
An example of adding all relevant environment variables is below:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import os
|
||||||
|
os.environ["LANGCHAIN_HANDLER"] = "langchain"
|
||||||
|
os.environ["LANGCHAIN_ENDPOINT"] = "https://langchain-api-gateway-57eoxz8z.uc.gateway.dev"
|
||||||
|
os.environ["LANGCHAIN_API_KEY"] = "my_api_key" # Don't commit this to your repo! Better to set it in your terminal.
|
||||||
|
```
|
Loading…
Reference in New Issue