diff --git a/examples/Conversational_agents_using_tools.ipynb b/examples/Conversational_agents_using_tools.ipynb new file mode 100644 index 00000000..c4635168 --- /dev/null +++ b/examples/Conversational_agents_using_tools.ipynb @@ -0,0 +1,1084 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "30995a82", + "metadata": {}, + "source": [ + "# Conversational Agents using Tools\n", + "\n", + "This notebook takes you through how to build an end-to-end LLM application powered by Langchain and OpenAI models, with access to external tools. This is a common design pattern for building products with LLMs that require:\n", + "- Access to proprietary information and APIs\n", + "- Multi-step logical reasoning\n", + "- Enhanced reliability\n", + "\n", + "By the end of this cookbook you'll be able to create tailored LLM agents that use custom tools to solve user queries.\n", + "\n", + "\n", + "## What is Langchain\n", + "LangChain is a [framework](https://python.langchain.com/en/latest/index.html) for developing applications powered by language models. Their framework enables you to build layered LLM-powered applications that are context-aware and able to interact dynamically with their environment as agents, leading to simplified code for you and a more dynamic user experience for your customers.\n", + "\n", + "## Why do LLMs need to use Tools\n", + "One of the most common challenges with LLMs is overcoming the lack of recency and specificity in their training data - answers can be out of date, and they are prone to hallucinations given the huge variety in their knowledge base. Tools are a great method of allowing an LLM to answer within a controlled context that draws on your existing knowledge bases and internal APIs - instead of trying to prompt engineer the LLM all the way to your intended answer, you allow it access to tools that it calls on dynamically for info, parses, and serves to customer. \n", + "\n", + "A classic example is Search, where rather than answering using stale training data your LLM can use a browser to source information and summarise the answer back. Tools can also work [together](https://python.langchain.com/en/latest/modules/agents/agents/examples/mrkl_chat.html), with a Search tool to return numbers and a Calculator to execute mathematical functions being one common example.\n", + "\n", + "### Flow\n", + "- **Setup:** Import packages, set any required variables and connect to a vector database.\n", + "- **LLM Agent:** Build an agent that leverages a modified version of the [ReAct](https://react-lm.github.io/) framework to do chain-of-thought reasoning.\n", + "- **LLM Agent with History:** Add the concept of memory so the LLM can draw on conversation context.\n", + "- **Knowledge Base:** Create a knowledge base of Stuff You Should Know podcast episodes that we can use as a tool.\n", + "- **LLM Agent with Tools:** Extend the agent with access to two tools and test that it uses both to answer questions." + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "id": "9c069980", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The autoreload extension is already loaded. To reload it, use:\n", + " %reload_ext autoreload\n" + ] + } + ], + "source": [ + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "markdown", + "id": "7a254003", + "metadata": {}, + "source": [ + "# Setup\n", + "\n", + "Import libraries and set up a connection to a Pinecone vector database for our knowledge base.\n", + "\n", + "You can substitute Pinecone for any other vectorstore or database - there are a [selection](https://python.langchain.com/en/latest/modules/indexes/vectorstores.html) that are supported by Langchain natively, while other connectors will need to be developed yourself." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "80fc6b58", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install pinecone-client\n", + "!pip install pandas\n", + "!pip install typing\n", + "!pip install tqdm\n", + "!pip install langchain" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "id": "18be3d9f", + "metadata": {}, + "outputs": [], + "source": [ + "# Imports\n", + "import os\n", + "import pinecone\n", + "import openai\n", + "import pandas as pd\n", + "import re\n", + "from typing import List, Union\n", + "import zipfile\n", + "import json\n", + "from tqdm.auto import tqdm\n", + "import datetime\n", + "\n", + "\n", + "# Langchain imports\n", + "from langchain.agents import Tool, AgentExecutor, LLMSingleActionAgent, AgentOutputParser\n", + "from langchain.prompts import BaseChatPromptTemplate, ChatPromptTemplate\n", + "from langchain import SerpAPIWrapper, LLMChain\n", + "from langchain.schema import AgentAction, AgentFinish, HumanMessage, SystemMessage\n", + "## LLM\n", + "from langchain.chat_models import ChatOpenAI\n", + "from langchain import OpenAI\n", + "## Langchain memory\n", + "from langchain.memory import ConversationBufferWindowMemory\n", + "# Embeddings and vectorstore\n", + "from langchain.embeddings.openai import OpenAIEmbeddings\n", + "from langchain.vectorstores import Pinecone\n", + "\n", + "# Vectorstore Index\n", + "index_name = 'podcasts'" + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "id": "af825644", + "metadata": {}, + "outputs": [], + "source": [ + "api_key = os.getenv(\"PINECONE_API_KEY\")\n", + "pinecone.init(api_key=api_key)" + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "id": "c0518596", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['podcasts']" + ] + }, + "execution_count": 68, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pinecone.list_indexes()" + ] + }, + { + "cell_type": "markdown", + "id": "7e7644cd", + "metadata": {}, + "source": [ + "Run this code block if you want to clear the index, or if the index doesn't exist yet\n", + "\n", + "```\n", + "# Check whether the index with the same name already exists - if so, delete it\n", + "if index_name in pinecone.list_indexes():\n", + " pinecone.delete_index(index_name)\n", + " \n", + "# Creates new index\n", + "pinecone.create_index(name=index_name, dimension=1536)\n", + "index = pinecone.Index(index_name=index_name)\n", + "\n", + "# Confirm our index was created\n", + "pinecone.list_indexes()\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "0eb63e37", + "metadata": {}, + "source": [ + "## LLM Agent\n", + "\n", + "An [LLM agent](https://python.langchain.com/en/latest/modules/agents/agents/custom_llm_agent.html) in Langchain has many configurable components, which are detailed in the Langchain documentation.\n", + "\n", + "We'll employ a few of the core concepts to make an agent who talks in the way we want, can use tools to answer questions, and uses the right base LLM to power the conversation.\n", + "- **Prompt Template:** The input template to control the LLM's behaviour and how it accepts inputs and produces outputs - this is the brain that drives your application ([docs](https://python.langchain.com/en/latest/modules/prompts/prompt_templates.html)).\n", + "- **Output Parser:** A method of parsing the output from the prompt. If the LLM produces output using certain headers, you can enable complex interactions where variables are generated by the LLM in their response and passed into the next step of the chain ([docs](https://python.langchain.com/en/latest/modules/prompts/output_parsers.html)).\n", + "- **LLM Chain:** A Chain brings together a prompt template with an LLM that will execute it - in this case we'll be using ```gpt-3.5-turbo``` but this framework can be used with OpenAI completions models, or other LLMs entirely ([docs](https://python.langchain.com/en/latest/modules/chains.html)).\n", + "- **Tool:** An external service that the LLM can use to retrieve information or execute commands should the user require it ([docs](https://python.langchain.com/en/latest/modules/agents/tools.html)).\n", + "- **Agent:** The glue that brings all of this together, an agent can call multiple LLM Chains, each with their own tools. Agents can be extended with your own logic to allow retries, error handling and any other methods you choose to add durability to your application ([docs](https://python.langchain.com/en/latest/modules/agents.html)).\n", + "\n", + "**NB:** Before using this cookbook with the Search tool you'll need to sign up on https://serpapi.com/ and generate an API key. Once you have it, store it in an environment variable named ```SERPAPI_API_KEY```" + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "id": "3f813668", + "metadata": {}, + "outputs": [], + "source": [ + "# Initiate a Search tool - note you'll need to have set SERPAPI_API_KEY as an environment variable as per the above instructions\n", + "search = SerpAPIWrapper()\n", + "\n", + "# Define a list of \n", + "tools = [\n", + " Tool(\n", + " name = \"Search\",\n", + " func=search.run,\n", + " description=\"useful for when you need to answer questions about current events\"\n", + " )\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "id": "effae642", + "metadata": {}, + "outputs": [], + "source": [ + "# Set up the prompt with input variables for tools, user input and a scratchpad for the model to record its workings\n", + "template = \"\"\"Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:\n", + "\n", + "{tools}\n", + "\n", + "Use the following format:\n", + "\n", + "Question: the input question you must answer\n", + "Thought: you should always think about what to do\n", + "Action: the action to take, should be one of [{tool_names}]\n", + "Action Input: the input to the action\n", + "Observation: the result of the action\n", + "... (this Thought/Action/Action Input/Observation can repeat N times)\n", + "Thought: I now know the final answer\n", + "Final Answer: the final answer to the original input question\n", + "\n", + "Begin! Remember to speak as a pirate when giving your final answer. Use lots of \"Arg\"s\n", + "\n", + "Question: {input}\n", + "{agent_scratchpad}\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "id": "b4efd85d", + "metadata": {}, + "outputs": [], + "source": [ + "# Set up a prompt template\n", + "class CustomPromptTemplate(BaseChatPromptTemplate):\n", + " # The template to use\n", + " template: str\n", + " # The list of tools available\n", + " tools: List[Tool]\n", + " \n", + " def format_messages(self, **kwargs) -> str:\n", + " # Get the intermediate steps (AgentAction, Observation tuples)\n", + " \n", + " # Format them in a particular way\n", + " intermediate_steps = kwargs.pop(\"intermediate_steps\")\n", + " thoughts = \"\"\n", + " for action, observation in intermediate_steps:\n", + " thoughts += action.log\n", + " thoughts += f\"\\nObservation: {observation}\\nThought: \"\n", + " \n", + " # Set the agent_scratchpad variable to that value\n", + " kwargs[\"agent_scratchpad\"] = thoughts\n", + " \n", + " # Create a tools variable from the list of tools provided\n", + " kwargs[\"tools\"] = \"\\n\".join([f\"{tool.name}: {tool.description}\" for tool in self.tools])\n", + " \n", + " # Create a list of tool names for the tools provided\n", + " kwargs[\"tool_names\"] = \", \".join([tool.name for tool in self.tools])\n", + " formatted = self.template.format(**kwargs)\n", + " return [HumanMessage(content=formatted)]\n", + " \n", + "prompt = CustomPromptTemplate(\n", + " template=template,\n", + " tools=tools,\n", + " # This omits the `agent_scratchpad`, `tools`, and `tool_names` variables because those are generated dynamically\n", + " # This includes the `intermediate_steps` variable because that is needed\n", + " input_variables=[\"input\", \"intermediate_steps\"]\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "id": "533657a5", + "metadata": {}, + "outputs": [], + "source": [ + "class CustomOutputParser(AgentOutputParser):\n", + " \n", + " def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:\n", + " \n", + " # Check if agent should finish\n", + " if \"Final Answer:\" in llm_output:\n", + " return AgentFinish(\n", + " # Return values is generally always a dictionary with a single `output` key\n", + " # It is not recommended to try anything else at the moment :)\n", + " return_values={\"output\": llm_output.split(\"Final Answer:\")[-1].strip()},\n", + " log=llm_output,\n", + " )\n", + " \n", + " # Parse out the action and action input\n", + " regex = r\"Action: (.*?)[\\n]*Action Input:[\\s]*(.*)\"\n", + " match = re.search(regex, llm_output, re.DOTALL)\n", + " \n", + " # If it can't parse the output it raises an error\n", + " # You can add your own logic here to handle errors in a different way i.e. pass to a human, give a canned response\n", + " if not match:\n", + " raise ValueError(f\"Could not parse LLM output: `{llm_output}`\")\n", + " action = match.group(1).strip()\n", + " action_input = match.group(2)\n", + " \n", + " # Return the action and action input\n", + " return AgentAction(tool=action, tool_input=action_input.strip(\" \").strip('\"'), log=llm_output)\n", + " \n", + "output_parser = CustomOutputParser()" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "id": "4c7d93d1", + "metadata": {}, + "outputs": [], + "source": [ + "# Initiate our LLM - default is 'gpt-3.5-turbo'\n", + "llm = ChatOpenAI(temperature=0)\n", + "\n", + "# LLM chain consisting of the LLM and a prompt\n", + "llm_chain = LLMChain(llm=llm, prompt=prompt)\n", + "\n", + "# Using tools, the LLM chain and output_parser to make an agent\n", + "tool_names = [tool.name for tool in tools]\n", + "\n", + "agent = LLMSingleActionAgent(\n", + " llm_chain=llm_chain, \n", + " output_parser=output_parser,\n", + " # We use \"Observation\" as our stop sequence so it will stop when it receives Tool output\n", + " # If you change your prompt template you'll need to adjust this as well\n", + " stop=[\"\\nObservation:\"], \n", + " allowed_tools=tool_names\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "id": "dd893fa5", + "metadata": {}, + "outputs": [], + "source": [ + "# Initiate the agent that will respond to our queries\n", + "# Set verbose=True to share the CoT reasoning the LLM goes through\n", + "agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "id": "c7ca3ee2", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", + "\u001b[32;1m\u001b[1;3mThought: Hmm...I don't be knowin' the answer to that off the top of me head.\n", + "Action: Search\n", + "Action Input: \"Canada population 2023\"\u001b[0m\n", + "\n", + "Observation:\u001b[36;1m\u001b[1;3mThe current population of Canada is 38,661,927 as of Sunday, April 16, 2023, based on Worldometer elaboration of the latest United Nations data.\u001b[0m\u001b[32;1m\u001b[1;3mAhoy, that be a lot of landlubbers! \n", + "Final Answer: The current population of Canada as of 2023 be 38,661,927. Arg!\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "'The current population of Canada as of 2023 be 38,661,927. Arg!'" + ] + }, + "execution_count": 75, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent_executor.run(\"How many people live in canada as of 2023?\")" + ] + }, + { + "cell_type": "markdown", + "id": "c3b51817", + "metadata": {}, + "source": [ + "## LLM Agent with History\n", + "\n", + "Extend the LLM Agent with the ability to retain a [memory](https://python.langchain.com/en/latest/modules/agents/agents/custom_llm_agent.html#adding-memory) and use it as context as it continues the conversation.\n", + "\n", + "We use a simple ConversationBufferWindowMemory for this example that keeps a rolling window of the last two conversation turns, but LangChain have many other [memory options](https://python.langchain.com/en/latest/modules/memory.html) depending on your use case." + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "id": "85737eea", + "metadata": {}, + "outputs": [], + "source": [ + "# Set up a prompt template which can interpolate the history\n", + "template_with_history = \"\"\"You are SearchGPT, a professional search engine who aims to provide informative answers to users. Answer the following questions as best you can. You have access to the following tools:\n", + "\n", + "{tools}\n", + "\n", + "Use the following format:\n", + "\n", + "Question: the input question you must answer\n", + "Thought: you should always think about what to do\n", + "Action: the action to take, should be one of [{tool_names}]\n", + "Action Input: the input to the action\n", + "Observation: the result of the action\n", + "... (this Thought/Action/Action Input/Observation can repeat N times)\n", + "Thought: I now know the final answer\n", + "Final Answer: the final answer to the original input question\n", + "\n", + "Begin! Remember to give detailed, informative answers\n", + "\n", + "Previous conversation history:\n", + "{history}\n", + "\n", + "New question: {input}\n", + "{agent_scratchpad}\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "id": "4d85c309", + "metadata": {}, + "outputs": [], + "source": [ + "prompt_with_history = CustomPromptTemplate(\n", + " template=template_with_history,\n", + " tools=tools,\n", + " # The history template includes \"history\" as an input variable so we can interpolate it into the prompt\n", + " input_variables=[\"input\", \"intermediate_steps\", \"history\"]\n", + ")\n", + "\n", + "llm_chain = LLMChain(llm=llm, prompt=prompt_with_history)\n", + "tool_names = [tool.name for tool in tools]\n", + "agent = LLMSingleActionAgent(\n", + " llm_chain=llm_chain, \n", + " output_parser=output_parser,\n", + " stop=[\"\\nObservation:\"], \n", + " allowed_tools=tool_names\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 93, + "id": "4f8c8fea", + "metadata": {}, + "outputs": [], + "source": [ + "# Initiate the memory with k=2 to keep the last two turns\n", + "# Provide the memory to the agent\n", + "memory = ConversationBufferWindowMemory(k=2)\n", + "agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True, memory=memory)" + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "id": "e1f031bb", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", + "\u001b[32;1m\u001b[1;3mThought: This is a question asking for a factual number. I should use the Search tool to look for reputable sources that provide current population estimates for Canada.\n", + "Action: Search\n", + "Action Input: \"Canada population 2023\"\u001b[0m\n", + "\n", + "Observation:\u001b[36;1m\u001b[1;3mThe current population of Canada is 38,661,927 as of Sunday, April 16, 2023, based on Worldometer elaboration of the latest United Nations data.\u001b[0m\u001b[32;1m\u001b[1;3mBased on the observation, I can now provide the final answer to the question.\n", + "Final Answer: As of 2023, the population of Canada is estimated to be 38,661,927.\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "'As of 2023, the population of Canada is estimated to be 38,661,927.'" + ] + }, + "execution_count": 94, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent_executor.run(\"How many people live in canada as of 2023?\")" + ] + }, + { + "cell_type": "code", + "execution_count": 95, + "id": "9d28d73f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", + "\u001b[32;1m\u001b[1;3mThought: I need to find the population of Mexico as of the most recent data available.\n", + "Action: Search\n", + "Action Input: \"Mexico population [current year]\"\u001b[0m\n", + "\n", + "Observation:\u001b[36;1m\u001b[1;3mUnited Nations projections are also included through the year 2100. The current population of Mexico in 2023 is 128,455,567, a 0.75% increase from 2022. The population of Mexico in 2022 was 127,504,125, a 0.63% increase from 2021. The population of Mexico in 2021 was 126,705,138, a 0.56% increase from 2020.\u001b[0m\u001b[32;1m\u001b[1;3mI now have the most recent data available for the population of Mexico.\n", + "Final Answer: The population of Mexico as of 2023 is estimated to be 128,455,567.\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "'The population of Mexico as of 2023 is estimated to be 128,455,567.'" + ] + }, + "execution_count": 95, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent_executor.run(\"how about in mexico?\")" + ] + }, + { + "cell_type": "markdown", + "id": "62356eed", + "metadata": {}, + "source": [ + "## Knowledge base\n", + "\n", + "Create a custom vectorstore for the Agent to use as a Tool to answer questions with. We'll store the results in [Pinecone](https://docs.pinecone.io/docs/quickstart), which is supported by LangChain ([Docs](https://python.langchain.com/en/latest/modules/indexes/vectorstores/examples/pinecone.html), [API reference](https://python.langchain.com/en/latest/reference/modules/vectorstore.html)). For help getting started with Pinecone or other vector databases, we have a [cookbook](https://github.com/openai/openai-cookbook/blob/colin/examples/vector_databases/Using_vector_databases_for_embeddings_search.ipynb) to help you get started.\n", + "\n", + "You can check the LangChain documentation to see what other [vectorstores](https://python.langchain.com/en/latest/modules/indexes/vectorstores.html) and [databases](https://python.langchain.com/en/latest/modules/chains/examples/sqlite.html) are available.\n", + "\n", + "For this example we'll use the transcripts of the Stuff You Should Know podcast, which was provided thanks to OSF DOI [10.17605/OSF.IO/VM9NT](https://doi.org/10.17605/OSF.IO/VM9NT)" + ] + }, + { + "cell_type": "code", + "execution_count": 109, + "id": "87f86008", + "metadata": {}, + "outputs": [], + "source": [ + "# TODO: Need to update this cell to have a link to the below zip file on the CDN\n", + "\n", + "import wget\n", + "# embeddings_url = 'TBD'\n", + "\n", + "#### The file is ~541 MB so this will take some time\n", + "#wget.download(embeddings_url)" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "id": "4889c292", + "metadata": {}, + "outputs": [], + "source": [ + "with zipfile.ZipFile(\"sysk_podcast_transcripts_embedded.json.zip\",\"r\") as zip_ref:\n", + " zip_ref.extractall(\"./data\")\n", + "\n", + "# Opening JSON file\n", + "f = open('./data/sysk_podcast_transcripts_embedded.json')\n", + " \n", + "# Loads podcasts as a dictionary\n", + "processed_podcasts = json.load(f)" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "id": "1070ef60", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
idfilenametitleurltext_chunkembeddingcleaned_id
0sysk_with_transcripts_SYSK Selects How Crime S...sysk_with_transcripts_SYSK Selects How Crime S...\\n\\nSYSK Selects How Crime Scene Cleanup Workshttps://chtbl.com/track/5899E/podtrac.com/pts/...Title: sysk_with_transcripts_SYSK Selects How ...[0.021279960870742798, -0.005817972123622894, ...sysk_with_transcripts_SYSK Selects How Crime S...
1sysk_with_transcripts_SYSK Selects How Crime S...sysk_with_transcripts_SYSK Selects How Crime S...\\n\\nSYSK Selects How Crime Scene Cleanup Workshttps://chtbl.com/track/5899E/podtrac.com/pts/...Title: sysk_with_transcripts_SYSK Selects How ...[0.013859338127076626, 0.00857278611510992, 0....sysk_with_transcripts_SYSK Selects How Crime S...
2sysk_with_transcripts_SYSK Selects How Crime S...sysk_with_transcripts_SYSK Selects How Crime S...\\n\\nSYSK Selects How Crime Scene Cleanup Workshttps://chtbl.com/track/5899E/podtrac.com/pts/...Title: sysk_with_transcripts_SYSK Selects How ...[0.015242221765220165, 0.016030369326472282, 0...sysk_with_transcripts_SYSK Selects How Crime S...
3sysk_with_transcripts_SYSK Selects How Crime S...sysk_with_transcripts_SYSK Selects How Crime S...\\n\\nSYSK Selects How Crime Scene Cleanup Workshttps://chtbl.com/track/5899E/podtrac.com/pts/...Title: sysk_with_transcripts_SYSK Selects How ...[0.004371842369437218, -0.003036574460566044, ...sysk_with_transcripts_SYSK Selects How Crime S...
4sysk_with_transcripts_SYSK Selects How Crime S...sysk_with_transcripts_SYSK Selects How Crime S...\\n\\nSYSK Selects How Crime Scene Cleanup Workshttps://chtbl.com/track/5899E/podtrac.com/pts/...Title: sysk_with_transcripts_SYSK Selects How ...[0.017309172078967094, 0.015154214575886726, 0...sysk_with_transcripts_SYSK Selects How Crime S...
\n", + "
" + ], + "text/plain": [ + " id \\\n", + "0 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "1 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "2 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "3 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "4 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "\n", + " filename \\\n", + "0 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "1 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "2 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "3 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "4 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "\n", + " title \\\n", + "0 \\n\\nSYSK Selects How Crime Scene Cleanup Works \n", + "1 \\n\\nSYSK Selects How Crime Scene Cleanup Works \n", + "2 \\n\\nSYSK Selects How Crime Scene Cleanup Works \n", + "3 \\n\\nSYSK Selects How Crime Scene Cleanup Works \n", + "4 \\n\\nSYSK Selects How Crime Scene Cleanup Works \n", + "\n", + " url \\\n", + "0 https://chtbl.com/track/5899E/podtrac.com/pts/... \n", + "1 https://chtbl.com/track/5899E/podtrac.com/pts/... \n", + "2 https://chtbl.com/track/5899E/podtrac.com/pts/... \n", + "3 https://chtbl.com/track/5899E/podtrac.com/pts/... \n", + "4 https://chtbl.com/track/5899E/podtrac.com/pts/... \n", + "\n", + " text_chunk \\\n", + "0 Title: sysk_with_transcripts_SYSK Selects How ... \n", + "1 Title: sysk_with_transcripts_SYSK Selects How ... \n", + "2 Title: sysk_with_transcripts_SYSK Selects How ... \n", + "3 Title: sysk_with_transcripts_SYSK Selects How ... \n", + "4 Title: sysk_with_transcripts_SYSK Selects How ... \n", + "\n", + " embedding \\\n", + "0 [0.021279960870742798, -0.005817972123622894, ... \n", + "1 [0.013859338127076626, 0.00857278611510992, 0.... \n", + "2 [0.015242221765220165, 0.016030369326472282, 0... \n", + "3 [0.004371842369437218, -0.003036574460566044, ... \n", + "4 [0.017309172078967094, 0.015154214575886726, 0... \n", + "\n", + " cleaned_id \n", + "0 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "1 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "2 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "3 sysk_with_transcripts_SYSK Selects How Crime S... \n", + "4 sysk_with_transcripts_SYSK Selects How Crime S... " + ] + }, + "execution_count": 41, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Have a look at the contents\n", + "pd.DataFrame(processed_podcasts).head()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f9715d1f", + "metadata": {}, + "outputs": [], + "source": [ + "batch_size = 100 # how many embeddings we create and insert at once\n", + "\n", + "for i in tqdm(range(0, len(processed_podcasts), batch_size)):\n", + " # find end of batch\n", + " i_end = min(len(processed_podcasts), i+batch_size)\n", + " meta_batch = processed_podcasts[i:i_end]\n", + " # get ids\n", + " ids_batch = [x['cleaned_id'] for x in meta_batch]\n", + " # get texts to encode\n", + " texts = [x['text_chunk'] for x in meta_batch]\n", + " # add embeddings\n", + " embeds = [x['embedding'] for x in meta_batch]\n", + " # cleanup metadata\n", + " meta_batch = [{\n", + " 'filename': x['filename'],\n", + " 'title': x['title'],\n", + " 'text_chunk': x['text_chunk'],\n", + " 'url': x['url']\n", + " } for x in meta_batch]\n", + " to_upsert = list(zip(ids_batch, embeds, meta_batch))\n", + " # upsert to Pinecone\n", + " index.upsert(vectors=to_upsert)" + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "id": "3c5bac10", + "metadata": {}, + "outputs": [], + "source": [ + "# Configuring the embeddings to be used by our retriever to be OpenAI Embeddings, matching our embedded corpus\n", + "embeddings = OpenAIEmbeddings()\n", + "\n", + "# Loads a docsearch object from an existing Pinecone index so we can retrieve from it\n", + "docsearch = Pinecone.from_existing_index(index_name,embeddings,text_key='text_chunk')" + ] + }, + { + "cell_type": "code", + "execution_count": 98, + "id": "5dd21d92", + "metadata": {}, + "outputs": [], + "source": [ + "retriever = docsearch.as_retriever()" + ] + }, + { + "cell_type": "code", + "execution_count": 99, + "id": "8abd34e5", + "metadata": {}, + "outputs": [], + "source": [ + "query_docs = retriever.get_relevant_documents(\"can you live without a bank account\")" + ] + }, + { + "cell_type": "code", + "execution_count": 107, + "id": "016bd458", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Title: sysk: Can You Live Without a Bank Account?\n", + "\n", + "Title: sysk_with_transcripts_Can you live without a bank account.json; And if you had a life, you didn't necessarily rectify your bank checkbook every day. Oh, wait, what is balancing a checkbook mean? Seriously? Yeah. Thank God for my wife. So another reason you might avoid a bank is philosophically. There may be a longstanding distrust of banks in your family that you don't want to put your money in, or you may just want to be like, you know what? I don't want to take part in this modern society. I want to kind of drop out a bit. And a really good first move is to shut your bank account down. That's a big statement. Oh, yeah, it is. But a lot of people that are underbanked and don't have accounts aren't there on purpose. It's not some philosophical statement. A lot of times it's simply because they are poor and they don't have a lot of alternatives. Yeah. And the other thing about not having a bank account, not only do you not have a bank account, you also are, like, basically just avoiding banks altogether. There's plenty of other things that banks offer, like loans and mortgage, lollipops, stuff like that. Yeah. Maybe some free nasty coffee. So when you don't have a banking account, that's like the most basic unit of the banking world. Right. If you don't have that, you obviously aren't going to be exposed to all these other things that can help. Things like build your credit history through like a revolving loan or a mortgage or a car loan or something like that that you can build up your credit for and ultimately save money. So when you don't have a bank account, for whatever reason, you are effectively out of the banking system. The problem is you can live parallel to the banking system outside of it, but it can be really dangerous, especially if you're just dealing with cash, because that cash has to stay somewhere, whether it's on you or in your mattress or in a coffee can in your backyard. You're exposed for having that readily available to anybody who finds it or comes into your house with a gun to get it. Yes.\n", + "\n", + "\n", + "Title: sysk: Can You Live Without a Bank Account?\n", + "\n", + "Title: sysk_with_transcripts_Can you live without a bank account.json; And it doesn't have to be an everyday thing. You can host when you want. Like, let's say you're taking a week's vacation. Why not host your home? Because that money could go toward paying for your current vacation or towards your retirement fund or even towards your kids college fund. Yeah. For anything. And listen, if you're worried about your stuff, don't be. Air cover for hosts. Let hosts welcome guests into their home without having to worry. You get $1 million in damage protection anytime you're hosting. Plus pet damage protection and income loss protection, too. And are you ready for this? Air cover for host is completely free every time you host on airbnb. Free with a capital F, with air cover for Host. It makes hosting a no brainer, and the benefits really start adding up. So learn more and host with peace of mind at Airbnb comaircoverforhosts. Capital One offers commercial solutions you can bank on. Now more than ever, your business faces specific challenges and unique opportunities. That's why Capital One offers a comprehensive suite of financial services custom tailored to your short and long term goals, backed by the expertise, strategy and resources of a top ten commercial bank, a dedicated team works with you to support your success and help you achieve your goals. Explore the possibilities at CapitalOne. comCOMMERCIAL all right, so if you live in modern society today, it is pretty tough to get by without a bank. Most cases these days you have well, I don't know about most cases, but in many cases you have automatic deposits of your work checks. Sure. A lot of people pay their bills wirelessly, online, directly from their bank. You might have a student loan, you might have a car loan, you might have your house mortgage, you might pay your credit card bills. All this stuff is running through a bank, most likely. And you would think it's probably impossible to not have a bank account these days. And I would say pretty much all Americans have them. Not true. Well, pretty much all Americans do. Like 93% do. Yeah, but that's not all. No, it's true.\n", + "\n", + "\n", + "Title: sysk: Can You Live Without a Bank Account?\n", + "\n", + "Title: sysk_with_transcripts_Can you live without a bank account.json; Yeah. 7% of Americans do not have bank accounts. About 9 million people last year in 2015 did not have bank accounts. 9 million people is a lot of people. No, it really is. And apparently that's household sorry, not people. Yeah, right. You're that is a big distinction, too. And the FDIC said, man, that's the lowest since we've been tracking this by far. And someone said, well, how long have you been tracking this? They said, well, the last six years. Really? Yeah, which I'm like. Really? That's when they started tracking it, but apparently so 2009. So if you want another number, the 9 million American households don't have bank accounts at all, then there are 25 million households in addition to that. So that makes almost like 34 million households, which that's a substantial number at this point. Sure. The 25 million are what's called underbanked, meaning they may have a bank account, but they don't use the bank account. Yeah. They don't use it because they are probably afraid of overdraft fees. Or they have maybe a bank account that got grandfathered in so that they don't have to pay minimum amount fees. And who knows? There's all sorts of reasons for people to not use a bank account that they have, but probably cheap among them is overdressed, which you'll talk more about. Yeah. And the majority of these underbank people in the United States are poor, usually. A lot of times they're minorities, a lot of times they're less educated. And these communities, there's a few reasons why they may not want to use a bank one. Maybe they don't trust banks. And if you look in the history of the United States or certainly even we're just talking about the Wells Fargo scandal, when you see stuff like that on the news, it should be upsetting to everyone. But obviously if you're poor and you don't have a lot of money, that may scare you into not wanting to use a bank at all. Right? Yeah.\n", + "\n", + "\n", + "Title: sysk: Can You Live Without a Bank Account?\n", + "\n", + "Title: sysk_with_transcripts_Can you live without a bank account.json; Maybe at the time, I might be making it up. I seem to remember them saying that, and I was like, I don't want that. Just let the check bounce and I'll take it up with them. Yes. The way it was marketed, though, was like, hey, we value you. We want to make sure that you can pay all your bills. So if something happens and you're overdrafted we'll cover it. We're just going to charge you a fee. And it sounds good, but again, when you go from high to low and all of a sudden your overdraft fees go from one to four or five or however many, that's a huge problem. Well, and the people that are overdrafting and the people that are at least able to afford those fees. Exactly. So it's a disproportionate burden on the poor, which makes it, as a scam, one of the more evil scams around. Yes. It's just wrong, then the idea that if you open an account, you should not opt in for overdraft protection. And it's easy to say when you're talking about checks for, like you're writing a check for a Mountain Dew and some cheetos. Yeah, who cares if you're short for that? You can go without that. But when you're talking about your rent check or like an actual grocery bill or something like that, it sucks that you can't get that stuff. But it's better to have to put a couple of things back than to pay $35 for one $2 item that you went over by, right? Yeah, that's a good point. And this was in my case, too. This is also back in the day when you I mean, a lot of times it was a mystery how much you had in your account. Right. Like, you couldn't just get on your phone before you write the check and be like, oh, well, no, I don't have enough money to cover this. Yeah, because even if you balanced your checkbook, sometimes you forgot to carry the one, it wasn't always 100% accurate.\n", + "\n", + "\n" + ] + } + ], + "source": [ + "# Print out the title and content for the most relevant retrieved documents\n", + "print(\"\\n\".join(['Title: ' + x.metadata['title'].strip() + '\\n\\n' + x.page_content + '\\n\\n' for x in query_docs]))" + ] + }, + { + "cell_type": "markdown", + "id": "2bf10f53", + "metadata": {}, + "source": [ + "## LLM Agent with Tools\n", + "\n", + "Extend our list of tools by creating a [RetrievalQA](https://python.langchain.com/en/latest/modules/chains/index_examples/vector_db_qa.html) chain leveraging our Pinecone knowledge base." + ] + }, + { + "cell_type": "code", + "execution_count": 101, + "id": "a6cae4cf", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.chains import RetrievalQA\n", + "\n", + "retrieval_llm = OpenAI(temperature=0)\n", + "\n", + "podcast_retriever = RetrievalQA.from_chain_type(llm=retrieval_llm, chain_type=\"stuff\", retriever=docsearch.as_retriever())" + ] + }, + { + "cell_type": "code", + "execution_count": 102, + "id": "0cc282cd", + "metadata": {}, + "outputs": [], + "source": [ + "expanded_tools = [\n", + " Tool(\n", + " name = \"Search\",\n", + " func=search.run,\n", + " description=\"useful for when you need to answer questions about current events\"\n", + " ),\n", + " Tool(\n", + " name = 'Knowledge Base',\n", + " func=podcast_retriever.run,\n", + " description=\"Useful for general questions about how to do things and for details on interesting topics. Input should be a fully formed question.\"\n", + " )\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 103, + "id": "d1e12121", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['Search', 'Knowledge Base']\n" + ] + } + ], + "source": [ + "# Reinitiate the prompt with our new list of tools\n", + "prompt_with_history = CustomPromptTemplate(\n", + " template=template_with_history,\n", + " tools=expanded_tools,\n", + " input_variables=[\"input\", \"intermediate_steps\", \"history\"]\n", + ")\n", + "\n", + "llm_chain = LLMChain(llm=llm, prompt=prompt_with_history)\n", + "multi_tool_names = [tool.name for tool in expanded_tools]\n", + "\n", + "# Print these to check we have our new tool represented\n", + "print(multi_tool_names)\n", + "multi_tool_agent = LLMSingleActionAgent(\n", + " llm_chain=llm_chain, \n", + " output_parser=output_parser,\n", + " stop=[\"\\nObservation:\"], \n", + " allowed_tools=multi_tool_names\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 104, + "id": "2ac8eaa2", + "metadata": {}, + "outputs": [], + "source": [ + "multi_tool_memory=ConversationBufferWindowMemory(k=2)\n", + "multi_tool_executor = AgentExecutor.from_agent_and_tools(agent=multi_tool_agent, tools=expanded_tools, verbose=True, memory=multi_tool_memory)" + ] + }, + { + "cell_type": "code", + "execution_count": 105, + "id": "f27bb1de", + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", + "\u001b[32;1m\u001b[1;3mThought: This is an interesting question. I think it would be best to consult our Knowledge Base for some information on this.\n", + "Action: Knowledge Base\n", + "Action Input: \"How can you live without a bank account?\"\u001b[0m\n", + "\n", + "Observation:\u001b[33;1m\u001b[1;3m You can live without a bank account by avoiding banks altogether and dealing with cash. This can be dangerous, however, as cash can be stolen or lost. You can also host on Airbnb and use the money to pay for your current vacation or towards your retirement fund or even towards your kids college fund.\u001b[0m\u001b[32;1m\u001b[1;3mThat was a helpful insight, but it would be useful to know more about the risks and benefits of living without a bank account.\n", + "Action: Search\n", + "Action Input: \"Pros and cons of living without a bank account\"\u001b[0m\n", + "\n", + "Observation:\u001b[36;1m\u001b[1;3mAnother major disadvantage of living without a bank account is that there is no paper or electronic trail in place.\u001b[0m\u001b[32;1m\u001b[1;3mI now know the final answer. \n", + "Final Answer: Living without a bank account is possible by dealing with cash, but it comes with risks such as theft and loss of money. It also means there is no paper or electronic trail. One alternative is to use services like Airbnb to generate income that can be used towards expenses or investments.\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "'Living without a bank account is possible by dealing with cash, but it comes with risks such as theft and loss of money. It also means there is no paper or electronic trail. One alternative is to use services like Airbnb to generate income that can be used towards expenses or investments.'" + ] + }, + "execution_count": 105, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "multi_tool_executor.run(\"Hi, I'd like to know how you can live without a bank account\")" + ] + }, + { + "cell_type": "code", + "execution_count": 106, + "id": "cba815cf", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", + "\u001b[32;1m\u001b[1;3mThought: There are many factors to consider when evaluating whether zoos are good or bad for animals.\n", + "Action: Knowledge Base\n", + "Action Input: \"What are the arguments for and against zoos?\"\u001b[0m\n", + "\n", + "Observation:\u001b[33;1m\u001b[1;3m The arguments for zoos include that they have gotten a lot better in the last 30-40 years, they participate in research and conservation projects, and they can help save species from extinction. The arguments against zoos include that they are still businesses, they can be counterproductive in terms of educating the public, and they can have a negative impact on the life span of animals in captivity.\u001b[0m\u001b[32;1m\u001b[1;3mThis information provides a good foundation for answering the original question but I need to do more research to find interesting facts.\n", + "Action: Search\n", + "Action Input: \"Interesting facts about zoos and their impact on animals\"\u001b[0m\n", + "\n", + "Observation:\u001b[36;1m\u001b[1;3mAnimal Abuse in Zoos Statistics\u001b[0m\u001b[32;1m\u001b[1;3mThis information highlights a negative aspect of zoos that should be taken into consideration when evaluating their overall impact on animals.\n", + "Action: Knowledge Base\n", + "Action Input: \"What are some positive effects of zoos on animals?\"\u001b[0m\n", + "\n", + "Observation:\u001b[33;1m\u001b[1;3m Zoos have gotten a lot better in the last 4000 years and a lot better in the last 30 years from what I've seen. They have kind of kept in step with the progress of mental institutions. They have replaced fences with moats, so the animals can't get to visitors. They have created more miniature habitats that the animals would normally live in. They participate in research and conservation projects in more than 80 countries. They help to preserve species that would otherwise be extinct, such as the deer, the pere David's deer, and the California condor.\u001b[0m\u001b[32;1m\u001b[1;3mThis information provides a more balanced view of the impact of zoos on animals, but I want to find a specific fact that will be interesting to the user.\n", + "Action: Search\n", + "Action Input: \"Interesting fact about zoos and animal behavior\"\u001b[0m\n", + "\n", + "Observation:\u001b[36;1m\u001b[1;3m1. Zoos are miserable places for animals · 2. Zoos can't provide sufficient space · 3. Animals suffer in zoos · 4. Animals die prematurely in zoos.\u001b[0m\u001b[32;1m\u001b[1;3mThis information is not specific enough to answer the original question. I will refine my search to find a specific fact.\n", + "Action: Search\n", + "Action Input: \"Interesting fact about animal behavior in zoos\"\u001b[0m\n", + "\n", + "Observation:\u001b[36;1m\u001b[1;3m1. Zoos are miserable places for animals · 2. Zoos can't provide sufficient space · 3. Animals suffer in zoos · 4. Animals die prematurely in zoos.\u001b[0m\u001b[32;1m\u001b[1;3mThis search did not provide me with any new information. I will try a different approach.\n", + "Action: Search\n", + "Action Input: \"Examples of behavior changes in animals in zoos compared to the wild\"\u001b[0m\n", + "\n", + "Observation:\u001b[36;1m\u001b[1;3mIf you've ever been to see a mammal in captivity, you've likely seen them exhibit behaviors they wouldn't engage in in the wild. Repetitive, often destructive behaviors like swimming in circles, grinding their teeth, and bobbing their heads are all symptoms of the effects of captivity on the animal's brain.\u001b[0m\u001b[32;1m\u001b[1;3mThis information provides an interesting insight into the impact of zoos on animal behavior, but I want to find more examples.\n", + "Action: Search\n", + "Action Input: \"Specific examples of animal behavior changes in zoos\"\u001b[0m\n", + "\n", + "Observation:\u001b[36;1m\u001b[1;3mIf you've ever been to see a mammal in captivity, you've likely seen them exhibit behaviors they wouldn't engage in in the wild. Repetitive, often destructive behaviors like swimming in circles, grinding their teeth, and bobbing their heads are all symptoms of the effects of captivity on the animal's brain.\u001b[0m\u001b[32;1m\u001b[1;3mThis seems to be the only information available on this topic, but it is still an interesting fact that zoos can cause behavior changes in animals. \n", + "Final Answer: Zoos can have both positive and negative impacts on animals. They participate in conservation and research projects and can help preserve endangered species. However, zoos can also cause behavior changes in animals due to the effects of captivity on their brains. It is important to consider the pros and cons when evaluating whether zoos are good or bad for animals.\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "'Zoos can have both positive and negative impacts on animals. They participate in conservation and research projects and can help preserve endangered species. However, zoos can also cause behavior changes in animals due to the effects of captivity on their brains. It is important to consider the pros and cons when evaluating whether zoos are good or bad for animals.'" + ] + }, + "execution_count": 106, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "multi_tool_executor.run('Can you tell me some interesting facts about whether zoos are good or bad for animals')" + ] + }, + { + "cell_type": "markdown", + "id": "efe408a3", + "metadata": {}, + "source": [ + "You now have a template to deploy conversational agents with tools. If you want to extend this with a Custom Agent to add your own retry behaviour or treatment of input/output variables, then follow [this article](https://python.langchain.com/en/latest/modules/agents/agents/custom_agent.html).\n", + "\n", + "We look forward to seeing what you build!" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "pchain_env", + "language": "python", + "name": "pchain_env" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.10" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}