langchain-notebooks/llms.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "jukit_cell_id": "gOPXQazOh0"
   },
   "source": [
    "# Getting Started"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "jukit_cell_id": "yMwLn4kyVM"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side!'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.llms import OpenAI\n",
    "\n",
    "# n: how many completions to generate for each prompt\n",
    "llm = OpenAI(model_name='text-ada-001', n=2, best_of=2, temperature=1)\n",
    "llm(\"tell me a joke !\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jukit_cell_id": "5F2CisYISi"
   },
   "source": [
    "you can call it with a list of inputs, getting back a more complete response than just the text. This complete response includes things like multiple top responses, as well as LLM provider specific information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jukit_cell_id": "4hSGpG9eHG"
   },
   "outputs": [],
   "source": [
    "llm_result = llm.generate([\"Tell me a joke\", \"Tell me a poem\"] * 15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "jukit_cell_id": "kZMW1cS1Qk"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Generation(text='\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side!', generation_info={'finish_reason': 'stop', 'logprobs': None}), Generation(text='\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side.', generation_info={'finish_reason': 'stop', 'logprobs': None})]\n",
      "\n",
      "\n",
      "\n",
      "[Generation(text=\"\\n\\nThere's no need for words\\n\\nWhen we're by your side\\n\\nWe'll know all the way\\n\\nTo the way your driving us\\n\\nOur love will be real\\n\\nNot based on looks or looks only\\n\\nWe'll know what love is\\n\\nOnce we'll be together\\n\\nAnd nothing will help us die\\n\\nOur love is real\\n\\nAnd you don't have to be a part\\n\\nOf our love to be with us\\n\\nAnd it's not about the stars\\n\\nIt's about the way we're driving\\n\\nAnd the way our loves will fly\\n\\n survivin' in each other's sight\\n\\n- birds in the distance-\\n\\nAnd we'll be together\\n\\nWhat a wonderful surprise\\n\\nWe'll be together\\n\\nAnd the wonderful plan!\\n\\nWhy did the ambitious plan\\n\\nFail me every time\\n\\nAnd it's because they were only ambitious\\n\\nTo be in a relationship\\n\\nNow our love is real\\n\\nAnd we're not working for just anyone\\n\\nWe're working for us and our future\\n\\n togethe\", generation_info={'finish_reason': 'stop', 'logprobs': None}), Generation(text=\"\\n\\nWhen I was younger\\nI thought that love\\nI was something like a fairytale\\nI would find my prince\\nAnd we would be together\\nForever\\nI was naïve\\nAnd I was wrong\\nLove is not a fairytale\\nIt's something else entirely\\nSomething that should be cherished\\nAnd loved\\nAnd never taken for granted\\nLove is something that you cannot do without\\nIt's something that you must protect and serve\\nAnd it is this that makes love so wonderful\\n\\nThere is nothing like love\\nIt should beorgans and revolutions\\nIt is the light that leads you through the darkest parts of your heart\\nIt should be constantly Pulled me out of Bed Night\\nAnd keep me awake into the morning\\nIt should be something that is cherished\\nAnd loved\\nAnd never taken for granted\\nLove is something that you cannot do without\\nIt's something that you must protect and serve\", generation_info={'finish_reason': 'stop', 'logprobs': None})]\n"
     ]
    }
   ],
   "source": [
    "len(llm_result.generations)\n",
    "\n",
    "#llm_result.to_dict() # see result details\n",
    "\n",
    "print(llm_result.generations[0])\n",
    "print(\"\\n\\n\")\n",
    "print(llm_result.generations[-1])\n",
    "llm_result.llm_output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jukit_cell_id": "1Kg9Ct1muS"
   },
   "source": [
    "## estimate number of tokens in prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "jukit_cell_id": "qz7lnBufdW"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm.get_num_tokens(\"what is a joke\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jukit_cell_id": "Q9fgYuBKEK"
   },
   "source": [
    "# Key Concepts\n",
    "\n",
    "- Core method exposed by llms is `generate`: takes list of str returns LLMResult\n",
    "- Can also be called directly with single string as input and returns a stirng\n",
    "- Main result is `LLMResult`, input list of strings -> list of LLMResult\n",
    "  Each result is a list of generations (since you can request `n` generations per input str)\n",
    "- `llm_output` contains provider specific ouptput"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jukit_cell_id": "3k0hrhqP7F"
   },
   "source": [
    "## LLM Serialization\n",
    "\n",
    "Wrinting and reading llms to disk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jukit_cell_id": "9V7QPvLLcT"
   },
   "outputs": [],
   "source": [
    "from langchain.llms.loading import load_llm\n",
    "\n",
    "llm.save(\"llm.json\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jukit_cell_id": "rcdMSwUd3W"
   },
   "outputs": [],
   "source": [
    "llm = load_llm(\"llm.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jukit_cell_id": "JfkBS05EUP"
   },
   "source": [
    "## Token Usage Tracking"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jukit_cell_id": "FUnKToXsu6"
   },
   "outputs": [],
   "source": [
    "from langchain.callbacks import get_openai_callback"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jukit_cell_id": "9AoUdMfzg7"
   },
   "outputs": [],
   "source": [
    "llm = OpenAI(model_name=\"text-davinci-002\", n=2, best_of=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "jukit_cell_id": "aiTKa2iDUx",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "42\n"
     ]
    }
   ],
   "source": [
    "with get_openai_callback() as cb:\n",
    "    result = llm(\"tell me a joke\")\n",
    "    print(cb.total_tokens)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jukit_cell_id": "6G5rwEeItx"
   },
   "source": [
    "Anything inside the context manager will get tracked.\n",
    "\n",
    "Example tracking multiple calls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "jukit_cell_id": "uDJtapBCby"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "80\n"
     ]
    }
   ],
   "source": [
    "with get_openai_callback() as cb:\n",
    "    result = llm(\"Tell me a joke\")\n",
    "    result2 = llm(\"Tell me a funny joke\")\n",
    "    print(cb.total_tokens)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jukit_cell_id": "sLuvD8dXnk"
   },
   "source": [
    "If a chain or agent with multiple steps in it is used, it will track all those steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jukit_cell_id": "uUQOyO8XIw"
   },
   "outputs": [],
   "source": [
    "from langchain.agents import load_tools\n",
    "from langchain.agents import initialize_agent\n",
    "from langchain.llms import OpenAI\n",
    "\n",
    "llm = OpenAI(temperature=0)\n",
    "tools = load_tools([\"searx-search\", \"llm-math\"], llm=llm)\n",
    "agent = initialize_agent(tools, llm, agent=\"zero-shot-react-description\", verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "jukit_cell_id": "nv0tixnGfg",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m I need to find the temperature in each city\n",
      "Action: Search\n",
      "Action Input: \"temperature in Paris Berlin and Granada\"\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mHumidity: 40%; Visibility: --Visibility: Not available; Pressure: 1029mb, Falling. Observation Station: Granada/Armilla (Lat: 37.1333 | Long: -3.6333) ...\n",
      "\n",
      "Paris 14 Day Extended Forecast Time/General Weather Time Zone DST Changes Sun & Moon Weather Today Weather Hourly 14 Day Forecast Yesterday/Past Weather Climate (Averages) Currently: 41 °F. Passing clouds. (Weather station: Villacoublay, France). See more current weather Paris Extended Forecast with high and low temperatures Feb 6 – Feb 12 Lo:37\n",
      "\n",
      "Humidity: 40%; Visibility: --Visibility: Not available; Pressure: 1029mb, Falling. Observation Station: Granada/Armilla (Lat: 37.1333 | Long: -3.6333) ...\n",
      "\n",
      "10 Day Weather - Paris, France As of 9:14 am CET Today 50°/ 40° 10% Sat 11 | Day 50° 10% SE 3 mph Mostly cloudy skies this morning will become partly cloudy this afternoon. High near 50F. Winds...\n",
      "\n",
      "Humidity: 87%; Visibility: --Visibility: Not available; Pressure: 1025mb. Observation Station: Granada/Armilla (Lat: 37.1333 | Long: -3.6333) ...\n",
      "\n",
      "Paris' low season sees temperatures in the 40 and 30-degree range. It's only during November that travelers may get a glimpse of 50-degree days. Heavy – yet chic – coats are necessary this ...\n",
      "\n",
      "What is the weather like in Granada compared to Paris? The average temperature in Paris the average temperature is 33.34 degrees in summer, with precipitation ...\n",
      "\n",
      "June Weather in Paris Range of daily high temperatures during June: 62°F to 101°F (17°C to 38°C) Range of daily low temperatures during June: 38°F to 68° F (3°C to 20°C) Record June high temperature (1991-present): 101°F (38°C) in 2022 Record June low temperature (1991-present): 38°F (3°C) in 2006 Average June high temperature: 75°F (24°C)\n",
      "\n",
      "What is the weather like in Paris compared to Granada?\n",
      "\n",
      "Typically, the best time to visit Paris is from May through mid-September. Mid-May through June is especially pleasant, with long days and milder temperatures. Spring, while beautiful, can be quite chilly. Read on for more information about Paris's climate as well as a season-by-season breakdown of the weather. 1.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I need to compare the temperatures in Paris and Granada\n",
      "Action: Calculator\n",
      "Action Input: 33.34 (Paris average temperature) - 37.13 (Granada average temperature)\u001b[0m\n",
      "Observation: \u001b[33;1m\u001b[1;3mAnswer: -3.79\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
      "Final Answer: The average temperature in Granada is 3.79 degrees Celsius lower than the average temperature in Paris.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "with get_openai_callback() as cb:\n",
    "    res = agent.run(\"What is the temperature in Paris  Berlin and Granada ?  \\\n",
    "                    Print every city's temperature in Celcius and Fahrenheit. Think step by step\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "jukit_cell_id": "vpDGqKnagk"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2192\n"
     ]
    }
   ],
   "source": [
    "    print(cb.total_tokens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jukit_cell_id": "xUoMvs6XZ0"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}