langchain/docs/examples/prompts/llm_functionality.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "20ac6b98",
   "metadata": {},
   "source": [
    "# LLM Functionality\n",
    "\n",
    "This notebook goes over all the different features of the LLM class in LangChain.\n",
    "\n",
    "We will work with an OpenAI LLM wrapper, although these functionalities should exist for all LLM types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "df924055",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.llms import OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "182b484c",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = OpenAI(model_name=\"text-ada-001\", n=2, best_of=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9695ccfc",
   "metadata": {},
   "source": [
    "**Generate Text:** The most basic functionality an LLM has is just the ability to call it, passing in a string and getting back a string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9d12ac26",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side!'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm(\"Tell me a joke\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7d4d42d",
   "metadata": {},
   "source": [
    "**Generate:** More broadly, you can call it with a list of inputs, getting back a more complete response than just the text. This complete response includes things like multiple top responses, as well as LLM provider specific information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f4dc241a",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm_result = llm.generate([\"Tell me a joke\", \"Tell me a poem\"]*15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "740392f6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "30"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(llm_result.generations)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ab6cdcf1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Generation(text='\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side.'),\n",
       " Generation(text='\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side!')]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_result.generations[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4946a778",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Generation(text=\"\\n\\nA rose by the side of the road\\n\\nIs all I need to find my way\\n\\nTo the place I've been searching for\\n\\nAnd my heart is singing with joy\\n\\nWhen I look at this rose\\n\\nIt reminds me of the love I've found\\n\\nAnd I know that wherever I go\\n\\nI'll always find my rose by the side of the road.\"),\n",
       " Generation(text=\"\\n\\nWhen I was younger\\nI thought that love\\nI was something like a fairytale\\nI would find my prince and they would be my people\\nI was naïve\\nI thought that\\n\\nLove was a something that happened\\nWhen I was younger\\nI was it for my fairytale prince\\nNow I realize\\nThat love is something that waits\\nFor when my prince comes\\nAnd when I am ready to be his wife\\nI'll tell you a poem\\n\\nWhen I was younger\\nI thought that love\\nI was something like a fairytale\\nI would find my prince and they would be my people\\nI was naïve\\nI thought that\\n\\nLove was a something that happened\\nAnd I would be happy\\nWhen my prince came\\nAnd I was ready to be his wife\")]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_result.generations[-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "242e4527",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'token_usage': {'completion_tokens': 3722,\n",
       "  'prompt_tokens': 120,\n",
       "  'total_tokens': 3842}}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Provider specific info\n",
    "llm_result.llm_output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bde8e04f",
   "metadata": {},
   "source": [
    "**Number of Tokens:** You can also estimate how many tokens a piece of text will be in that model. This is useful because models have a context length (and cost more for more tokens), which means you need to be aware of how long the text you are passing in is.\n",
    "\n",
    "Notice that by default the tokens are estimated using a HuggingFace tokenizer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b623c774",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm.get_num_tokens(\"what a joke\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee6fcf8d",
   "metadata": {},
   "source": [
    "### Caching\n",
    "With LangChain, you can also enable caching of LLM calls. Note that currently this only applies for individual LLM calls."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2626ca48",
   "metadata": {},
   "outputs": [],
   "source": [
    "import langchain\n",
    "from langchain.cache import InMemoryCache\n",
    "langchain.llm_cache = InMemoryCache()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "97762272",
   "metadata": {},
   "outputs": [],
   "source": [
    "# To make the caching really obvious, lets use a slower model.\n",
    "llm = OpenAI(model_name=\"text-davinci-002\", n=2, best_of=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e80c65e4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 31.2 ms, sys: 11.8 ms, total: 43.1 ms\n",
      "Wall time: 1.75 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side!'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "# The first time, it is not yet in cache, so it should take longer\n",
    "llm(\"Tell me a joke\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "678408ec",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 51 µs, sys: 1 µs, total: 52 µs\n",
      "Wall time: 67.2 µs\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side!'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "# The second time it is, so it goes faster\n",
    "llm(\"Tell me a joke\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3f0ac8d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can do the same thing with a SQLite cache\n",
    "from langchain.cache import SQLiteCache\n",
    "langchain.llm_cache = SQLiteCache(database_path=\".langchain.db\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0e1dcce3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 26.6 ms, sys: 11.2 ms, total: 37.7 ms\n",
      "Wall time: 1.89 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side.'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "# The first time, it is not yet in cache, so it should take longer\n",
    "llm(\"Tell me a joke\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "efadd750",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2.69 ms, sys: 1.57 ms, total: 4.27 ms\n",
      "Wall time: 2.73 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side.'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "# The second time it is, so it goes faster\n",
    "llm(\"Tell me a joke\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6053408b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# You can use SQLAlchemyCache to cache with any SQL database supported by SQLAlchemy.\n",
    "from langchain.cache import SQLAlchemyCache\n",
    "from sqlalchemy import create_engine\n",
    "\n",
    "engine = create_engine(\"postgresql://postgres:postgres@localhost:5432/postgres\")\n",
    "langchain.llm_cache = SQLAlchemyCache(engine)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12 (main, Jun  1 2022, 06:34:44) \n[Clang 12.0.0 ]"
  },
  "vscode": {
   "interpreter": {
    "hash": "1235b9b19e8e9828b5c1fdb2cd89fe8d3de0fcde5ef5f3db36e4b671adb8660f"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}