langchain/docs/examples/data_augmented_generation/summarize.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d9a0131f",
   "metadata": {},
   "source": [
    "# Summarization\n",
    "\n",
    "This notebook walks through how to use LangChain for summarization over a list of documents. It covers three different chain types: `stuff`, `map_reduce`, and `refine`. For a more in depth explanation of what these chain types are, see [here](../../explanation/combine_docs.md)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b5660bf",
   "metadata": {},
   "source": [
    "### Prepare Data\n",
    "First we prepare the data. For this example we create multiple documents from one long one, but these documents could be fetched in any manner (the point of this notebook to highlight what to do AFTER you fetch the documents)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e9db25f3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain import OpenAI, PromptTemplate, LLMChain\n",
    "from langchain.text_splitter import CharacterTextSplitter\n",
    "from langchain.chains.mapreduce import MapReduceChain\n",
    "\n",
    "llm = OpenAI(temperature=0)\n",
    "\n",
    "\n",
    "text_splitter = CharacterTextSplitter()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "99bbe19b",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('../state_of_the_union.txt') as f:\n",
    "    state_of_the_union = f.read()\n",
    "texts = text_splitter.split_text(state_of_the_union)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "baa6e808",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.docstore.document import Document"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8dff4f43",
   "metadata": {},
   "outputs": [],
   "source": [
    "docs = [Document(page_content=t) for t in texts[:3]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "27989fc4",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains.summarize import load_summarize_chain"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea2d5c99",
   "metadata": {},
   "source": [
    "### The `stuff` Chain\n",
    "\n",
    "This sections shows results of using the `stuff` Chain to do summarization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f01f3196",
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = load_summarize_chain(llm, chain_type=\"stuff\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "da4d9801",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' In his speech, President Biden addressed the ongoing conflict between Russia and Ukraine, and the need for the United States and its allies to stand with Ukraine. He also discussed the American Rescue Plan, the Bipartisan Infrastructure Law, and the Bipartisan Innovation Act, which will help to create jobs, modernize infrastructure, and level the playing field with China. He also emphasized the importance of buying American products to support American jobs.'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.run(docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c868e86",
   "metadata": {},
   "source": [
    "### The `map_reduce` Chain\n",
    "\n",
    "This sections shows results of using the `map_reduce` Chain to do summarization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ef28e1d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = load_summarize_chain(llm, chain_type=\"map_reduce\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f82c5f9f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\" In response to Vladimir Putin's aggression in Ukraine, the US and its allies have taken action to hold him accountable, including economic sanctions, cutting off access to technology, and seizing the assets of Russian oligarchs. They are also providing military, economic, and humanitarian assistance to the Ukrainians, and releasing 60 million barrels of oil from reserves around the world. President Biden has passed several laws to provide economic relief to Americans and create jobs, and is making sure taxpayer dollars support American jobs and businesses.\""
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.run(docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f61350f9",
   "metadata": {},
   "source": [
    "### The `refine` Chain\n",
    "\n",
    "This sections shows results of using the `refine` Chain to do summarization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "3bcbe31e",
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = load_summarize_chain(llm, chain_type=\"refine\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "c8cad866",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"\\nIn this speech, the speaker addresses the American people and their allies, discussing the recent aggression of Russia's Vladimir Putin in Ukraine. The speaker outlines the actions taken by the United States and its allies to hold Putin accountable, including economic sanctions, cutting off access to technology, and seizing the assets of Russian oligarchs. The speaker also announces the closing of American airspace to Russian flights, further isolating Russia and adding an additional squeeze on their economy. The Russian stock market has lost 40% of its value and trading remains suspended. Together with our allies, the United States is providing military, economic, and humanitarian assistance to Ukraine, and has mobilized forces to protect NATO countries. The speaker also announces the release of 60 million barrels of oil from reserves around the world, with the United States releasing 30 million barrels from its own Strategic Petroleum Reserve. The speaker emphasizes that the United States and its allies will defend every inch of NATO territory and that Putin will pay a high price for his aggression. The speaker also acknowledges the hardships faced by the American people due to the pandemic and the American Rescue Plan, which has provided immediate economic relief for tens of millions of Americans, helped put food on their table, keep a roof over their heads, and cut the cost of health insurance. The speaker\""
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.run(docs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0da92750",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}