langchain/docs/extras/modules/data_connection/document_loaders/integrations/google_drive.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b0ed136e-6983-4893-ae1b-b75753af05f8",
   "metadata": {},
   "source": [
    "# Google Drive\n",
    "\n",
    ">[Google Drive](https://en.wikipedia.org/wiki/Google_Drive) is a file storage and synchronization service developed by Google.\n",
    "\n",
    "This notebook covers how to load documents from `Google Drive`. Currently, only `Google Docs` are supported.\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "1. Create a Google Cloud project or use an existing project\n",
    "1. Enable the [Google Drive API](https://console.cloud.google.com/flows/enableapi?apiid=drive.googleapis.com)\n",
    "1. [Authorize credentials for desktop app](https://developers.google.com/drive/api/quickstart/python#authorize_credentials_for_a_desktop_application)\n",
    "1. `pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib`\n",
    "\n",
    "## 🧑 Instructions for ingesting your Google Docs data\n",
    "By default, the `GoogleDriveLoader` expects the `credentials.json` file to be `~/.credentials/credentials.json`, but this is configurable using the `credentials_path` keyword argument. Same thing with `token.json` - `token_path`. Note that `token.json` will be created automatically the first time you use the loader.\n",
    "\n",
    "`GoogleDriveLoader` can load from a list of Google Docs document ids or a folder id. You can obtain your folder and document id from the URL:\n",
    "* Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
    "* Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6e40071c-3a65-4e26-b497-3e2be0bd86b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "878928a6-a5ae-4f74-b351-64e3b01733fe",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.document_loaders import GoogleDriveLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2216c83f-68e4-4d2f-8ea2-5878fb18bbe7",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "loader = GoogleDriveLoader(\n",
    "    folder_id=\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\",\n",
    "    # Optional: configure whether to recursively fetch files from subfolders. Defaults to False.\n",
    "    recursive=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8f3b6aa0-b45d-4e37-8c50-5bebe70fdb9d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "docs = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2721ba8a",
   "metadata": {},
   "source": [
    "When you pass a `folder_id` by default all files of type document, sheet and pdf are loaded. You can modify this behaviour by passing a `file_types` argument "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ff83b4c",
   "metadata": {},
   "outputs": [],
   "source": [
    "loader = GoogleDriveLoader(\n",
    "    folder_id=\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\",\n",
    "    file_types=[\"document\", \"sheet\"]\n",
    "    recursive=False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6b80931",
   "metadata": {},
   "source": [
    "## Passing in Optional File Loaders\n",
    "\n",
    "When processing files other than Google Docs and Google Sheets, it can be helpful to pass an optional file loader to `GoogleDriveLoader`. If you pass in a file loader, that file loader will be used on documents that do not have a Google Docs or Google Sheets MIME type. Here is an example of how to load an Excel document from Google Drive using a file loader. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "94207e39",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import GoogleDriveLoader\n",
    "from langchain.document_loaders import UnstructuredFileIOLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a15fbee0",
   "metadata": {},
   "outputs": [],
   "source": [
    "file_id=\"1x9WBtFPWMEAdjcJzPScRsjpjQvpSo_kz\"\n",
    "loader = GoogleDriveLoader(\n",
    "    file_ids=[file_id],\n",
    "    file_loader_cls=UnstructuredFileIOLoader,\n",
    "    file_loader_kwargs={\"mode\": \"elements\"}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "98410bda",
   "metadata": {},
   "outputs": [],
   "source": [
    "docs = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e3e72221",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content='\\n  \\n    \\n      Team\\n      Location\\n      Stanley Cups\\n    \\n    \\n      Blues\\n      STL\\n      1\\n    \\n    \\n      Flyers\\n      PHI\\n      2\\n    \\n    \\n      Maple Leafs\\n      TOR\\n      13\\n    \\n  \\n', metadata={'filetype': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'page_number': 1, 'page_name': 'Stanley Cups', 'text_as_html': '<table border=\"1\" class=\"dataframe\">\\n  <tbody>\\n    <tr>\\n      <td>Team</td>\\n      <td>Location</td>\\n      <td>Stanley Cups</td>\\n    </tr>\\n    <tr>\\n      <td>Blues</td>\\n      <td>STL</td>\\n      <td>1</td>\\n    </tr>\\n    <tr>\\n      <td>Flyers</td>\\n      <td>PHI</td>\\n      <td>2</td>\\n    </tr>\\n    <tr>\\n      <td>Maple Leafs</td>\\n      <td>TOR</td>\\n      <td>13</td>\\n    </tr>\\n  </tbody>\\n</table>', 'category': 'Table', 'source': 'https://drive.google.com/file/d/1aA6L2AR3g0CR-PW03HEZZo4NaVlKpaP7/view'})"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "docs[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "238cd06f",
   "metadata": {},
   "source": [
    "You can also process a folder with a mix of files and Google Docs/Sheets using the following pattern:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0e2d093f",
   "metadata": {},
   "outputs": [],
   "source": [
    "folder_id=\"1asMOHY1BqBS84JcRbOag5LOJac74gpmD\"\n",
    "loader = GoogleDriveLoader(\n",
    "    folder_id=folder_id,\n",
    "    file_loader_cls=UnstructuredFileIOLoader,\n",
    "    file_loader_kwargs={\"mode\": \"elements\"}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b35ddcc6",
   "metadata": {},
   "outputs": [],
   "source": [
    "docs = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3cc141e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content='\\n  \\n    \\n      Team\\n      Location\\n      Stanley Cups\\n    \\n    \\n      Blues\\n      STL\\n      1\\n    \\n    \\n      Flyers\\n      PHI\\n      2\\n    \\n    \\n      Maple Leafs\\n      TOR\\n      13\\n    \\n  \\n', metadata={'filetype': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'page_number': 1, 'page_name': 'Stanley Cups', 'text_as_html': '<table border=\"1\" class=\"dataframe\">\\n  <tbody>\\n    <tr>\\n      <td>Team</td>\\n      <td>Location</td>\\n      <td>Stanley Cups</td>\\n    </tr>\\n    <tr>\\n      <td>Blues</td>\\n      <td>STL</td>\\n      <td>1</td>\\n    </tr>\\n    <tr>\\n      <td>Flyers</td>\\n      <td>PHI</td>\\n      <td>2</td>\\n    </tr>\\n    <tr>\\n      <td>Maple Leafs</td>\\n      <td>TOR</td>\\n      <td>13</td>\\n    </tr>\\n  </tbody>\\n</table>', 'category': 'Table', 'source': 'https://drive.google.com/file/d/1aA6L2AR3g0CR-PW03HEZZo4NaVlKpaP7/view'})"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "docs[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e312268a",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}