diff --git a/docs/docs/integrations/document_loaders/google_cloud_sql_pg.ipynb b/docs/docs/integrations/document_loaders/google_cloud_sql_pg.ipynb index 672fec9606..4f1a091a6a 100644 --- a/docs/docs/integrations/document_loaders/google_cloud_sql_pg.ipynb +++ b/docs/docs/integrations/document_loaders/google_cloud_sql_pg.ipynb @@ -209,9 +209,9 @@ "source": [ "### Cloud SQL Engine\n", "\n", - "One of the requirements and arguments to establish PostgreSQL as a document loader is a `PostgreSQLEngine` object. The `PostgreSQLEngine` configures a connection pool to your Cloud SQL for PostgreSQL database, enabling successful connections from your application and following industry best practices.\n", + "One of the requirements and arguments to establish PostgreSQL as a document loader is a `PostgresEngine` object. The `PostgresEngine` configures a connection pool to your Cloud SQL for PostgreSQL database, enabling successful connections from your application and following industry best practices.\n", "\n", - "To create a `PostgreSQLEngine` using `PostgreSQLEngine.from_instance()` you need to provide only 4 things:\n", + "To create a `PostgresEngine` using `PostgresEngine.from_instance()` you need to provide only 4 things:\n", "\n", "1. `project_id` : Project ID of the Google Cloud Project where the Cloud SQL instance is located.\n", "1. `region` : Region where the Cloud SQL instance is located.\n", @@ -220,7 +220,7 @@ "\n", "By default, [IAM database authentication](https://cloud.google.com/sql/docs/postgres/iam-authentication) will be used as the method of database authentication. This library uses the IAM principal belonging to the [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials) sourced from the environment.\n", "\n", - "Optionally, [built-in database authentication](https://cloud.google.com/sql/docs/postgres/users) using a username and password to access the Cloud SQL database can also be used. Just provide the optional `user` and `password` arguments to `PostgreSQLEngine.from_instance()`:\n", + "Optionally, [built-in database authentication](https://cloud.google.com/sql/docs/postgres/users) using a username and password to access the Cloud SQL database can also be used. Just provide the optional `user` and `password` arguments to `PostgresEngine.from_instance()`:\n", "\n", "* `user` : Database user to use for built-in database authentication and login\n", "* `password` : Database password to use for built-in database authentication and login.\n" @@ -239,9 +239,9 @@ "metadata": {}, "outputs": [], "source": [ - "from langchain_google_cloud_sql_pg import PostgreSQLEngine\n", + "from langchain_google_cloud_sql_pg import PostgresEngine\n", "\n", - "engine = await PostgreSQLEngine.afrom_instance(\n", + "engine = await PostgresEngine.afrom_instance(\n", " project_id=PROJECT_ID,\n", " region=REGION,\n", " instance=INSTANCE,\n", @@ -255,7 +255,7 @@ "id": "e1tl0aNx7SWy" }, "source": [ - "### Create PostgreSQLLoader" + "### Create PostgresLoader" ] }, { @@ -266,10 +266,10 @@ }, "outputs": [], "source": [ - "from langchain_google_cloud_sql_pg import PostgreSQLLoader\n", + "from langchain_google_cloud_sql_pg import PostgresLoader\n", "\n", "# Creating a basic PostgreSQL object\n", - "loader = await PostgreSQLLoader.create(engine, table_name=TABLE_NAME)" + "loader = await PostgresLoader.create(engine, table_name=TABLE_NAME)" ] }, { @@ -291,10 +291,10 @@ }, "outputs": [], "source": [ - "from langchain_google_cloud_sql_pg import PostgreSQLLoader\n", + "from langchain_google_cloud_sql_pg import PostgresLoader\n", "\n", - "# Creating a basic PostgreSQLLoader object\n", - "loader = await PostgreSQLLoader.create(engine, table_name=TABLE_NAME)\n", + "# Creating a basic PostgresLoader object\n", + "loader = await PostgresLoader.create(engine, table_name=TABLE_NAME)\n", "\n", "docs = await loader.aload()\n", "print(docs)" @@ -315,7 +315,7 @@ "metadata": {}, "outputs": [], "source": [ - "loader = await PostgreSQLLoader.create(\n", + "loader = await PostgresLoader.create(\n", " engine,\n", " table_name=TABLE_NAME,\n", " content_columns=[\"product_name\"], # Optional\n", @@ -343,7 +343,7 @@ }, "outputs": [], "source": [ - "loader = await PostgreSQLLoader.create(\n", + "loader = await PostgresLoader.create(\n", " engine,\n", " table_name=\"products\",\n", " content_columns=[\"product_name\", \"description\"],\n", diff --git a/docs/docs/integrations/document_loaders/google_el_carro.ipynb b/docs/docs/integrations/document_loaders/google_el_carro.ipynb new file mode 100644 index 0000000000..9a70bdaf81 --- /dev/null +++ b/docs/docs/integrations/document_loaders/google_el_carro.ipynb @@ -0,0 +1,566 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "NKbPFu-GWFDV" + }, + "source": [ + "# Google El Carro Oracle Operator\n", + ">\n", + "Google [El Carro Oracle Operator](https://github.com/GoogleCloudPlatform/elcarro-oracle-operator)\n", + "offers a way to run Oracle databases in Kubernetes as a portable, open source,\n", + "community driven, no vendor lock-in container orchestration system. El Carro\n", + "provides a powerful declarative API for comprehensive and consistent\n", + "configuration and deployment as well as for real-time operations and\n", + "monitoring..\n", + "Extend your database application to build AI-powered experiences leveraging\n", + "Oracle Langchain integrations.\n", + "\n", + "This guide goes over how to use El Carro Langchain integration to\n", + "[save, load and delete langchain documents](https://python.langchain.com/docs/modules/data_connection/document_loaders/)\n", + "with `ElCarroLoader` and `ElCarroDocumentSaver`." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ZqONzXRcWMJg" + }, + "source": [ + "## Before You Begin\n", + "\n", + "Please complete\n", + "the [Getting Started](https://github.com/googleapis/langchain-google-el-carro-python/tree/main/README.md#getting-started)\n", + "section of\n", + "the README to set up your El Carro Oracle database." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "imbbHxKfWPso" + }, + "source": [ + "### 🦜🔗 Library Installation\n", + "\n", + "The integration lives in its own `langchain-google-el-carro` package, so\n", + "we need to install it." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Su5BMP2zWRwM" + }, + "outputs": [], + "source": [ + "%pip install --upgrade --quiet langchain-google-el-carro" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "azV0k45WWSVI" + }, + "source": [ + "## Basic Usage\n", + "\n", + "### Set Up Oracle Database Connection\n", + "\n", + "ElCarroEngine configures a connection pool to your Oracle database,\n", + "enabling successful connections from your application and following industry\n", + "best practices.\n", + "\n", + "You can find the hostname and port values in the status of the El Carro\n", + "Kubernetes instance.\n", + "Use the user password you created for your PDB." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "xG1mYFkEWbkp" + }, + "outputs": [], + "source": [ + "from langchain_google_el_carro import ElCarroEngine\n", + "\n", + "elcarro_engine = ElCarroEngine.from_instance(\n", + " db_host=\"127.0.0.1\",\n", + " db_port=3307,\n", + " db_name=\"PDB1\",\n", + " db_user=\"scott\",\n", + " db_password=\"tiger\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ICW3k_qUWgyv" + }, + "source": [ + "### Initialize a table\n", + "\n", + "Initialize a table of default schema\n", + "via `elcarro_engine.init_document_table()`. Table Columns:\n", + "\n", + "- page_content (type: text)\n", + "- langchain_metadata (type: JSON)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "JmlGLukoWdfS" + }, + "outputs": [], + "source": [ + "TABLE_NAME = \"doc_table\"\n", + "elcarro_engine.init_document_table(\n", + " TABLE_NAME=TABLE_NAME,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "kaI3avj5Wn5O" + }, + "source": [ + "### Save documents\n", + "\n", + "Save langchain documents with `ElCarroDocumentSaver.add_documents()`.\n", + "To initialize `ElCarroDocumentSaver` class you need to provide 2 things:\n", + "\n", + "1. `elcarro_engine` - An instance of a `ElCarroEngine` engine.\n", + "2. `TABLE_NAME` - The name of the table within the Oracle database to store\n", + " langchain documents." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "skaXpthSWpeg" + }, + "outputs": [], + "source": [ + "from langchain_core.documents import Document\n", + "from langchain_google_el_carro import ElCarroDocumentSaver\n", + "\n", + "doc = Document(\n", + " page_content=\"Banana\",\n", + " metadata={\"type\": \"fruit\", \"weight\": 100, \"organic\": 1},\n", + ")\n", + "\n", + "saver = ElCarroDocumentSaver(\n", + " elcarro_engine=elcarro_engine,\n", + " TABLE_NAME=TABLE_NAME,\n", + ")\n", + "saver.add_documents([doc])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "owTYQdNyWs9s" + }, + "source": [ + "### Load documents\n", + "\n", + "Load langchain documents with `ElCarroLoader.load()`\n", + "or `ElCarroLoader.lazy_load()`.\n", + "`lazy_load` returns a generator that only queries database during the iteration.\n", + "To initialize `ElCarroLoader` class you need to provide:\n", + "\n", + "1. `elcarro_engine` - An instance of a `ElCarroEngine` engine.\n", + "2. `TABLE_NAME` - The name of the table within the Oracle database to store\n", + " langchain documents.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "CM6p11amWvYp" + }, + "outputs": [], + "source": [ + "from langchain_google_el_carro import ElCarroLoader\n", + "\n", + "loader = ElCarroLoader(elcarro_engine=elcarro_engine, TABLE_NAME=TABLE_NAME)\n", + "docs = loader.lazy_load()\n", + "for doc in docs:\n", + " print(\"Loaded documents:\", doc)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "OTIDGiZ8WyS3" + }, + "source": [ + "### Load documents via query\n", + "\n", + "Other than loading documents from a table, we can also choose to load documents\n", + "from a view generated from a SQL query. For example:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "p3OB9AwgWzrq" + }, + "outputs": [], + "source": [ + "from langchain_google_el_carro import ElCarroLoader\n", + "\n", + "loader = ElCarroLoader(\n", + " elcarro_engine=elcarro_engine,\n", + " query=f\"SELECT * FROM {TABLE_NAME} WHERE json_value(extra_json_metadata, '$.shape') = 'round'\",\n", + ")\n", + "onedoc = loader.load()\n", + "print(onedoc)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "E6Fl7YNvW3Ep" + }, + "source": [ + "The view generated from SQL query can have different schema than default table.\n", + "In such cases, the behavior of ElCarroLoader is the same as loading from table\n", + "with non-default schema. Please refer to\n", + "section [Load documents with customized document page content & metadata](#load-documents-with-customized-document-page-content--metadata)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "QgsP78MhW4wc" + }, + "source": [ + "### Delete documents\n", + "\n", + "Delete a list of langchain documents from an Oracle table\n", + "with `ElCarroDocumentSaver.delete()`.\n", + "\n", + "For a table with a default schema (page_content, langchain_metadata), the\n", + "deletion criteria is:\n", + "\n", + "A `row` should be deleted if there exists a `document` in the list, such that\n", + "\n", + "- `document.page_content` equals `row[page_content]`\n", + "- `document.metadata` equals `row[langchain_metadata]`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "QSYRHGHXW6IN" + }, + "outputs": [], + "source": [ + "docs = loader.load()\n", + "print(\"Documents before delete:\", docs)\n", + "saver.delete(onedoc)\n", + "print(\"Documents after delete:\", loader.load())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "RerPkBRAW8yR" + }, + "source": [ + "## Advanced Usage\n", + "\n", + "### Load documents with customized document page content & metadata\n", + "\n", + "First we prepare an example table with non-default schema, and populate it with\n", + "some arbitrary data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "u0Fd46aqW-8k" + }, + "outputs": [], + "source": [ + "import sqlalchemy\n", + "\n", + "create_table_query = f\"\"\"CREATE TABLE {TABLE_NAME} (\n", + " fruit_id NUMBER GENERATED BY DEFAULT AS IDENTITY (START WITH 1),\n", + " fruit_name VARCHAR2(100) NOT NULL,\n", + " variety VARCHAR2(50),\n", + " quantity_in_stock NUMBER(10) NOT NULL,\n", + " price_per_unit NUMBER(6,2) NOT NULL,\n", + " organic NUMBER(3) NOT NULL\n", + ")\"\"\"\n", + "\n", + "with elcarro_engine.connect() as conn:\n", + " conn.execute(sqlalchemy.text(create_table_query))\n", + " conn.commit()\n", + " conn.execute(\n", + " sqlalchemy.text(\n", + " f\"\"\"\n", + " INSERT INTO {TABLE_NAME} (fruit_name, variety, quantity_in_stock, price_per_unit, organic)\n", + " VALUES ('Apple', 'Granny Smith', 150, 0.99, 1)\n", + " \"\"\"\n", + " )\n", + " )\n", + " conn.execute(\n", + " sqlalchemy.text(\n", + " f\"\"\"\n", + " INSERT INTO {TABLE_NAME} (fruit_name, variety, quantity_in_stock, price_per_unit, organic)\n", + " VALUES ('Banana', 'Cavendish', 200, 0.59, 0)\n", + " \"\"\"\n", + " )\n", + " )\n", + " conn.execute(\n", + " sqlalchemy.text(\n", + " f\"\"\"\n", + " INSERT INTO {TABLE_NAME} (fruit_name, variety, quantity_in_stock, price_per_unit, organic)\n", + " VALUES ('Orange', 'Navel', 80, 1.29, 1)\n", + " \"\"\"\n", + " )\n", + " )\n", + " conn.commit()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hGPYiTu7XBh3" + }, + "source": [ + "If we still load langchain documents with default parameters of `ElCarroLoader`\n", + "from this example table, the `page_content` of loaded documents will be the\n", + "first column of the table, and `metadata` will be consisting of key-value pairs\n", + "of all the other columns." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "eQbRapM_XC1S" + }, + "outputs": [], + "source": [ + "loader = ElCarroLoader(\n", + " elcarro_engine=elcarro_engine,\n", + " TABLE_NAME=TABLE_NAME,\n", + ")\n", + "loader.load()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tOH6i2jWXFqz" + }, + "source": [ + "We can specify the content and metadata we want to load by setting\n", + "the `content_columns` and `metadata_columns` when initializing\n", + "the `ElCarroLoader`.\n", + "\n", + "1. `content_columns`: The columns to write into the `page_content` of the\n", + " document.\n", + "2. `metadata_columns`: The columns to write into the `metadata` of the document.\n", + "\n", + "For example here, the values of columns in `content_columns` will be joined\n", + "together into a space-separated string, as `page_content` of loaded documents,\n", + "and `metadata` of loaded documents will only contain key-value pairs of columns\n", + "specified in `metadata_columns`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "9gCFWqgGXHD3" + }, + "outputs": [], + "source": [ + "loader = ElCarroLoader(\n", + " elcarro_engine=elcarro_engine,\n", + " TABLE_NAME=TABLE_NAME,\n", + " content_columns=[\n", + " \"variety\",\n", + " \"quantity_in_stock\",\n", + " \"price_per_unit\",\n", + " \"organic\",\n", + " ],\n", + " metadata_columns=[\"fruit_id\", \"fruit_name\"],\n", + ")\n", + "loaded_docs = loader.load()\n", + "print(f\"Loaded Documents: [{loaded_docs}]\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4KlSfvPJXKgM" + }, + "source": [ + "### Save document with customized page content & metadata\n", + "\n", + "In order to save langchain document into table with customized metadata fields.\n", + "We need first create such a table via `ElCarroEngine.init_document_table()`, and\n", + "specify the list of `metadata_columns` we want it to have. In this example, the\n", + "created table will have table columns:\n", + "\n", + "- description (type: text): for storing fruit description.\n", + "- fruit_name (type text): for storing fruit name.\n", + "- organic (type tinyint(1)): to tell if the fruit is organic.\n", + "- other_metadata (type: JSON): for storing other metadata information of the\n", + " fruit.\n", + "\n", + "We can use the following parameters\n", + "with `elcarro_engine.init_document_table()` to create the table:\n", + "\n", + "1. `TABLE_NAME`: The name of the table within the Oracle database to store\n", + " langchain documents.\n", + "2. `metadata_columns`: A list of `sqlalchemy.Column` indicating the list of\n", + " metadata columns we need.\n", + "3. `content_column`: column name to store `page_content` of langchain\n", + " document. Default: `\"page_content\", \"VARCHAR2(4000)\"`\n", + "4. `metadata_json_column`: column name to store extra\n", + " JSON `metadata` of langchain document.\n", + " Default: `\"langchain_metadata\", \"VARCHAR2(4000)\"`.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "1Wqs05gpXMW9" + }, + "outputs": [], + "source": [ + "elcarro_engine.init_document_table(\n", + " TABLE_NAME=TABLE_NAME,\n", + " metadata_columns=[\n", + " sqlalchemy.Column(\"type\", sqlalchemy.dialects.oracle.VARCHAR2(200)),\n", + " sqlalchemy.Column(\"weight\", sqlalchemy.INT),\n", + " ],\n", + " content_column=\"content\",\n", + " metadata_json_column=\"extra_json_metadata\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "bVEWHYU-XPFt" + }, + "source": [ + "Save documents with `ElCarroDocumentSaver.add_documents()`. As you\n", + "can see in this example,\n", + "\n", + "- `document.page_content` will be saved into `page_content` column.\n", + "- `document.metadata.type` will be saved into `type` column.\n", + "- `document.metadata.weight` will be saved into `weight` column.\n", + "- `document.metadata.organic` will be saved into `extra_json_metadata` column in\n", + " JSON format.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Iy4wRZLPXQn5" + }, + "outputs": [], + "source": [ + "doc = Document(\n", + " page_content=\"Banana\",\n", + " metadata={\"type\": \"fruit\", \"weight\": 100, \"organic\": 1},\n", + ")\n", + "\n", + "print(f\"Original Document: [{doc}]\")\n", + "\n", + "saver = ElCarroDocumentSaver(\n", + " elcarro_engine=elcarro_engine,\n", + " TABLE_NAME=TABLE_NAME,\n", + " content_column=\"content\",\n", + " metadata_json_column=\"extra_json_metadata\",\n", + ")\n", + "saver.add_documents([doc])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "x0vkL7PKXUmU" + }, + "source": [ + "### Delete documents with customized page content & metadata\n", + "\n", + "We can also delete documents from table with customized metadata columns\n", + "via `ElCarroDocumentSaver.delete()`. The deletion criteria is:\n", + "\n", + "A `row` should be deleted if there exists a `document` in the list, such that\n", + "\n", + "- `document.page_content` equals `row[page_content]`\n", + "- For every metadata field `k` in `document.metadata`\n", + " - `document.metadata[k]` equals `row[k]` or `document.metadata[k]`\n", + " equals `row[langchain_metadata][k]`\n", + "- There is no extra metadata field present in `row` but not\n", + " in `document.metadata`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "OcJPeCuKXWSa" + }, + "outputs": [], + "source": [ + "saver.delete(loaded_docs)\n", + "print(f\"Documents left: {len(loader.load())}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "S4SxUoY-XsPN" + }, + "source": [ + "## More examples\n", + "\n", + "Please look\n", + "at [demo_doc_loader_basic.py](https://github.com/googleapis/langchain-google-el-carro-python/tree/main/samples/demo_doc_loader_basic.py)\n", + "and [demo_doc_loader_advanced.py](https://github.com/googleapis/langchain-google-el-carro-python/tree/main/samples/demo_doc_loader_advanced.py)\n", + "for\n", + "complete code examples.\n" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/docs/docs/integrations/memory/google_cloud_sql_pg.ipynb b/docs/docs/integrations/memory/google_cloud_sql_pg.ipynb index 57676a4b46..bec86bb4f0 100644 --- a/docs/docs/integrations/memory/google_cloud_sql_pg.ipynb +++ b/docs/docs/integrations/memory/google_cloud_sql_pg.ipynb @@ -1,554 +1,554 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "f22eab3f84cbeb37", - "metadata": { - "id": "f22eab3f84cbeb37" - }, - "source": [ - "# Google Cloud SQL for PostgreSQL\n", - "\n", - "> [Cloud SQL](https://cloud.google.com/sql) is a fully managed relational database service that offers high performance, seamless integration, and impressive scalability. It offers MySQL, PostgreSQL, and SQL Server database engines. Extend your database application to build AI-powered experiences leveraging Cloud SQL's Langchain integrations.\n", - "\n", - "This notebook goes over how to use `Cloud SQL for PostgreSQL` to store chat message history with the `PostgreSQLChatMessageHistory` class." - ] - }, - { - "cell_type": "markdown", - "id": "da400c79-a360-43e2-be60-401fd02b2819", - "metadata": { - "id": "da400c79-a360-43e2-be60-401fd02b2819" - }, - "source": [ - "## Before You Begin\n", - "\n", - "To run this notebook, you will need to do the following:\n", - "\n", - " * [Create a Google Cloud Project](https://developers.google.com/workspace/guides/create-project)\n", - " * [Create a Cloud SQL for PostgreSQL instance](https://cloud.google.com/sql/docs/postgres/create-instance)\n", - " * [Create a Cloud SQL database](https://cloud.google.com/sql/docs/mysql/create-manage-databases)\n", - " * [Add an IAM database user to the database](https://cloud.google.com/sql/docs/postgres/add-manage-iam-users#creating-a-database-user) (Optional)" - ] - }, - { - "cell_type": "markdown", - "id": "Mm7-fG_LltD7", - "metadata": { - "id": "Mm7-fG_LltD7" - }, - "source": [ - "### 🦜🔗 Library Installation\n", - "The integration lives in its own `langchain-google-cloud-sql-pg` package, so we need to install it." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "1VELXvcj8AId", - "metadata": { - "id": "1VELXvcj8AId" - }, - "outputs": [], - "source": [ - "%pip install --upgrade --quiet langchain-google-cloud-sql-pg langchain-google-vertexai" - ] - }, - { - "cell_type": "markdown", - "id": "98TVoM3MNDHu", - "metadata": { - "id": "98TVoM3MNDHu" - }, - "source": [ - "**Colab only:** Uncomment the following cell to restart the kernel or use the button to restart the kernel. For Vertex AI Workbench you can restart the terminal using the button on top." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "v6jBDnYnNM08", - "metadata": { - "id": "v6jBDnYnNM08" - }, - "outputs": [], - "source": [ - "# # Automatically restart kernel after installs so that your environment can access the new packages\n", - "# import IPython\n", - "\n", - "# app = IPython.Application.instance()\n", - "# app.kernel.do_shutdown(True)" - ] - }, - { - "cell_type": "markdown", - "id": "yygMe6rPWxHS", - "metadata": { - "id": "yygMe6rPWxHS" - }, - "source": [ - "### 🔐 Authentication\n", - "Authenticate to Google Cloud as the IAM user logged into this notebook in order to access your Google Cloud Project.\n", - "\n", - "* If you are using Colab to run this notebook, use the cell below and continue.\n", - "* If you are using Vertex AI Workbench, check out the setup instructions [here](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/setup-env)." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "PTXN1_DSXj2b", - "metadata": { - "id": "PTXN1_DSXj2b" - }, - "outputs": [], - "source": [ - "from google.colab import auth\n", - "\n", - "auth.authenticate_user()" - ] - }, - { - "cell_type": "markdown", - "id": "NEvB9BoLEulY", - "metadata": { - "id": "NEvB9BoLEulY" - }, - "source": [ - "### ☁ Set Your Google Cloud Project\n", - "Set your Google Cloud project so that you can leverage Google Cloud resources within this notebook.\n", - "\n", - "If you don't know your project ID, try the following:\n", - "\n", - "* Run `gcloud config list`.\n", - "* Run `gcloud projects list`.\n", - "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "gfkS3yVRE4_W", - "metadata": { - "cellView": "form", - "id": "gfkS3yVRE4_W" - }, - "outputs": [], - "source": [ - "# @markdown Please fill in the value below with your Google Cloud project ID and then run the cell.\n", - "\n", - "PROJECT_ID = \"my-project-id\" # @param {type:\"string\"}\n", - "\n", - "# Set the project id\n", - "!gcloud config set project {PROJECT_ID}" - ] - }, - { - "cell_type": "markdown", - "id": "rEWWNoNnKOgq", - "metadata": { - "id": "rEWWNoNnKOgq" - }, - "source": [ - "### 💡 API Enablement\n", - "The `langchain-google-cloud-sql-pg` package requires that you [enable the Cloud SQL Admin API](https://console.cloud.google.com/flows/enableapi?apiid=sqladmin.googleapis.com) in your Google Cloud Project." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "5utKIdq7KYi5", - "metadata": { - "id": "5utKIdq7KYi5" - }, - "outputs": [], - "source": [ - "# enable Cloud SQL Admin API\n", - "!gcloud services enable sqladmin.googleapis.com" - ] - }, - { - "cell_type": "markdown", - "id": "f8f2830ee9ca1e01", - "metadata": { - "id": "f8f2830ee9ca1e01" - }, - "source": [ - "## Basic Usage" - ] - }, - { - "cell_type": "markdown", - "id": "OMvzMWRrR6n7", - "metadata": { - "id": "OMvzMWRrR6n7" - }, - "source": [ - "### Set Cloud SQL database values\n", - "Find your database values, in the [Cloud SQL Instances page](https://console.cloud.google.com/sql?_ga=2.223735448.2062268965.1707700487-2088871159.1707257687)." - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "irl7eMFnSPZr", - "metadata": { - "id": "irl7eMFnSPZr" - }, - "outputs": [], - "source": [ - "# @title Set Your Values Here { display-mode: \"form\" }\n", - "REGION = \"us-central1\" # @param {type: \"string\"}\n", - "INSTANCE = \"my-postgresql-instance\" # @param {type: \"string\"}\n", - "DATABASE = \"my-database\" # @param {type: \"string\"}\n", - "TABLE_NAME = \"message_store\" # @param {type: \"string\"}" - ] - }, - { - "cell_type": "markdown", - "id": "QuQigs4UoFQ2", - "metadata": { - "id": "QuQigs4UoFQ2" - }, - "source": [ - "### PostgreSQLEngine Connection Pool\n", - "\n", - "One of the requirements and arguments to establish Cloud SQL as a ChatMessageHistory memory store is a `PostgreSQLEngine` object. The `PostgreSQLEngine` configures a connection pool to your Cloud SQL database, enabling successful connections from your application and following industry best practices.\n", - "\n", - "To create a `PostgreSQLEngine` using `PostgreSQLEngine.from_instance()` you need to provide only 4 things:\n", - "\n", - "1. `project_id` : Project ID of the Google Cloud Project where the Cloud SQL instance is located.\n", - "1. `region` : Region where the Cloud SQL instance is located.\n", - "1. `instance` : The name of the Cloud SQL instance.\n", - "1. `database` : The name of the database to connect to on the Cloud SQL instance.\n", - "\n", - "By default, [IAM database authentication](https://cloud.google.com/sql/docs/postgres/iam-authentication#iam-db-auth) will be used as the method of database authentication. This library uses the IAM principal belonging to the [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials) sourced from the envionment.\n", - "\n", - "For more informatin on IAM database authentication please see:\n", - "\n", - "* [Configure an instance for IAM database authentication](https://cloud.google.com/sql/docs/postgres/create-edit-iam-instances)\n", - "* [Manage users with IAM database authentication](https://cloud.google.com/sql/docs/postgres/add-manage-iam-users)\n", - "\n", - "Optionally, [built-in database authentication](https://cloud.google.com/sql/docs/postgres/built-in-authentication) using a username and password to access the Cloud SQL database can also be used. Just provide the optional `user` and `password` arguments to `PostgreSQLEngine.from_instance()`:\n", - "\n", - "* `user` : Database user to use for built-in database authentication and login\n", - "* `password` : Database password to use for built-in database authentication and login.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "4576e914a866fb40", - "metadata": { - "ExecuteTime": { - "end_time": "2023-08-28T10:04:38.077748Z", - "start_time": "2023-08-28T10:04:36.105894Z" + "cells": [ + { + "cell_type": "markdown", + "id": "f22eab3f84cbeb37", + "metadata": { + "id": "f22eab3f84cbeb37" + }, + "source": [ + "# Google Cloud SQL for PostgreSQL\n", + "\n", + "> [Cloud SQL](https://cloud.google.com/sql) is a fully managed relational database service that offers high performance, seamless integration, and impressive scalability. It offers MySQL, PostgreSQL, and SQL Server database engines. Extend your database application to build AI-powered experiences leveraging Cloud SQL's Langchain integrations.\n", + "\n", + "This notebook goes over how to use `Cloud SQL for PostgreSQL` to store chat message history with the `PostgresChatMessageHistory` class." + ] + }, + { + "cell_type": "markdown", + "id": "da400c79-a360-43e2-be60-401fd02b2819", + "metadata": { + "id": "da400c79-a360-43e2-be60-401fd02b2819" + }, + "source": [ + "## Before You Begin\n", + "\n", + "To run this notebook, you will need to do the following:\n", + "\n", + " * [Create a Google Cloud Project](https://developers.google.com/workspace/guides/create-project)\n", + " * [Create a Cloud SQL for PostgreSQL instance](https://cloud.google.com/sql/docs/postgres/create-instance)\n", + " * [Create a Cloud SQL database](https://cloud.google.com/sql/docs/mysql/create-manage-databases)\n", + " * [Add an IAM database user to the database](https://cloud.google.com/sql/docs/postgres/add-manage-iam-users#creating-a-database-user) (Optional)" + ] + }, + { + "cell_type": "markdown", + "id": "Mm7-fG_LltD7", + "metadata": { + "id": "Mm7-fG_LltD7" + }, + "source": [ + "### 🦜🔗 Library Installation\n", + "The integration lives in its own `langchain-google-cloud-sql-pg` package, so we need to install it." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1VELXvcj8AId", + "metadata": { + "id": "1VELXvcj8AId" + }, + "outputs": [], + "source": [ + "%pip install --upgrade --quiet langchain-google-cloud-sql-pg langchain-google-vertexai" + ] + }, + { + "cell_type": "markdown", + "id": "98TVoM3MNDHu", + "metadata": { + "id": "98TVoM3MNDHu" + }, + "source": [ + "**Colab only:** Uncomment the following cell to restart the kernel or use the button to restart the kernel. For Vertex AI Workbench you can restart the terminal using the button on top." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "v6jBDnYnNM08", + "metadata": { + "id": "v6jBDnYnNM08" + }, + "outputs": [], + "source": [ + "# # Automatically restart kernel after installs so that your environment can access the new packages\n", + "# import IPython\n", + "\n", + "# app = IPython.Application.instance()\n", + "# app.kernel.do_shutdown(True)" + ] + }, + { + "cell_type": "markdown", + "id": "yygMe6rPWxHS", + "metadata": { + "id": "yygMe6rPWxHS" + }, + "source": [ + "### 🔐 Authentication\n", + "Authenticate to Google Cloud as the IAM user logged into this notebook in order to access your Google Cloud Project.\n", + "\n", + "* If you are using Colab to run this notebook, use the cell below and continue.\n", + "* If you are using Vertex AI Workbench, check out the setup instructions [here](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/setup-env)." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "PTXN1_DSXj2b", + "metadata": { + "id": "PTXN1_DSXj2b" + }, + "outputs": [], + "source": [ + "from google.colab import auth\n", + "\n", + "auth.authenticate_user()" + ] + }, + { + "cell_type": "markdown", + "id": "NEvB9BoLEulY", + "metadata": { + "id": "NEvB9BoLEulY" + }, + "source": [ + "### ☁ Set Your Google Cloud Project\n", + "Set your Google Cloud project so that you can leverage Google Cloud resources within this notebook.\n", + "\n", + "If you don't know your project ID, try the following:\n", + "\n", + "* Run `gcloud config list`.\n", + "* Run `gcloud projects list`.\n", + "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "gfkS3yVRE4_W", + "metadata": { + "cellView": "form", + "id": "gfkS3yVRE4_W" + }, + "outputs": [], + "source": [ + "# @markdown Please fill in the value below with your Google Cloud project ID and then run the cell.\n", + "\n", + "PROJECT_ID = \"my-project-id\" # @param {type:\"string\"}\n", + "\n", + "# Set the project id\n", + "!gcloud config set project {PROJECT_ID}" + ] + }, + { + "cell_type": "markdown", + "id": "rEWWNoNnKOgq", + "metadata": { + "id": "rEWWNoNnKOgq" + }, + "source": [ + "### 💡 API Enablement\n", + "The `langchain-google-cloud-sql-pg` package requires that you [enable the Cloud SQL Admin API](https://console.cloud.google.com/flows/enableapi?apiid=sqladmin.googleapis.com) in your Google Cloud Project." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "5utKIdq7KYi5", + "metadata": { + "id": "5utKIdq7KYi5" + }, + "outputs": [], + "source": [ + "# enable Cloud SQL Admin API\n", + "!gcloud services enable sqladmin.googleapis.com" + ] + }, + { + "cell_type": "markdown", + "id": "f8f2830ee9ca1e01", + "metadata": { + "id": "f8f2830ee9ca1e01" + }, + "source": [ + "## Basic Usage" + ] + }, + { + "cell_type": "markdown", + "id": "OMvzMWRrR6n7", + "metadata": { + "id": "OMvzMWRrR6n7" + }, + "source": [ + "### Set Cloud SQL database values\n", + "Find your database values, in the [Cloud SQL Instances page](https://console.cloud.google.com/sql?_ga=2.223735448.2062268965.1707700487-2088871159.1707257687)." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "irl7eMFnSPZr", + "metadata": { + "id": "irl7eMFnSPZr" + }, + "outputs": [], + "source": [ + "# @title Set Your Values Here { display-mode: \"form\" }\n", + "REGION = \"us-central1\" # @param {type: \"string\"}\n", + "INSTANCE = \"my-postgresql-instance\" # @param {type: \"string\"}\n", + "DATABASE = \"my-database\" # @param {type: \"string\"}\n", + "TABLE_NAME = \"message_store\" # @param {type: \"string\"}" + ] + }, + { + "cell_type": "markdown", + "id": "QuQigs4UoFQ2", + "metadata": { + "id": "QuQigs4UoFQ2" + }, + "source": [ + "### PostgresEngine Connection Pool\n", + "\n", + "One of the requirements and arguments to establish Cloud SQL as a ChatMessageHistory memory store is a `PostgresEngine` object. The `PostgresEngine` configures a connection pool to your Cloud SQL database, enabling successful connections from your application and following industry best practices.\n", + "\n", + "To create a `PostgresEngine` using `PostgresEngine.from_instance()` you need to provide only 4 things:\n", + "\n", + "1. `project_id` : Project ID of the Google Cloud Project where the Cloud SQL instance is located.\n", + "1. `region` : Region where the Cloud SQL instance is located.\n", + "1. `instance` : The name of the Cloud SQL instance.\n", + "1. `database` : The name of the database to connect to on the Cloud SQL instance.\n", + "\n", + "By default, [IAM database authentication](https://cloud.google.com/sql/docs/postgres/iam-authentication#iam-db-auth) will be used as the method of database authentication. This library uses the IAM principal belonging to the [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials) sourced from the envionment.\n", + "\n", + "For more informatin on IAM database authentication please see:\n", + "\n", + "* [Configure an instance for IAM database authentication](https://cloud.google.com/sql/docs/postgres/create-edit-iam-instances)\n", + "* [Manage users with IAM database authentication](https://cloud.google.com/sql/docs/postgres/add-manage-iam-users)\n", + "\n", + "Optionally, [built-in database authentication](https://cloud.google.com/sql/docs/postgres/built-in-authentication) using a username and password to access the Cloud SQL database can also be used. Just provide the optional `user` and `password` arguments to `PostgresEngine.from_instance()`:\n", + "\n", + "* `user` : Database user to use for built-in database authentication and login\n", + "* `password` : Database password to use for built-in database authentication and login.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "4576e914a866fb40", + "metadata": { + "ExecuteTime": { + "end_time": "2023-08-28T10:04:38.077748Z", + "start_time": "2023-08-28T10:04:36.105894Z" + }, + "id": "4576e914a866fb40", + "jupyter": { + "outputs_hidden": false + } + }, + "outputs": [], + "source": [ + "from langchain_google_cloud_sql_pg import PostgresEngine\n", + "\n", + "engine = PostgresEngine.from_instance(\n", + " project_id=PROJECT_ID, region=REGION, instance=INSTANCE, database=DATABASE\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "qPV8WfWr7O54", + "metadata": { + "id": "qPV8WfWr7O54" + }, + "source": [ + "### Initialize a table\n", + "The `PostgresChatMessageHistory` class requires a database table with a specific schema in order to store the chat message history.\n", + "\n", + "The `PostgresEngine` engine has a helper method `init_chat_history_table()` that can be used to create a table with the proper schema for you." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "TEu4VHArRttE", + "metadata": { + "id": "TEu4VHArRttE" + }, + "outputs": [], + "source": [ + "engine.init_chat_history_table(table_name=TABLE_NAME)" + ] + }, + { + "cell_type": "markdown", + "id": "zSYQTYf3UfOi", + "metadata": { + "id": "zSYQTYf3UfOi" + }, + "source": [ + "### PostgresChatMessageHistory\n", + "\n", + "To initialize the `PostgresChatMessageHistory` class you need to provide only 3 things:\n", + "\n", + "1. `engine` - An instance of a `PostgresEngine` engine.\n", + "1. `session_id` - A unique identifier string that specifies an id for the session.\n", + "1. `table_name` : The name of the table within the Cloud SQL database to store the chat message history." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "Kq7RLtfOq0wi", + "metadata": { + "id": "Kq7RLtfOq0wi" + }, + "outputs": [], + "source": [ + "from langchain_google_cloud_sql_pg import PostgresChatMessageHistory\n", + "\n", + "history = PostgresChatMessageHistory.create_sync(\n", + " engine, session_id=\"test_session\", table_name=TABLE_NAME\n", + ")\n", + "history.add_user_message(\"hi!\")\n", + "history.add_ai_message(\"whats up?\")" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "b476688cbb32ba90", + "metadata": { + "ExecuteTime": { + "end_time": "2023-08-28T10:04:38.929396Z", + "start_time": "2023-08-28T10:04:38.915727Z" + }, + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "b476688cbb32ba90", + "jupyter": { + "outputs_hidden": false + }, + "outputId": "a19e5cd8-4225-476a-d28d-e870c6b838bb" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[HumanMessage(content='hi!'), AIMessage(content='whats up?')]" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "history.messages" + ] + }, + { + "cell_type": "markdown", + "id": "ss6CbqcTTedr", + "metadata": { + "id": "ss6CbqcTTedr" + }, + "source": [ + "#### Cleaning up\n", + "When the history of a specific session is obsolete and can be deleted, it can be done the following way.\n", + "\n", + "**Note:** Once deleted, the data is no longer stored in Cloud SQL and is gone forever." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "3khxzFxYO7x6", + "metadata": { + "id": "3khxzFxYO7x6" + }, + "outputs": [], + "source": [ + "history.clear()" + ] + }, + { + "cell_type": "markdown", + "id": "2e5337719d5614fd", + "metadata": { + "id": "2e5337719d5614fd" + }, + "source": [ + "## 🔗 Chaining\n", + "\n", + "We can easily combine this message history class with [LCEL Runnables](/docs/expression_language/how_to/message_history)\n", + "\n", + "To do this we will use one of [Google's Vertex AI chat models](https://python.langchain.com/docs/integrations/chat/google_vertex_ai_palm) which requires that you [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com) in your Google Cloud Project.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "hYtHM3-TOMCe", + "metadata": { + "id": "hYtHM3-TOMCe" + }, + "outputs": [], + "source": [ + "# enable Vertex AI API\n", + "!gcloud services enable aiplatform.googleapis.com" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "6558418b-0ece-4d01-9661-56d562d78f7a", + "metadata": { + "id": "6558418b-0ece-4d01-9661-56d562d78f7a" + }, + "outputs": [], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n", + "from langchain_core.runnables.history import RunnableWithMessageHistory\n", + "from langchain_google_vertexai import ChatVertexAI" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "82149122-61d3-490d-9bdb-bb98606e8ba1", + "metadata": { + "id": "82149122-61d3-490d-9bdb-bb98606e8ba1" + }, + "outputs": [], + "source": [ + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\"system\", \"You are a helpful assistant.\"),\n", + " MessagesPlaceholder(variable_name=\"history\"),\n", + " (\"human\", \"{question}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | ChatVertexAI(project=PROJECT_ID)" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "2df90853-b67c-490f-b7f8-b69d69270b9c", + "metadata": { + "id": "2df90853-b67c-490f-b7f8-b69d69270b9c" + }, + "outputs": [], + "source": [ + "chain_with_history = RunnableWithMessageHistory(\n", + " chain,\n", + " lambda session_id: PostgresChatMessageHistory.create_sync(\n", + " engine,\n", + " session_id=session_id,\n", + " table_name=TABLE_NAME,\n", + " ),\n", + " input_messages_key=\"question\",\n", + " history_messages_key=\"history\",\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "0ce596b8-3b78-48fd-9f92-46dccbbfd58b", + "metadata": { + "id": "0ce596b8-3b78-48fd-9f92-46dccbbfd58b" + }, + "outputs": [], + "source": [ + "# This is where we configure the session id\n", + "config = {\"configurable\": {\"session_id\": \"test_session\"}}" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "38e1423b-ba86-4496-9151-25932fab1a8b", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "38e1423b-ba86-4496-9151-25932fab1a8b", + "outputId": "d5c93570-4b0b-4fe8-d19c-4b361fe74291" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=' Hello Bob, how can I help you today?')" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chain_with_history.invoke({\"question\": \"Hi! I'm bob\"}, config=config)" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "2ee4ee62-a216-4fb1-bf33-57476a84cf16", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "2ee4ee62-a216-4fb1-bf33-57476a84cf16", + "outputId": "288fe388-3f60-41b8-8edb-37cfbec18981" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=' Your name is Bob.')" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chain_with_history.invoke({\"question\": \"Whats my name\"}, config=config)" + ] + } + ], + "metadata": { + "colab": { + "provenance": [], + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" + } }, - "id": "4576e914a866fb40", - "jupyter": { - "outputs_hidden": false - } - }, - "outputs": [], - "source": [ - "from langchain_google_cloud_sql_pg import PostgreSQLEngine\n", - "\n", - "engine = PostgreSQLEngine.from_instance(\n", - " project_id=PROJECT_ID, region=REGION, instance=INSTANCE, database=DATABASE\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "qPV8WfWr7O54", - "metadata": { - "id": "qPV8WfWr7O54" - }, - "source": [ - "### Initialize a table\n", - "The `PostgreSQLChatMessageHistory` class requires a database table with a specific schema in order to store the chat message history.\n", - "\n", - "The `PostgreSQLEngine` engine has a helper method `init_chat_history_table()` that can be used to create a table with the proper schema for you." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "TEu4VHArRttE", - "metadata": { - "id": "TEu4VHArRttE" - }, - "outputs": [], - "source": [ - "engine.init_chat_history_table(table_name=TABLE_NAME)" - ] - }, - { - "cell_type": "markdown", - "id": "zSYQTYf3UfOi", - "metadata": { - "id": "zSYQTYf3UfOi" - }, - "source": [ - "### PostgreSQLChatMessageHistory\n", - "\n", - "To initialize the `PostgreSQLChatMessageHistory` class you need to provide only 3 things:\n", - "\n", - "1. `engine` - An instance of a `PostgreSQLEngine` engine.\n", - "1. `session_id` - A unique identifier string that specifies an id for the session.\n", - "1. `table_name` : The name of the table within the Cloud SQL database to store the chat message history." - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "Kq7RLtfOq0wi", - "metadata": { - "id": "Kq7RLtfOq0wi" - }, - "outputs": [], - "source": [ - "from langchain_google_cloud_sql_pg import PostgreSQLChatMessageHistory\n", - "\n", - "history = PostgreSQLChatMessageHistory.create_sync(\n", - " engine, session_id=\"test_session\", table_name=TABLE_NAME\n", - ")\n", - "history.add_user_message(\"hi!\")\n", - "history.add_ai_message(\"whats up?\")" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "b476688cbb32ba90", - "metadata": { - "ExecuteTime": { - "end_time": "2023-08-28T10:04:38.929396Z", - "start_time": "2023-08-28T10:04:38.915727Z" - }, - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "b476688cbb32ba90", - "jupyter": { - "outputs_hidden": false - }, - "outputId": "a19e5cd8-4225-476a-d28d-e870c6b838bb" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "[HumanMessage(content='hi!'), AIMessage(content='whats up?')]" - ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "history.messages" - ] - }, - { - "cell_type": "markdown", - "id": "ss6CbqcTTedr", - "metadata": { - "id": "ss6CbqcTTedr" - }, - "source": [ - "#### Cleaning up\n", - "When the history of a specific session is obsolete and can be deleted, it can be done the following way.\n", - "\n", - "**Note:** Once deleted, the data is no longer stored in Cloud SQL and is gone forever." - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "id": "3khxzFxYO7x6", - "metadata": { - "id": "3khxzFxYO7x6" - }, - "outputs": [], - "source": [ - "history.clear()" - ] - }, - { - "cell_type": "markdown", - "id": "2e5337719d5614fd", - "metadata": { - "id": "2e5337719d5614fd" - }, - "source": [ - "## 🔗 Chaining\n", - "\n", - "We can easily combine this message history class with [LCEL Runnables](/docs/expression_language/how_to/message_history)\n", - "\n", - "To do this we will use one of [Google's Vertex AI chat models](https://python.langchain.com/docs/integrations/chat/google_vertex_ai_palm) which requires that you [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com) in your Google Cloud Project.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "id": "hYtHM3-TOMCe", - "metadata": { - "id": "hYtHM3-TOMCe" - }, - "outputs": [], - "source": [ - "# enable Vertex AI API\n", - "!gcloud services enable aiplatform.googleapis.com" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "id": "6558418b-0ece-4d01-9661-56d562d78f7a", - "metadata": { - "id": "6558418b-0ece-4d01-9661-56d562d78f7a" - }, - "outputs": [], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n", - "from langchain_core.runnables.history import RunnableWithMessageHistory\n", - "from langchain_google_vertexai import ChatVertexAI" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "id": "82149122-61d3-490d-9bdb-bb98606e8ba1", - "metadata": { - "id": "82149122-61d3-490d-9bdb-bb98606e8ba1" - }, - "outputs": [], - "source": [ - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\"system\", \"You are a helpful assistant.\"),\n", - " MessagesPlaceholder(variable_name=\"history\"),\n", - " (\"human\", \"{question}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | ChatVertexAI(project=PROJECT_ID)" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "id": "2df90853-b67c-490f-b7f8-b69d69270b9c", - "metadata": { - "id": "2df90853-b67c-490f-b7f8-b69d69270b9c" - }, - "outputs": [], - "source": [ - "chain_with_history = RunnableWithMessageHistory(\n", - " chain,\n", - " lambda session_id: PostgreSQLChatMessageHistory.create_sync(\n", - " engine,\n", - " session_id=session_id,\n", - " table_name=TABLE_NAME,\n", - " ),\n", - " input_messages_key=\"question\",\n", - " history_messages_key=\"history\",\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "id": "0ce596b8-3b78-48fd-9f92-46dccbbfd58b", - "metadata": { - "id": "0ce596b8-3b78-48fd-9f92-46dccbbfd58b" - }, - "outputs": [], - "source": [ - "# This is where we configure the session id\n", - "config = {\"configurable\": {\"session_id\": \"test_session\"}}" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "id": "38e1423b-ba86-4496-9151-25932fab1a8b", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "38e1423b-ba86-4496-9151-25932fab1a8b", - "outputId": "d5c93570-4b0b-4fe8-d19c-4b361fe74291" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content=' Hello Bob, how can I help you today?')" - ] - }, - "execution_count": 18, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "chain_with_history.invoke({\"question\": \"Hi! I'm bob\"}, config=config)" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "id": "2ee4ee62-a216-4fb1-bf33-57476a84cf16", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "2ee4ee62-a216-4fb1-bf33-57476a84cf16", - "outputId": "288fe388-3f60-41b8-8edb-37cfbec18981" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content=' Your name is Bob.')" - ] - }, - "execution_count": 19, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "chain_with_history.invoke({\"question\": \"Whats my name\"}, config=config)" - ] - } - ], - "metadata": { - "colab": { - "provenance": [], - "toc_visible": true - }, - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.8" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/docs/docs/integrations/memory/google_el_carro.ipynb b/docs/docs/integrations/memory/google_el_carro.ipynb new file mode 100644 index 0000000000..10780c74ca --- /dev/null +++ b/docs/docs/integrations/memory/google_el_carro.ipynb @@ -0,0 +1,368 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Google El Carro Oracle Operator\n", + "\n", + "> Google [El Carro Oracle Operator](https://github.com/GoogleCloudPlatform/elcarro-oracle-operator) offers a way to run Oracle databases in Kubernetes as a portable, open source, community driven, no vendor lock-in container orchestration system. El Carro provides a powerful declarative API for comprehensive and consistent configuration and deployment as well as for real-time operations and monitoring. Extend your database application to build AI-powered experiences leveraging Oracle Langchain integrations.\n", + "\n", + "This guide goes over how to use the El Carro Langchain integration to store chat message history with the `ElCarroChatMessageHistory` class.\n", + "\n", + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/googleapis/langchain-google-el-carro-python/blob/main/docs/chat_message_history.ipynb)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Before You Begin\n", + "\n", + "To run this notebook, you will need to do the following:\n", + " * Complete the [Getting Started](https://github.com/googleapis/langchain-google-el-carro-python/tree/main/README.md#getting-started) section" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 🦜🔗 Library Installation\n", + "The integration lives in its own `langchain-google-el-carro` package, so we need to install it." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install --upgrade --quiet langchain-google-el-carro langchain-google-vertexai langchain" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Colab only:** Uncomment the following cell to restart the kernel or use the button to restart the kernel. For Vertex AI Workbench you can restart the terminal using the button on top." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# # Automatically restart kernel after installs so that your environment can access the new packages\n", + "# import IPython\n", + "\n", + "# app = IPython.Application.instance()\n", + "# app.kernel.do_shutdown(True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 🔐 Authentication\n", + "Authenticate to Google Cloud as the IAM user logged into this notebook in order to access your Google Cloud Project.\n", + "\n", + "* If you are using Colab to run this notebook, use the cell below and continue.\n", + "* If you are using Vertex AI Workbench, check out the setup instructions [here](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/setup-env)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from google.colab import auth\n", + "\n", + "auth.authenticate_user()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### ☁ Set Your Google Cloud Project\n", + "Set your Google Cloud project so that you can leverage Google Cloud resources within this notebook.\n", + "\n", + "If you don't know your project ID, try the following:\n", + "\n", + "* Run `gcloud config list`.\n", + "* Run `gcloud projects list`.\n", + "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# @markdown Please fill in the value below with your Google Cloud project ID and then run the cell.\n", + "\n", + "PROJECT_ID = \"my-project-id\" # @param {type:\"string\"}\n", + "\n", + "# Set the project id\n", + "!gcloud config set project {PROJECT_ID}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Basic Usage" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Set Up Oracle Database Connection\n", + "You can find the hostname and port values in the status of the El Carro\n", + "Kubernetes instance. Use the user password you created for your PDB." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# @title Set Your Values Here { display-mode: \"form\" }\n", + "HOST = \"127.0.0.1\" # @param {type: \"string\"}\n", + "PORT = 3307 # @param {type: \"integer\"}\n", + "DATABASE = \"my-database\" # @param {type: \"string\"}\n", + "TABLE_NAME = \"message_store\" # @param {type: \"string\"}\n", + "USER = \"my-user\" # @param {type: \"string\"}\n", + "PASSWORD = input(\"Please provide a password to be used for the database user: \")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### ElCarroEngine Connection Pool\n", + "\n", + "`ElCarroEngine` configures a connection pool to your Oracle database, enabling successful connections from your application and following industry best practices." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_google_el_carro import ElCarroEngine\n", + "\n", + "elcarro_engine = ElCarroEngine.from_instance(\n", + " db_host=HOST,\n", + " db_port=PORT,\n", + " db_name=DATABASE,\n", + " db_user=USER,\n", + " db_password=PASSWORD,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Initialize a table\n", + "The `ElCarroChatMessageHistory` class requires a database table with a specific\n", + "schema in order to store the chat message history.\n", + "\n", + "The `ElCarroEngine` class has a\n", + "method `init_chat_history_table()` that can be used to create a table with the\n", + "proper schema for you." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "elcarro_engine.init_chat_history_table(table_name=TABLE_NAME)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### ElCarroChatMessageHistory\n", + "\n", + "To initialize the `ElCarroChatMessageHistory` class you need to provide only 3\n", + "things:\n", + "\n", + "1. `elcarro_engine` - An instance of an `ElCarroEngine` engine.\n", + "1. `session_id` - A unique identifier string that specifies an id for the\n", + " session.\n", + "1. `table_name` : The name of the table within the Oracle database to store the\n", + " chat message history." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_google_el_carro import ElCarroChatMessageHistory\n", + "\n", + "history = ElCarroChatMessageHistory(\n", + " elcarro_engine=elcarro_engine, session_id=\"test_session\", table_name=TABLE_NAME\n", + ")\n", + "history.add_user_message(\"hi!\")\n", + "history.add_ai_message(\"whats up?\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "history.messages" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Cleaning up\n", + "When the history of a specific session is obsolete and can be deleted, it can be done the following way.\n", + "\n", + "**Note:** Once deleted, the data is no longer stored in AlloyDB and is gone forever." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "history.clear()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 🔗 Chaining\n", + "\n", + "We can easily combine this message history class with [LCEL Runnables](/docs/expression_language/how_to/message_history)\n", + "\n", + "To do this we will use one of [Google's Vertex AI chat models](https://python.langchain.com/docs/integrations/chat/google_vertex_ai_palm) which requires that you [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com) in your Google Cloud Project.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# enable Vertex AI API\n", + "!gcloud services enable aiplatform.googleapis.com" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n", + "from langchain_core.runnables.history import RunnableWithMessageHistory\n", + "from langchain_google_vertexai import ChatVertexAI" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\"system\", \"You are a helpful assistant.\"),\n", + " MessagesPlaceholder(variable_name=\"history\"),\n", + " (\"human\", \"{question}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | ChatVertexAI(project=PROJECT_ID)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "chain_with_history = RunnableWithMessageHistory(\n", + " chain,\n", + " lambda session_id: ElCarroChatMessageHistory(\n", + " elcarro_engine,\n", + " session_id=session_id,\n", + " table_name=TABLE_NAME,\n", + " ),\n", + " input_messages_key=\"question\",\n", + " history_messages_key=\"history\",\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# This is where we configure the session id\n", + "config = {\"configurable\": {\"session_id\": \"test_session\"}}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "chain_with_history.invoke({\"question\": \"Hi! I'm bob\"}, config=config)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "chain_with_history.invoke({\"question\": \"Whats my name\"}, config=config)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.5" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/docs/docs/integrations/platforms/google.mdx b/docs/docs/integrations/platforms/google.mdx index 70c3dbd413..6584ae2c28 100644 --- a/docs/docs/integrations/platforms/google.mdx +++ b/docs/docs/integrations/platforms/google.mdx @@ -168,7 +168,7 @@ See [Googel Cloud usage example](/docs/integrations/document_loaders/google_bigt from langchain_google_bigtable import BigtableLoader ``` -### SQL for MySQL +### Cloud SQL for MySQL > [Google Cloud SQL for MySQL](https://cloud.google.com/sql) is a fully-managed database service that helps you set up, maintain, manage, and administer your MySQL relational databases on Google Cloud. Install the python package: @@ -183,7 +183,7 @@ See [usage example](/docs/integrations/document_loaders/google_cloud_sql_mysql). from langchain_google_cloud_sql_mysql import MySQLEngine, MySQLDocumentLoader ``` -### SQL for SQL Server +### Cloud SQL for SQL Server > [Google Cloud SQL for SQL Server](https://cloud.google.com/sql) is a fully-managed database service that helps you set up, maintain, manage, and administer your SQL Server databases on Google Cloud. Install the python package: @@ -198,7 +198,7 @@ See [usage example](/docs/integrations/document_loaders/google_cloud_sql_mssql). from langchain_google_cloud_sql_mssql import MSSQLEngine, MSSQLLoader ``` -### SQL for PostgreSQL +### Cloud SQL for PostgreSQL > [Google Cloud SQL for PostgreSQL](https://cloud.google.com/sql) is a fully-managed database service that helps you set up, maintain, manage, and administer your PostgreSQL relational databases on Google Cloud. Install the python package: @@ -315,6 +315,22 @@ See [usage example](/docs/integrations/document_loaders/google_spanner). from langchain_google_spanner import SpannerLoader ``` +## El Carro Oracle Operator + +> Google [El Carro Oracle Operator](https://github.com/GoogleCloudPlatform/elcarro-oracle-operator) +offers a way to run Oracle databases in Kubernetes as a portable, open source, +community driven, no vendor lock-in container orchestration system. + +```bash +pip install langchain-google-el-carro +``` + +See [usage example](/docs/integrations/document_loaders/google_el_carro). + +```python +from langchain_google_el_carro import ElCarroLoader +``` + ### Speech-to-Text > [Google Cloud Speech-to-Text](https://cloud.google.com/speech-to-text) is an audio transcription API powered by Google's speech recognition models in Google Cloud. @@ -451,7 +467,7 @@ See [usage example](/docs/integrations/vectorstores/google_spanner). from langchain_google_spanner import SpannerVectorStore ``` -### SQL for PostgreSQL +### Cloud SQL for PostgreSQL > [Google Cloud SQL for PostgreSQL](https://cloud.google.com/sql) is a fully-managed database service that helps you set up, maintain, manage, and administer your PostgreSQL relational databases on Google Cloud. Install the python package: @@ -857,6 +873,22 @@ See [usage example](/docs/integrations/memory/google_datastore). from langchain_google_datastore import DatastoreChatMessageHistory ``` +## El Carro Oracle Operator + +> Google [El Carro Oracle Operator](https://github.com/GoogleCloudPlatform/elcarro-oracle-operator) +offers a way to run Oracle databases in Kubernetes as a portable, open source, +community driven, no vendor lock-in container orchestration system. + +```bash +pip install langchain-google-el-carro +``` + +See [usage example](/docs/integrations/memory/google_el_carro). + +```python +from langchain_google_el_carro import ElCarroChatMessageHistory +``` + ## Chat Loaders ### GMail diff --git a/docs/docs/integrations/vectorstores/google_sql_pg.ipynb b/docs/docs/integrations/vectorstores/google_cloud_sql_pg.ipynb similarity index 99% rename from docs/docs/integrations/vectorstores/google_sql_pg.ipynb rename to docs/docs/integrations/vectorstores/google_cloud_sql_pg.ipynb index 8051833ee4..54c1223738 100644 --- a/docs/docs/integrations/vectorstores/google_sql_pg.ipynb +++ b/docs/docs/integrations/vectorstores/google_cloud_sql_pg.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Google SQL for PostgreSQL\n", + "# Google Cloud SQL for PostgreSQL\n", "\n", "> [Google Cloud SQL](https://cloud.google.com/sql) is a fully managed relational database service that offers high performance, seamless integration, and impressive scalability. It offers PostgreSQL, PostgreSQL, and SQL Server database engines. Extend your database application to build AI-powered experiences leveraging Cloud SQL's Langchain integrations.\n", "\n",