From ed118950fe62bf28b0905e8acfbb026636767698 Mon Sep 17 00:00:00 2001 From: jzaldi Date: Thu, 18 Jan 2024 18:45:27 +0100 Subject: [PATCH] docs: Updated integration docs structure for llm/google_vertex_ai_palm (#16091) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - **Description**: Updated doc for llm/google_vertex_ai_palm with new functions: `invoke`, `stream`... Changed structure of the document to match the required one. - **Issue**: #15664 - **Dependencies**: None - **Twitter handle**: None --------- Co-authored-by: Jorge Zaldívar --- .../llms/google_vertex_ai_palm.ipynb | 546 ++++++++---------- 1 file changed, 238 insertions(+), 308 deletions(-) diff --git a/docs/docs/integrations/llms/google_vertex_ai_palm.ipynb b/docs/docs/integrations/llms/google_vertex_ai_palm.ipynb index 9a4edee338..4f53a75c92 100644 --- a/docs/docs/integrations/llms/google_vertex_ai_palm.ipynb +++ b/docs/docs/integrations/llms/google_vertex_ai_palm.ipynb @@ -11,29 +11,30 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "xazoWTniN8Xa" - }, + "metadata": {}, "source": [ "# Google Cloud Vertex AI\n", "\n", - "**Note:** This is separate from the `Google Generative AI` integration, it exposes [Vertex AI Generative API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on `Google Cloud`.\n" + "**Note:** This is separate from the `Google Generative AI` integration, it exposes [Vertex AI Generative API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on `Google Cloud`.\n", + "\n", + "VertexAI exposes all foundational models available in google cloud:\n", + "- Gemini (`gemini-pro` and `gemini-pro-vision`)\n", + "- Palm 2 for Text (`text-bison`)\n", + "- Codey for Code Generation (`code-bison`)\n", + "\n", + "For a full and updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview)" ] }, { "cell_type": "markdown", - "metadata": { - "id": "Q_UoF2FKN8Xb" - }, + "metadata": {}, "source": [ - "## Setting up" + "## Setup" ] }, { "cell_type": "markdown", - "metadata": { - "id": "8uImJzc4N8Xb" - }, + "metadata": {}, "source": [ "By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) customer data to train its foundation models as part of Google Cloud's AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n", "\n", @@ -52,78 +53,29 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.2\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], "source": [ "%pip install --upgrade --quiet langchain-core langchain-google-vertexai" ] }, { - "cell_type": "code", - "execution_count": 5, + "cell_type": "markdown", "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - " **Pros of Python:**\n", - "\n", - "* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners. It also has a large and supportive community, with many resources available online.\n", - "* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and artificial intelligence.\n", - "* **Powerful:** Python has a rich library of built-in functions and modules, making it easy to perform complex tasks without having to write a lot of code.\n", - "* **Cross-platform:** Python can be run on a variety of operating systems\n" - ] - } - ], "source": [ - "from langchain_google_vertexai import VertexAI\n", + "## Usage\n", "\n", - "llm = VertexAI()\n", - "print(llm(\"What are some of the pros and cons of Python as a programming language?\"))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "38S1FS3qN8Xc" - }, - "source": [ - "You can also use Gemini model (in preview) with VertexAI:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "**Pros of Python:**\n", - "\n", - "* **Easy to learn and use:** Python is known for its simplicity and readability, making it a great choice for beginners and experienced programmers alike. Its syntax is straightforward and intuitive, allowing developers to quickly pick up the language and start writing code.\n", - "\n", - "\n", - "* **Versatile:** Python is a general-purpose language that can be used for a wide range of applications, including web development, data science, machine learning, and scripting. Its extensive standard library and vast ecosystem of third-party modules make it suitable for a variety of tasks.\n", - "\n", - "\n", - "* **Cross-platform:** Python is compatible with multiple operating systems, including\n" - ] - } - ], - "source": [ - "llm = VertexAI(model_name=\"gemini-pro\")\n", - "print(llm(\"What are some of the pros and cons of Python as a programming language?\"))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "_-9MhhN8N8Xc" - }, - "source": [ - "## Using in a chain" + "VertexAI supports all [LLM](/docs/modules/model_io/llms/) functionality." ] }, { @@ -131,204 +83,199 @@ "execution_count": null, "metadata": {}, "outputs": [], + "source": [ + "from langchain_google_vertexai import VertexAI" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "model = VertexAI(model_name=\"gemini-pro\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a'" + ] + }, + "execution_count": null, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "message = \"What are some of the pros and cons of Python as a programming language?\"\n", + "model.invoke(message)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a'" + ] + }, + "execution_count": null, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "await model.ainvoke(message)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "**Pros:**\n", + "\n", + "* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\n", + "* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\n", + "* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\n", + "* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\n", + "* **Cross-platform:** Python is available for a" + ] + } + ], + "source": [ + "for chunk in model.stream(message):\n", + " print(chunk, end=\"\", flush=True)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a']" + ] + }, + "execution_count": null, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model.batch([message])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can use the `generate` method to get back extra metadata like [safety attributes](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai#safety_attribute_confidence_scoring) and not just text completions." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[[GenerationChunk(text='**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a')]]" + ] + }, + "execution_count": null, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result = model.generate([message])\n", + "result.generations" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[[GenerationChunk(text='**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a')]]" + ] + }, + "execution_count": null, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result = await model.agenerate([message])\n", + "result.generations" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can also easily combine with a prompt template for easy structuring of user input. We can do this using [LCEL](/docs/expression_language)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1. You start with 5 apples.\n", + "2. You throw away 2 apples, so you have 5 - 2 = 3 apples left.\n", + "3. You eat 1 apple, so you have 3 - 1 = 2 apples left.\n", + "\n", + "Therefore, you have 2 apples left.\n" + ] + } + ], "source": [ "from langchain_core.prompts import PromptTemplate\n", "\n", "template = \"\"\"Question: {question}\n", "\n", "Answer: Let's think step by step.\"\"\"\n", - "prompt = PromptTemplate.from_template(template)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "chain = prompt | llm" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - " Justin Bieber was born on March 1, 1994. Bill Clinton was the president of the United States from January 20, 1993, to January 20, 2001.\n", - "The final answer is Bill Clinton\n" - ] - } - ], - "source": [ - "question = \"Who was the president in the year Justin Beiber was born?\"\n", + "prompt = PromptTemplate.from_template(template)\n", + "\n", + "chain = prompt | model\n", + "\n", + "question = \"\"\"\n", + "I have five apples. I throw two away. I eat one. How many apples do I have left?\n", + "\"\"\"\n", "print(chain.invoke({\"question\": question}))" ] }, { "cell_type": "markdown", - "metadata": { - "id": "AV7oXXuHN8Xd" - }, - "source": [ - "## Code generation example" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "3ZzVtF6tN8Xd" - }, - "source": [ - "You can now leverage the `Codey API` for code generation within `Vertex AI`.\n", - "\n", - "The model names are:\n", - "- `code-bison`: for code suggestion\n", - "- `code-gecko`: for code completion" - ] - }, - { - "cell_type": "code", - "execution_count": null, "metadata": {}, - "outputs": [], "source": [ - "llm = VertexAI(model_name=\"code-bison\", max_output_tokens=1000, temperature=0.3)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "question = \"Write a python function that checks if a string is a valid email address\"" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "```python\n", - "import re\n", - "\n", - "def is_valid_email(email):\n", - " pattern = re.compile(r\"[^@]+@[^@]+\\.[^@]+\")\n", - " return pattern.match(email)\n", - "```\n" - ] - } - ], - "source": [ - "print(llm(question))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "0WqyaSC2N8Xd" - }, - "source": [ - "## Full generation info\n", - "\n", - "We can use the `generate` method to get back extra metadata like [safety attributes](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai#safety_attribute_confidence_scoring) and not just text completions" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[[GenerationChunk(text='```python\\nimport re\\n\\ndef is_valid_email(email):\\n pattern = re.compile(r\"[^@]+@[^@]+\\\\.[^@]+\")\\n return pattern.match(email)\\n```', generation_info={'is_blocked': False, 'safety_attributes': {'Health': 0.1}})]]" - ] - }, - "execution_count": 23, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "result = llm.generate([question])\n", - "result.generations" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Wd5M4BBUN8Xd" - }, - "source": [ - "## Asynchronous calls\n", - "\n", - "With `agenerate` we can make asynchronous calls" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# If running in a Jupyter notebook you'll need to install nest_asyncio\n", - "\n", - "%pip install --upgrade --quiet nest_asyncio\n", - "\n", - "import nest_asyncio\n", - "\n", - "nest_asyncio.apply()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "LLMResult(generations=[[GenerationChunk(text='```python\\nimport re\\n\\ndef is_valid_email(email):\\n pattern = re.compile(r\"[^@]+@[^@]+\\\\.[^@]+\")\\n return pattern.match(email)\\n```', generation_info={'is_blocked': False, 'safety_attributes': {'Health': 0.1}})]], llm_output=None, run=[RunInfo(run_id=UUID('caf74e91-aefb-48ac-8031-0c505fcbbcc6'))])" - ] - }, - "execution_count": 25, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "import asyncio\n", - "\n", - "asyncio.run(llm.agenerate([question]))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "VLsy_4bZN8Xd" - }, - "source": [ - "## Streaming calls\n", - "\n", - "With `stream` we can stream results from the model" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import sys" + "You can use different foundational models for specialized in different tasks. \n", + "For an updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview)" ] }, { @@ -354,49 +301,38 @@ " True if the string is a valid email address, False otherwise.\n", " \"\"\"\n", "\n", - " # Check for a valid email address format.\n", - " if not re.match(r\"^[A-Za-z0-9\\.\\+_-]+@[A-Za-z0-9\\._-]+\\.[a-zA-Z]*$\", email):\n", - " return False\n", + " # Compile the regular expression for an email address.\n", + " regex = re.compile(r\"[^@]+@[^@]+\\.[^@]+\")\n", "\n", - " # Check if the domain name exists.\n", - " try:\n", - " domain = email.split(\"@\")[1]\n", - " socket.gethostbyname(domain)\n", - " except socket.gaierror:\n", - " return False\n", - "\n", - " return True\n", - "```" + " # Check if the string matches the regular expression.\n", + " return regex.match(email) is not None\n", + "```\n" ] } ], "source": [ - "for chunk in llm.stream(question):\n", - " sys.stdout.write(chunk)\n", - " sys.stdout.flush()" + "llm = VertexAI(model_name=\"code-bison\", max_output_tokens=1000, temperature=0.3)\n", + "question = \"Write a python function that checks if a string is a valid email address\"\n", + "print(model.invoke(question))" ] }, { "cell_type": "markdown", - "metadata": { - "id": "4VJ8GwhaN8Xd" - }, + "metadata": {}, "source": [ "## Multimodality" ] }, { "cell_type": "markdown", - "metadata": { - "id": "L7BovARaN8Xe" - }, + "metadata": {}, "source": [ "With Gemini, you can use LLM in a multimodal mode:" ] }, { "cell_type": "code", - "execution_count": 8, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -429,16 +365,14 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "3Vk3gQrrOaL9" - }, + "metadata": {}, "source": [ "Let's double-check it's a cat :)" ] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -448,7 +382,7 @@ "" ] }, - "execution_count": 9, + "execution_count": null, "metadata": {}, "output_type": "execute_result" } @@ -462,16 +396,14 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "1uEACSSm8AL2" - }, + "metadata": {}, "source": [ "You can also pass images as bytes:" ] }, { "cell_type": "code", - "execution_count": 18, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -506,18 +438,14 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "AuhF5WQuN8Xe" - }, + "metadata": {}, "source": [ "Please, note that you can also use the image stored in GCS (just point the `url` to the full GCS path, starting with `gs://` instead of a local one)." ] }, { "cell_type": "markdown", - "metadata": { - "id": "qaC2UmxS9WtB" - }, + "metadata": {}, "source": [ "And you can also pass a history of a previous chat to the LLM:" ] @@ -564,18 +492,14 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "VEYAfdBpN8Xe" - }, + "metadata": {}, "source": [ "## Vertex Model Garden" ] }, { "cell_type": "markdown", - "metadata": { - "id": "N3ptjr_LN8Xe" - }, + "metadata": {}, "source": [ "Vertex Model Garden [exposes](https://cloud.google.com/vertex-ai/docs/start/explore-models) open-sourced models that can be deployed and served on Vertex AI. If you have successfully deployed a model from Vertex Model Garden, you can find a corresponding Vertex AI [endpoint](https://cloud.google.com/vertex-ai/docs/general/deployment#what_happens_when_you_deploy_a_model) in the console or via API." ] @@ -604,14 +528,12 @@ "metadata": {}, "outputs": [], "source": [ - "print(llm(\"What is the meaning of life?\"))" + "llm.invoke(\"What is the meaning of life?\")" ] }, { "cell_type": "markdown", - "metadata": { - "id": "TDXoFZ6YN8Xe" - }, + "metadata": {}, "source": [ "Like all LLMs, we can then compose it with other components:" ] @@ -643,8 +565,16 @@ "name": "python3" }, "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", "name": "python", - "version": "3.11.4" + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.6" } }, "nbformat": 4,