- **Description**: Updated doc for llm/google_vertex_ai_palm with new
functions: `invoke`, `stream`... Changed structure of the document to
match the required one.
- **Issue**: #15664
- **Dependencies**: None
- **Twitter handle**: None
---------
Co-authored-by: Jorge Zaldívar <jzaldivar@google.com>
"**Note:** This is separate from the `Google Generative AI` integration, it exposes [Vertex AI Generative API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on `Google Cloud`.\n"
"**Note:** This is separate from the `Google Generative AI` integration, it exposes [Vertex AI Generative API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on `Google Cloud`.\n",
"\n",
"VertexAI exposes all foundational models available in google cloud:\n",
"- Gemini (`gemini-pro` and `gemini-pro-vision`)\n",
"- Palm 2 for Text (`text-bison`)\n",
"- Codey for Code Generation (`code-bison`)\n",
"\n",
"For a full and updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview)"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "Q_UoF2FKN8Xb"
},
"source": [
"source": [
"## Setting up"
"## Setup"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "8uImJzc4N8Xb"
},
"source": [
"source": [
"By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) customer data to train its foundation models as part of Google Cloud's AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n",
"By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) customer data to train its foundation models as part of Google Cloud's AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n",
"* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners. It also has a large and supportive community, with many resources available online.\n",
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.2\u001b[0m\n",
"* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and artificial intelligence.\n",
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
"* **Powerful:** Python has a rich library of built-in functions and modules, making it easy to perform complex tasks without having to write a lot of code.\n",
"Note: you may need to restart the kernel to use updated packages.\n"
"* **Cross-platform:** Python can be run on a variety of operating systems\n"
"print(llm(\"What are some of the pros and cons of Python as a programming language?\"))"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"id": "38S1FS3qN8Xc"
},
"source": [
"You can also use Gemini model (in preview) with VertexAI:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"**Pros of Python:**\n",
"\n",
"* **Easy to learn and use:** Python is known for its simplicity and readability, making it a great choice for beginners and experienced programmers alike. Its syntax is straightforward and intuitive, allowing developers to quickly pick up the language and start writing code.\n",
"\n",
"\n",
"* **Versatile:** Python is a general-purpose language that can be used for a wide range of applications, including web development, data science, machine learning, and scripting. Its extensive standard library and vast ecosystem of third-party modules make it suitable for a variety of tasks.\n",
"\n",
"\n",
"* **Cross-platform:** Python is compatible with multiple operating systems, including\n"
]
}
],
"source": [
"source": [
"llm = VertexAI(model_name=\"gemini-pro\")\n",
"## Usage\n",
"print(llm(\"What are some of the pros and cons of Python as a programming language?\"))"
"\n",
]
"VertexAI supports all [LLM](/docs/modules/model_io/llms/) functionality."
"'**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a'"
" Justin Bieber was born on March 1, 1994. Bill Clinton was the president of the United States from January 20, 1993, to January 20, 2001.\n",
]
"The final answer is Bill Clinton\n"
},
]
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
}
],
],
"source": [
"source": [
"question = \"Who was the president in the year Justin Beiber was born?\"\n",
"message = \"What are some of the pros and cons of Python as a programming language?\"\n",
"print(chain.invoke({\"question\": question}))"
"model.invoke(message)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AV7oXXuHN8Xd"
},
"source": [
"## Code generation example"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3ZzVtF6tN8Xd"
},
"source": [
"You can now leverage the `Codey API` for code generation within `Vertex AI`.\n",
"'**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a'"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"source": [
"question = \"Write a python function that checks if a string is a valid email address\""
"await model.ainvoke(message)"
]
]
},
},
{
{
@ -217,29 +146,19 @@
"name": "stdout",
"name": "stdout",
"output_type": "stream",
"output_type": "stream",
"text": [
"text": [
"```python\n",
"**Pros:**\n",
"import re\n",
"\n",
"\n",
"def is_valid_email(email):\n",
"* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\n",
"* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\n",
" return pattern.match(email)\n",
"* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\n",
"```\n"
"* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\n",
"* **Cross-platform:** Python is available for a"
]
]
}
}
],
],
"source": [
"source": [
"print(llm(question))"
"for chunk in model.stream(message):\n",
]
" print(chunk, end=\"\", flush=True)"
},
{
"cell_type": "markdown",
"metadata": {
"id": "0WqyaSC2N8Xd"
},
"source": [
"## Full generation info\n",
"\n",
"We can use the `generate` method to get back extra metadata like [safety attributes](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai#safety_attribute_confidence_scoring) and not just text completions"
"['**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a']"
]
]
},
},
"execution_count": 23,
"execution_count": null,
"metadata": {},
"metadata": {},
"output_type": "execute_result"
"output_type": "execute_result"
}
}
],
],
"source": [
"source": [
"result = llm.generate([question])\n",
"model.batch([message])"
"result.generations"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "Wd5M4BBUN8Xd"
},
"source": [
"source": [
"## Asynchronous calls\n",
"We can use the `generate` method to get back extra metadata like [safety attributes](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai#safety_attribute_confidence_scoring) and not just text completions."
"\n",
"With `agenerate` we can make asynchronous calls"
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": null,
"execution_count": null,
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [
{
"data": {
"text/plain": [
"[[GenerationChunk(text='**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a')]]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"source": [
"# If running in a Jupyter notebook you'll need to install nest_asyncio\n",
"[[GenerationChunk(text='**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a')]]"
]
]
},
},
"execution_count": 25,
"execution_count": null,
"metadata": {},
"metadata": {},
"output_type": "execute_result"
"output_type": "execute_result"
}
}
],
],
"source": [
"source": [
"import asyncio\n",
"result = await model.agenerate([message])\n",
"\n",
"result.generations"
"asyncio.run(llm.agenerate([question]))"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "VLsy_4bZN8Xd"
},
"source": [
"source": [
"## Streaming calls\n",
"You can also easily combine with a prompt template for easy structuring of user input. We can do this using [LCEL](/docs/expression_language)"
"\n",
"With `stream` we can stream results from the model"
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": null,
"execution_count": null,
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1. You start with 5 apples.\n",
"2. You throw away 2 apples, so you have 5 - 2 = 3 apples left.\n",
"3. You eat 1 apple, so you have 3 - 1 = 2 apples left.\n",
"I have five apples. I throw two away. I eat one. How many apples do I have left?\n",
"\"\"\"\n",
"print(chain.invoke({\"question\": question}))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"source": [
"import sys"
"You can use different foundational models for specialized in different tasks. \n",
"For an updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview)"
]
]
},
},
{
{
@ -354,49 +301,38 @@
" True if the string is a valid email address, False otherwise.\n",
" True if the string is a valid email address, False otherwise.\n",
" \"\"\"\n",
" \"\"\"\n",
"\n",
"\n",
" # Check for a valid email address format.\n",
" # Compile the regular expression for an email address.\n",
" if not re.match(r\"^[A-Za-z0-9\\.\\+_-]+@[A-Za-z0-9\\._-]+\\.[a-zA-Z]*$\", email):\n",
"question = \"Write a python function that checks if a string is a valid email address\"\n",
" sys.stdout.flush()"
"print(model.invoke(question))"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "4VJ8GwhaN8Xd"
},
"source": [
"source": [
"## Multimodality"
"## Multimodality"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "L7BovARaN8Xe"
},
"source": [
"source": [
"With Gemini, you can use LLM in a multimodal mode:"
"With Gemini, you can use LLM in a multimodal mode:"
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
@ -429,16 +365,14 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "3Vk3gQrrOaL9"
},
"source": [
"source": [
"Let's double-check it's a cat :)"
"Let's double-check it's a cat :)"
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
@ -448,7 +382,7 @@
"<vertexai.generative_models._generative_models.Image at 0x791ded5f1ed0>"
"<vertexai.generative_models._generative_models.Image at 0x791ded5f1ed0>"
]
]
},
},
"execution_count": 9,
"execution_count": null,
"metadata": {},
"metadata": {},
"output_type": "execute_result"
"output_type": "execute_result"
}
}
@ -462,16 +396,14 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "1uEACSSm8AL2"
},
"source": [
"source": [
"You can also pass images as bytes:"
"You can also pass images as bytes:"
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 18,
"execution_count": null,
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
@ -506,18 +438,14 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "AuhF5WQuN8Xe"
},
"source": [
"source": [
"Please, note that you can also use the image stored in GCS (just point the `url` to the full GCS path, starting with `gs://` instead of a local one)."
"Please, note that you can also use the image stored in GCS (just point the `url` to the full GCS path, starting with `gs://` instead of a local one)."
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "qaC2UmxS9WtB"
},
"source": [
"source": [
"And you can also pass a history of a previous chat to the LLM:"
"And you can also pass a history of a previous chat to the LLM:"
]
]
@ -564,18 +492,14 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "VEYAfdBpN8Xe"
},
"source": [
"source": [
"## Vertex Model Garden"
"## Vertex Model Garden"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "N3ptjr_LN8Xe"
},
"source": [
"source": [
"Vertex Model Garden [exposes](https://cloud.google.com/vertex-ai/docs/start/explore-models) open-sourced models that can be deployed and served on Vertex AI. If you have successfully deployed a model from Vertex Model Garden, you can find a corresponding Vertex AI [endpoint](https://cloud.google.com/vertex-ai/docs/general/deployment#what_happens_when_you_deploy_a_model) in the console or via API."
"Vertex Model Garden [exposes](https://cloud.google.com/vertex-ai/docs/start/explore-models) open-sourced models that can be deployed and served on Vertex AI. If you have successfully deployed a model from Vertex Model Garden, you can find a corresponding Vertex AI [endpoint](https://cloud.google.com/vertex-ai/docs/general/deployment#what_happens_when_you_deploy_a_model) in the console or via API."
]
]
@ -604,14 +528,12 @@
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
"print(llm(\"What is the meaning of life?\"))"
"llm.invoke(\"What is the meaning of life?\")"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {},
"id": "TDXoFZ6YN8Xe"
},
"source": [
"source": [
"Like all LLMs, we can then compose it with other components:"
"Like all LLMs, we can then compose it with other components:"
jzaldi
9 months ago
committed by
GitHub