- **Description**: Updated doc for llm/google_vertex_ai_palm with new
functions: `invoke`, `stream`... Changed structure of the document to
match the required one.
- **Issue**: #15664
- **Dependencies**: None
- **Twitter handle**: None
---------
Co-authored-by: Jorge Zaldívar <jzaldivar@google.com>
"**Note:** This is separate from the `Google Generative AI` integration, it exposes [Vertex AI Generative API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on `Google Cloud`.\n"
"**Note:** This is separate from the `Google Generative AI` integration, it exposes [Vertex AI Generative API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on `Google Cloud`.\n",
"\n",
"VertexAI exposes all foundational models available in google cloud:\n",
"- Gemini (`gemini-pro` and `gemini-pro-vision`)\n",
"- Palm 2 for Text (`text-bison`)\n",
"- Codey for Code Generation (`code-bison`)\n",
"\n",
"For a full and updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Q_UoF2FKN8Xb"
},
"metadata": {},
"source": [
"## Setting up"
"## Setup"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8uImJzc4N8Xb"
},
"metadata": {},
"source": [
"By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) customer data to train its foundation models as part of Google Cloud's AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n",
"* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners. It also has a large and supportive community, with many resources available online.\n",
"* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and artificial intelligence.\n",
"* **Powerful:** Python has a rich library of built-in functions and modules, making it easy to perform complex tasks without having to write a lot of code.\n",
"* **Cross-platform:** Python can be run on a variety of operating systems\n"
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.2\u001b[0m\n",
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
"Note: you may need to restart the kernel to use updated packages.\n"
"You can also use Gemini model (in preview) with VertexAI:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"**Pros of Python:**\n",
"\n",
"* **Easy to learn and use:** Python is known for its simplicity and readability, making it a great choice for beginners and experienced programmers alike. Its syntax is straightforward and intuitive, allowing developers to quickly pick up the language and start writing code.\n",
"\n",
"\n",
"* **Versatile:** Python is a general-purpose language that can be used for a wide range of applications, including web development, data science, machine learning, and scripting. Its extensive standard library and vast ecosystem of third-party modules make it suitable for a variety of tasks.\n",
"\n",
"\n",
"* **Cross-platform:** Python is compatible with multiple operating systems, including\n"
]
}
],
"source": [
"llm = VertexAI(model_name=\"gemini-pro\")\n",
"print(llm(\"What are some of the pros and cons of Python as a programming language?\"))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_-9MhhN8N8Xc"
},
"source": [
"## Using in a chain"
"## Usage\n",
"\n",
"VertexAI supports all [LLM](/docs/modules/model_io/llms/) functionality."
" Justin Bieber was born on March 1, 1994. Bill Clinton was the president of the United States from January 20, 1993, to January 20, 2001.\n",
"The final answer is Bill Clinton\n"
]
"data": {
"text/plain": [
"'**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a'"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"question = \"Who was the president in the year Justin Beiber was born?\"\n",
"print(chain.invoke({\"question\": question}))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AV7oXXuHN8Xd"
},
"source": [
"## Code generation example"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3ZzVtF6tN8Xd"
},
"source": [
"You can now leverage the `Codey API` for code generation within `Vertex AI`.\n",
"message = \"What are some of the pros and cons of Python as a programming language?\"\n",
"model.invoke(message)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"outputs": [
{
"data": {
"text/plain": [
"'**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a'"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"question = \"Write a python function that checks if a string is a valid email address\""
"* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\n",
"* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\n",
"* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\n",
"* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\n",
"* **Cross-platform:** Python is available for a"
]
}
],
"source": [
"print(llm(question))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0WqyaSC2N8Xd"
},
"source": [
"## Full generation info\n",
"\n",
"We can use the `generate` method to get back extra metadata like [safety attributes](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai#safety_attribute_confidence_scoring) and not just text completions"
"['**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a']"
]
},
"execution_count": 23,
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"result = llm.generate([question])\n",
"result.generations"
"model.batch([message])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Wd5M4BBUN8Xd"
},
"metadata": {},
"source": [
"## Asynchronous calls\n",
"\n",
"With `agenerate` we can make asynchronous calls"
"We can use the `generate` method to get back extra metadata like [safety attributes](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai#safety_attribute_confidence_scoring) and not just text completions."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"outputs": [
{
"data": {
"text/plain": [
"[[GenerationChunk(text='**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a')]]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# If running in a Jupyter notebook you'll need to install nest_asyncio\n",
"[[GenerationChunk(text='**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a')]]"
]
},
"execution_count": 25,
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import asyncio\n",
"\n",
"asyncio.run(llm.agenerate([question]))"
"result = await model.agenerate([message])\n",
"result.generations"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VLsy_4bZN8Xd"
},
"metadata": {},
"source": [
"## Streaming calls\n",
"\n",
"With `stream` we can stream results from the model"
"You can also easily combine with a prompt template for easy structuring of user input. We can do this using [LCEL](/docs/expression_language)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1. You start with 5 apples.\n",
"2. You throw away 2 apples, so you have 5 - 2 = 3 apples left.\n",
"3. You eat 1 apple, so you have 3 - 1 = 2 apples left.\n",
"I have five apples. I throw two away. I eat one. How many apples do I have left?\n",
"\"\"\"\n",
"print(chain.invoke({\"question\": question}))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"import sys"
"You can use different foundational models for specialized in different tasks. \n",
"For an updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview)"
]
},
{
@ -354,49 +301,38 @@
" True if the string is a valid email address, False otherwise.\n",
" \"\"\"\n",
"\n",
" # Check for a valid email address format.\n",
" if not re.match(r\"^[A-Za-z0-9\\.\\+_-]+@[A-Za-z0-9\\._-]+\\.[a-zA-Z]*$\", email):\n",
" return False\n",
"\n",
" # Check if the domain name exists.\n",
" try:\n",
" domain = email.split(\"@\")[1]\n",
" socket.gethostbyname(domain)\n",
" except socket.gaierror:\n",
" return False\n",
" # Compile the regular expression for an email address.\n",
"question = \"Write a python function that checks if a string is a valid email address\"\n",
"print(model.invoke(question))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4VJ8GwhaN8Xd"
},
"metadata": {},
"source": [
"## Multimodality"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "L7BovARaN8Xe"
},
"metadata": {},
"source": [
"With Gemini, you can use LLM in a multimodal mode:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
@ -429,16 +365,14 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "3Vk3gQrrOaL9"
},
"metadata": {},
"source": [
"Let's double-check it's a cat :)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"outputs": [
{
@ -448,7 +382,7 @@
"<vertexai.generative_models._generative_models.Image at 0x791ded5f1ed0>"
]
},
"execution_count": 9,
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
@ -462,16 +396,14 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "1uEACSSm8AL2"
},
"metadata": {},
"source": [
"You can also pass images as bytes:"
]
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": null,
"metadata": {},
"outputs": [
{
@ -506,18 +438,14 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "AuhF5WQuN8Xe"
},
"metadata": {},
"source": [
"Please, note that you can also use the image stored in GCS (just point the `url` to the full GCS path, starting with `gs://` instead of a local one)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qaC2UmxS9WtB"
},
"metadata": {},
"source": [
"And you can also pass a history of a previous chat to the LLM:"
]
@ -564,18 +492,14 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "VEYAfdBpN8Xe"
},
"metadata": {},
"source": [
"## Vertex Model Garden"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "N3ptjr_LN8Xe"
},
"metadata": {},
"source": [
"Vertex Model Garden [exposes](https://cloud.google.com/vertex-ai/docs/start/explore-models) open-sourced models that can be deployed and served on Vertex AI. If you have successfully deployed a model from Vertex Model Garden, you can find a corresponding Vertex AI [endpoint](https://cloud.google.com/vertex-ai/docs/general/deployment#what_happens_when_you_deploy_a_model) in the console or via API."
]
@ -604,14 +528,12 @@
"metadata": {},
"outputs": [],
"source": [
"print(llm(\"What is the meaning of life?\"))"
"llm.invoke(\"What is the meaning of life?\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TDXoFZ6YN8Xe"
},
"metadata": {},
"source": [
"Like all LLMs, we can then compose it with other components:"
jzaldi
9 months ago
committed by
GitHub