Archives/langchain
2
0
Fork 1
mirror of https://github.com/hwchase17/langchain synced 2024-11-18 09:25:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: Updated integration docs structure for llm/google_vertex_ai_palm (#16091)

Browse Source 
- **Description**: Updated doc for llm/google_vertex_ai_palm with new
functions: `invoke`, `stream`... Changed structure of the document to
match the required one.
- **Issue**: #15664 
- **Dependencies**: None
- **Twitter handle**: None

---------

Co-authored-by: Jorge Zaldívar <jzaldivar@google.com>
This commit is contained in:
jzaldi 2024-01-18 18:45:27 +01:00 committed by GitHub
parent aa2e642ce3
commit ed118950fe
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 238 additions and 308 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

546
docs/docs/integrations/llms/google_vertex_ai_palm.ipynb
View File

@ -11,29 +11,30 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "xazoWTniN8Xa"
},
"metadata": {},
"source": [
"# Google Cloud Vertex AI\n",
"\n",
"**Note:** This is separate from the `Google Generative AI` integration, it exposes [Vertex AI Generative API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on `Google Cloud`.\n"
"**Note:** This is separate from the `Google Generative AI` integration, it exposes [Vertex AI Generative API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on `Google Cloud`.\n",
"\n",
"VertexAI exposes all foundational models available in google cloud:\n",
"- Gemini (`gemini-pro` and `gemini-pro-vision`)\n",
"- Palm 2 for Text (`text-bison`)\n",
"- Codey for Code Generation (`code-bison`)\n",
"\n",
"For a full and updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Q_UoF2FKN8Xb"
},
"metadata": {},
"source": [
"## Setting up"
"## Setup"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8uImJzc4N8Xb"
},
"metadata": {},
"source": [
"By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) customer data to train its foundation models as part of Google Cloud's AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n",
"\n",
@ -52,78 +53,29 @@
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.2\u001b[0m\n",
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
"Note: you may need to restart the kernel to use updated packages.\n"
]
}
],
"source": [
"%pip install --upgrade --quiet langchain-core langchain-google-vertexai"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" **Pros of Python:**\n",
"\n",
"* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners. It also has a large and supportive community, with many resources available online.\n",
"* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and artificial intelligence.\n",
"* **Powerful:** Python has a rich library of built-in functions and modules, making it easy to perform complex tasks without having to write a lot of code.\n",
"* **Cross-platform:** Python can be run on a variety of operating systems\n"
]
}
],
"source": [
"from langchain_google_vertexai import VertexAI\n",
"\n",
"llm = VertexAI()\n",
"print(llm(\"What are some of the pros and cons of Python as a programming language?\"))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "38S1FS3qN8Xc"
},
"source": [
"You can also use Gemini model (in preview) with VertexAI:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"**Pros of Python:**\n",
"\n",
"* **Easy to learn and use:** Python is known for its simplicity and readability, making it a great choice for beginners and experienced programmers alike. Its syntax is straightforward and intuitive, allowing developers to quickly pick up the language and start writing code.\n",
"\n",
"\n",
"* **Versatile:** Python is a general-purpose language that can be used for a wide range of applications, including web development, data science, machine learning, and scripting. Its extensive standard library and vast ecosystem of third-party modules make it suitable for a variety of tasks.\n",
"\n",
"\n",
"* **Cross-platform:** Python is compatible with multiple operating systems, including\n"
]
}
],
"source": [
"llm = VertexAI(model_name=\"gemini-pro\")\n",
"print(llm(\"What are some of the pros and cons of Python as a programming language?\"))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_-9MhhN8N8Xc"
},
"source": [
"## Using in a chain"
"## Usage\n",
"\n",
"VertexAI supports all [LLM](/docs/modules/model_io/llms/) functionality."
]
},
{
@ -131,204 +83,199 @@
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_google_vertexai import VertexAI"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model = VertexAI(model_name=\"gemini-pro\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a'"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"message = \"What are some of the pros and cons of Python as a programming language?\"\n",
"model.invoke(message)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a'"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"await model.ainvoke(message)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"**Pros:**\n",
"\n",
"* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\n",
"* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\n",
"* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\n",
"* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\n",
"* **Cross-platform:** Python is available for a"
]
}
],
"source": [
"for chunk in model.stream(message):\n",
" print(chunk, end=\"\", flush=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a']"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.batch([message])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can use the `generate` method to get back extra metadata like [safety attributes](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai#safety_attribute_confidence_scoring) and not just text completions."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[[GenerationChunk(text='**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a')]]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"result = model.generate([message])\n",
"result.generations"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[[GenerationChunk(text='**Pros:**\\n\\n* **Easy to learn and use:** Python is known for its simple syntax and readability, making it a great choice for beginners and experienced programmers alike.\\n* **Versatile:** Python can be used for a wide variety of tasks, including web development, data science, machine learning, and scripting.\\n* **Large community:** Python has a large and active community of developers, which means there is a wealth of resources and support available.\\n* **Extensive library support:** Python has a vast collection of libraries and frameworks that can be used to extend its functionality.\\n* **Cross-platform:** Python is available for a')]]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"result = await model.agenerate([message])\n",
"result.generations"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can also easily combine with a prompt template for easy structuring of user input. We can do this using [LCEL](/docs/expression_language)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1. You start with 5 apples.\n",
"2. You throw away 2 apples, so you have 5 - 2 = 3 apples left.\n",
"3. You eat 1 apple, so you have 3 - 1 = 2 apples left.\n",
"\n",
"Therefore, you have 2 apples left.\n"
]
}
],
"source": [
"from langchain_core.prompts import PromptTemplate\n",
"\n",
"template = \"\"\"Question: {question}\n",
"\n",
"Answer: Let's think step by step.\"\"\"\n",
"prompt = PromptTemplate.from_template(template)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"chain = prompt | llm"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Justin Bieber was born on March 1, 1994. Bill Clinton was the president of the United States from January 20, 1993, to January 20, 2001.\n",
"The final answer is Bill Clinton\n"
]
}
],
"source": [
"question = \"Who was the president in the year Justin Beiber was born?\"\n",
"prompt = PromptTemplate.from_template(template)\n",
"\n",
"chain = prompt | model\n",
"\n",
"question = \"\"\"\n",
"I have five apples. I throw two away. I eat one. How many apples do I have left?\n",
"\"\"\"\n",
"print(chain.invoke({\"question\": question}))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AV7oXXuHN8Xd"
},
"source": [
"## Code generation example"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3ZzVtF6tN8Xd"
},
"source": [
"You can now leverage the `Codey API` for code generation within `Vertex AI`.\n",
"\n",
"The model names are:\n",
"- `code-bison`: for code suggestion\n",
"- `code-gecko`: for code completion"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"llm = VertexAI(model_name=\"code-bison\", max_output_tokens=1000, temperature=0.3)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"question = \"Write a python function that checks if a string is a valid email address\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"```python\n",
"import re\n",
"\n",
"def is_valid_email(email):\n",
" pattern = re.compile(r\"[^@]+@[^@]+\\.[^@]+\")\n",
" return pattern.match(email)\n",
"```\n"
]
}
],
"source": [
"print(llm(question))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0WqyaSC2N8Xd"
},
"source": [
"## Full generation info\n",
"\n",
"We can use the `generate` method to get back extra metadata like [safety attributes](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai#safety_attribute_confidence_scoring) and not just text completions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[[GenerationChunk(text='```python\\nimport re\\n\\ndef is_valid_email(email):\\n pattern = re.compile(r\"[^@]+@[^@]+\\\\.[^@]+\")\\n return pattern.match(email)\\n```', generation_info={'is_blocked': False, 'safety_attributes': {'Health': 0.1}})]]"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"result = llm.generate([question])\n",
"result.generations"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Wd5M4BBUN8Xd"
},
"source": [
"## Asynchronous calls\n",
"\n",
"With `agenerate` we can make asynchronous calls"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# If running in a Jupyter notebook you'll need to install nest_asyncio\n",
"\n",
"%pip install --upgrade --quiet nest_asyncio\n",
"\n",
"import nest_asyncio\n",
"\n",
"nest_asyncio.apply()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"LLMResult(generations=[[GenerationChunk(text='```python\\nimport re\\n\\ndef is_valid_email(email):\\n pattern = re.compile(r\"[^@]+@[^@]+\\\\.[^@]+\")\\n return pattern.match(email)\\n```', generation_info={'is_blocked': False, 'safety_attributes': {'Health': 0.1}})]], llm_output=None, run=[RunInfo(run_id=UUID('caf74e91-aefb-48ac-8031-0c505fcbbcc6'))])"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import asyncio\n",
"\n",
"asyncio.run(llm.agenerate([question]))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VLsy_4bZN8Xd"
},
"source": [
"## Streaming calls\n",
"\n",
"With `stream` we can stream results from the model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import sys"
"You can use different foundational models for specialized in different tasks. \n",
"For an updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview)"
]
},
{
@ -354,49 +301,38 @@
" True if the string is a valid email address, False otherwise.\n",
" \"\"\"\n",
"\n",
" # Check for a valid email address format.\n",
" if not re.match(r\"^[A-Za-z0-9\\.\\+_-]+@[A-Za-z0-9\\._-]+\\.[a-zA-Z]*$\", email):\n",
" return False\n",
" # Compile the regular expression for an email address.\n",
" regex = re.compile(r\"[^@]+@[^@]+\\.[^@]+\")\n",
"\n",
" # Check if the domain name exists.\n",
" try:\n",
" domain = email.split(\"@\")[1]\n",
" socket.gethostbyname(domain)\n",
" except socket.gaierror:\n",
" return False\n",
"\n",
" return True\n",
"```"
" # Check if the string matches the regular expression.\n",
" return regex.match(email) is not None\n",
"```\n"
]
}
],
"source": [
"for chunk in llm.stream(question):\n",
" sys.stdout.write(chunk)\n",
" sys.stdout.flush()"
"llm = VertexAI(model_name=\"code-bison\", max_output_tokens=1000, temperature=0.3)\n",
"question = \"Write a python function that checks if a string is a valid email address\"\n",
"print(model.invoke(question))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4VJ8GwhaN8Xd"
},
"metadata": {},
"source": [
"## Multimodality"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "L7BovARaN8Xe"
},
"metadata": {},
"source": [
"With Gemini, you can use LLM in a multimodal mode:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
@ -429,16 +365,14 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "3Vk3gQrrOaL9"
},
"metadata": {},
"source": [
"Let's double-check it's a cat :)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"outputs": [
{
@ -448,7 +382,7 @@
"<vertexai.generative_models._generative_models.Image at 0x791ded5f1ed0>"
]
},
"execution_count": 9,
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
@ -462,16 +396,14 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "1uEACSSm8AL2"
},
"metadata": {},
"source": [
"You can also pass images as bytes:"
]
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": null,
"metadata": {},
"outputs": [
{
@ -506,18 +438,14 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "AuhF5WQuN8Xe"
},
"metadata": {},
"source": [
"Please, note that you can also use the image stored in GCS (just point the `url` to the full GCS path, starting with `gs://` instead of a local one)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qaC2UmxS9WtB"
},
"metadata": {},
"source": [
"And you can also pass a history of a previous chat to the LLM:"
]
@ -564,18 +492,14 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "VEYAfdBpN8Xe"
},
"metadata": {},
"source": [
"## Vertex Model Garden"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "N3ptjr_LN8Xe"
},
"metadata": {},
"source": [
"Vertex Model Garden [exposes](https://cloud.google.com/vertex-ai/docs/start/explore-models) open-sourced models that can be deployed and served on Vertex AI. If you have successfully deployed a model from Vertex Model Garden, you can find a corresponding Vertex AI [endpoint](https://cloud.google.com/vertex-ai/docs/general/deployment#what_happens_when_you_deploy_a_model) in the console or via API."
]
@ -604,14 +528,12 @@
"metadata": {},
"outputs": [],
"source": [
"print(llm(\"What is the meaning of life?\"))"
"llm.invoke(\"What is the meaning of life?\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TDXoFZ6YN8Xe"
},
"metadata": {},
"source": [
"Like all LLMs, we can then compose it with other components:"
]
@ -643,8 +565,16 @@
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"version": "3.11.4"
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.6"
}
},
"nbformat": 4,