You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
openai-cookbook/examples/Parse_PDF_docs_for_RAG.ipynb

2227 lines
3.0 MiB
Plaintext

Added a new notebook: "Parse PDF docs for RAG applications" (#1080) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: prestontuggle <97747561+prestontuggle@users.noreply.github.com> Co-authored-by: Shyamal H Anadkat <shyamal@openai.com> Co-authored-by: Simón Fishman <simonpfish@gmail.com> Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com> Co-authored-by: aalmaksour82 <49364099+aalmaksour82@users.noreply.github.com> Co-authored-by: colin-openai <119888926+colin-openai@users.noreply.github.com> Co-authored-by: Michael Wu <mwu1993@users.noreply.github.com> Co-authored-by: Logan Kilpatrick <logan@openai.com> Co-authored-by: Viet Hoang Tran Duong <36019296+viethoangtranduong@users.noreply.github.com> Co-authored-by: Christine Belzie <105683440+CBID2@users.noreply.github.com> Co-authored-by: Eliah Kagan <degeneracypressure@gmail.com> Co-authored-by: recordcrash <recordcrash@users.noreply.github.com> Co-authored-by: Stefano Lottini <hemidactylus@users.noreply.github.com> Co-authored-by: Safa Asgar <70315479+SaFaUU@users.noreply.github.com> Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com> Co-authored-by: Will DePue <will@depue.net> Co-authored-by: ys64 <815824+ys64@users.noreply.github.com> Co-authored-by: Shawn Yuxuan Tong <tongyuxuan361@gmail.com> Co-authored-by: Steven Pousty <steve.pousty@gmail.com> Co-authored-by: Puneet Dhiman <142409038+PuneetDhimanShorthillsAI@users.noreply.github.com> Co-authored-by: Krista Pratico <krpratic@microsoft.com> Co-authored-by: dongqqcom <32085836+dongqqcom@users.noreply.github.com> Co-authored-by: Alvaro Videla <videlalvaro@gmail.com> Co-authored-by: DevilsWorkShop <ashokmanghat@gmail.com> Co-authored-by: Ashok Manghat <amanghat@rmplc.net> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matthew Jericho Go Sy <69558553+jerichosy@users.noreply.github.com> Co-authored-by: Farzad Sunavala <40604067+farzad528@users.noreply.github.com> Co-authored-by: Nirant <NirantK@users.noreply.github.com> Co-authored-by: Cathy Chen <cathykaichen@gmail.com> Co-authored-by: gusmally <hannahmbmoraes@gmail.com> Co-authored-by: Chuong Ho <31106432+chuongmep@users.noreply.github.com> Co-authored-by: ridrisa <138629783+ridrisa@users.noreply.github.com> Co-authored-by: Xin(Leo) Jing <jingxin@berkeley.edu> Co-authored-by: Per Harald Borgen <perhborgen@gmail.com> Co-authored-by: Hoang Viet Khoa <khoahv92@gmail.com> Co-authored-by: Stefano Fiorucci <44616784+anakin87@users.noreply.github.com> Co-authored-by: Albarqawi <barqawi.88@outlook.com> Co-authored-by: Saarika Bhasi <55930906+saarikabhasi@users.noreply.github.com> Co-authored-by: Daniel <10074684+danieltprice@users.noreply.github.com> Co-authored-by: Dhruv Anand <105786647+dhruv-anand-aintech@users.noreply.github.com> Co-authored-by: Jiří Hofman <jiri.hofman@gmail.com> Co-authored-by: Fayaz Rahman <fayazrahman4u@gmail.com> Co-authored-by: Anish Shah <93145909+ash0ts@users.noreply.github.com> Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Emil Sedgh <emilsedgh@kde.org> Co-authored-by: Megan O'Keefe <3137106+askmeegs@users.noreply.github.com> Co-authored-by: Joschka Braun <47435119+joschkabraun@users.noreply.github.com> Co-authored-by: Roger Zurawicki <zurawiki@users.noreply.github.com> Co-authored-by: pavlovp <pavel.pavlov1990@gmail.com> Co-authored-by: Surav Shrestha <98219089+suravshresth@users.noreply.github.com> Co-authored-by: vrushankportkey <134934501+vrushankportkey@users.noreply.github.com> Co-authored-by: Soonoh <chk0ndanger@gmail.com> Co-authored-by: Mayuresh Dharwadkar <98738585+Mayureshd-18@users.noreply.github.com> Co-authored-by: Yashwant Jodha <76436993+yashwantjodha@users.noreply.github.com> Co-authored-by: Guspan Tanadi <36249910+guspan-tanadi@users.noreply.github.com> Co-authored-by: Ana Martins <60753223+OutSystemsAMM@users.noreply.github.com> Co-authored-by: Greg Richardson <greg.nmr@gmail.com> Co-authored-by: john <johnoctubre7@gmail.com> Co-authored-by: John Octubre <johnoctubre@Johns-MacBook-Pro.local> Co-authored-by: jhills20 <70035505+jhills20@users.noreply.github.com> Co-authored-by: Tad <wptady@gmail.com> Co-authored-by: Ilan Bigio <ilanbigio@gmail.com> Co-authored-by: Ilan Bigio <ilan@openai.com> Co-authored-by: royziv11 <103690170+royziv11@users.noreply.github.com> Co-authored-by: Gabor Cselle <gaborcselle@users.noreply.github.com> Co-authored-by: D. Carpintero <6709785+dcarpintero@users.noreply.github.com> Co-authored-by: Ed Spencer <ed@edspencer.net> Co-authored-by: Ravi Theja <ravi03071991@gmail.com> Co-authored-by: dylanra-openai <149511600+dylanra-openai@users.noreply.github.com> Co-authored-by: Taranjeet Singh <reachtotj@gmail.com> Co-authored-by: Frode Jensen <jensen.frode@gmail.com> Co-authored-by: Lionel Cheng <60159831+lionelchg@users.noreply.github.com> Co-authored-by: lionelchg <Cheng.Lionel@bcg.com> Co-authored-by: Jing Ai <42414856+jingairpi@users.noreply.github.com> Co-authored-by: Jing Ai <jingai@jings-air-2020.lan> Co-authored-by: Spring_MT <today.is.sky.blue.sky@gmail.com> Co-authored-by: kevleininger <kevleininger@gmail.com> Co-authored-by: Prakul <discover.prakul@gmail.com> Co-authored-by: Logan Kilpatrick <23kilpatrick23@gmail.com> Co-authored-by: Jiang Yucheng <fatjyc@gmail.com> Co-authored-by: Haomin Liu <644074553@qq.com> Co-authored-by: Xavier Amatriain <xavier.amatriain@gmail.com> Co-authored-by: Caio Curitiba Marcellos <caiocuritiba@gmail.com> Co-authored-by: Kesku <62210496+kesku@users.noreply.github.com> Co-authored-by: markbigears <86395716+markbigears@users.noreply.github.com> Co-authored-by: bigears <mark.forsyth@yourbigears.com> Co-authored-by: Nghiauet <63385521+Nghiauet@users.noreply.github.com> Co-authored-by: Vince Fulco--Bighire.tools <vince@bighire.io> Co-authored-by: Wang22004K <152562528+Wang22004K@users.noreply.github.com> Co-authored-by: Shaurya Rohatgi <shauryr@gmail.com> Co-authored-by: Dhruv Singh <ds3638@columbia.edu> Co-authored-by: Adam Hendel <ChuckHend@users.noreply.github.com> Co-authored-by: Enoch Cheung <enoch@enochc.com> Co-authored-by: Zanie Blue <contact@zanie.dev> Co-authored-by: rissois <44072214+rissois@users.noreply.github.com> Co-authored-by: ayush rajgor <ayushrajgorar@gmail.com> Co-authored-by: teomusatoiu <156829031+teomusatoiu@users.noreply.github.com> Co-authored-by: James Briggs <35938317+jamescalam@users.noreply.github.com> Co-authored-by: Shivam Rastogi <shivamsupr@gmail.com> Co-authored-by: Alex Yang <himself65@outlook.com> Co-authored-by: Elmira Ghorbani <elmira.ghorbani96@gmail.com> Co-authored-by: gloryjain <glory@openai.com> Co-authored-by: Andrew Peng <apeng@berkeley.edu>
3 months ago
{
"cells": [
{
"cell_type": "markdown",
"id": "f6ce3d79",
"metadata": {},
"source": [
"# Parsing PDF documents for RAG applications\n",
"\n",
"This notebook shows how to leverage GPT-4V to turn rich PDF documents such as slide decks or exports from web pages into usable content for your RAG application.\n",
"\n",
"This technique can be used if you have a lot of unstructured data containing valuable information that you want to be able to retrieve as part of your RAG pipeline.\n",
"\n",
"For example, you could build a Knowledge Assistant that could answer user queries about your company or product based on information contained in PDF documents. \n",
"\n",
"The example documents used in this notebook are located at [data/example_pdfs](data/example_pdfs). They are related to OpenAI's APIs and various techniques that can be used as part of LLM projects."
]
},
{
"cell_type": "markdown",
"id": "6163ace6",
"metadata": {},
"source": [
"## Data preparation\n",
"\n",
"In this section, we will process our input data to prepare it for retrieval.\n",
"\n",
"We will do this in 2 ways:\n",
"\n",
"1. Extracting text with pdfminer\n",
"2. Converting the PDF pages to images to analyze them with GPT-4V\n",
"\n",
"You can skip the 1st method if you want to only use the content inferred from the image analysis."
]
},
{
"cell_type": "markdown",
"id": "70a87e9f",
"metadata": {},
"source": [
"### Setup\n",
"\n",
"We need to install a few libraries to convert the PDF to images and extract the text (optional).\n",
"\n",
"**Note: You need to install `poppler` on your machine for the `pdf2image` library to work. You can follow the instructions to install it [here](https://pypi.org/project/pdf2image/).**"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "29a05ae1",
"metadata": {},
"outputs": [],
"source": [
"%pip install pdf2image\n",
"%pip install pdfminer\n",
"%pip install openai\n",
"%pip install scikit-learn\n",
"%pip install rich\n",
"%pip install tqdm\n",
"%pip install concurrent"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "14c84b52",
"metadata": {},
"outputs": [],
"source": [
"# Imports\n",
"from pdf2image import convert_from_path\n",
"from pdf2image.exceptions import (\n",
" PDFInfoNotInstalledError,\n",
" PDFPageCountError,\n",
" PDFSyntaxError\n",
")\n",
"from pdfminer.high_level import extract_text\n",
"import base64\n",
"from io import BytesIO\n",
"import os\n",
"import concurrent\n",
"from tqdm import tqdm\n",
"from openai import OpenAI\n",
"import re\n",
"import pandas as pd \n",
"from sklearn.metrics.pairwise import cosine_similarity\n",
"import json\n",
"import numpy as np\n",
"from rich import print\n",
"from ast import literal_eval"
]
},
{
"cell_type": "markdown",
"id": "5cb75cb1",
"metadata": {},
"source": [
"### File processing"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "572ba89b",
"metadata": {},
"outputs": [],
"source": [
"def convert_doc_to_images(path):\n",
" images = convert_from_path(path)\n",
" return images\n",
"\n",
"def extract_text_from_doc(path):\n",
" text = extract_text(path)\n",
" page_text = []\n",
" return text"
]
},
{
"cell_type": "markdown",
"id": "ef1bca4c",
"metadata": {},
"source": [
"#### Testing with an example"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "cbbe40fb",
"metadata": {},
"outputs": [],
"source": [
"file_path = \"data/example_pdfs/fine-tuning-deck.pdf\"\n",
"\n",
"images = convert_doc_to_images(file_path)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "3a0c8653",
"metadata": {},
"outputs": [],
"source": [
"text = extract_text_from_doc(file_path)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "eab8b17b",
"metadata": {},
"outputs": [
{
"data": {
"image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCARlB9ADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD5/ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK
"image/png": "iVBORw0KGgoAAAANSUhEUgAAB9AAAARlCAIAAAButct1AADve0lEQVR4Aezdd5glVdkv7DMwpCEMOecoMGQBkRwkSYYXBVFRRBAkqqC8IAIGEAliQhRQxAQiCBhIknPOkgTJOTNk/X5axzr11a69e3ec7uHuP8ZVq1a8dzVe17NXP/V//o8fAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECBAgAABAgQIECB
"text/plain": [
"<PIL.PpmImagePlugin.PpmImageFile image mode=RGB size=2000x1125>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCARlB9ADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD5/ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK
"image/png": "iVBORw0KGgoAAAANSUhEUgAAB9AAAARlCAIAAAButct1AAEAAElEQVR4AezddaAV1fr/8auComITgmIhYoGigmDitRHswro2KnZh97W7u8VEQcXGVspCBQNQBCwEBQUs1N/nun7f5667JvbsPHuf8z5/6Jo1a9asec3szTnPrHnmH//gBwEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBB
"text/plain": [
"<PIL.PpmImagePlugin.PpmImageFile image mode=RGB size=2000x1125>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCARlB9ADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD5/ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK
"image/png": "iVBORw0KGgoAAAANSUhEUgAAB9AAAARlCAIAAAButct1AAEAAElEQVR4Aez9dYAcxRqwfT9A4GAhwSEEl+BJsAS34BLk4O4c3CE4BDu4uwZ3d5fgBBJcQvAAwd35rof6nn77tE3P7OzO7O61f0BPdXV19a97JjN3V9/1f/6PfwoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIA
"text/plain": [
"<PIL.PpmImagePlugin.PpmImageFile image mode=RGB size=2000x1125>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCARlB9ADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD5/ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK
"image/png": "iVBORw0KGgoAAAANSUhEUgAAB9AAAARlCAIAAAButct1AAEAAElEQVR4AezdB2AkZd3H8We2pCeX670DV+m9N+lFUEABpUlRmiDYUBApIgqIoLwoIipKkaLSpAtK7+0OjnqV6zV927y/2b3b7O3Mbjab3WSTfMcYZp555plnPjObm/3PM89jDBMCCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIA
"text/plain": [
"<PIL.PpmImagePlugin.PpmImageFile image mode=RGB size=2000x1125>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCARlB9ADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD5/ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK
"image/png": "iVBORw0KGgoAAAANSUhEUgAAB9AAAARlCAIAAAButct1AAEAAElEQVR4AezdB5jU1P648Z+C2EUUxQqCICoqIiJgAxGwYcHeC4qKir3ea++994q9KzYUsSCKBRURuyCiCIq99/J/r7n3/GOSmZ2dnd2dnX338eFmTk5OTj7JZO9+z8k3//d//iiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgoooIACCiiggAIKKKCAAgo
"text/plain": [
"<PIL.PpmImagePlugin.PpmImageFile image mode=RGB size=2000x1125>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCARlB9ADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD5/ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAorofDXgbxJ4ul26Npc08QOGuGGyJfXLnAz7dfavWNI/ZrunQPrPiGKJu8VnCX/8eYj/ANBoA8Gor6eT9nDwiEUPqetlsckTRAE/Ty6X/hnHwf8A9BLXP+/8P/xqgD5gor6f/wCGcfB//QS1z/v/AA//ABqj/hnHwf8A9BLXP+/8P/xqgD5gor6f/wCGcfB//QS1z/v/AA//ABqj/hnHwf8A9BLXP+/8P/xqgD5gor6f/wCGcfB//QS1z/v/AA//ABqj/hnHwf8A9BLXP+/8P/xqgD5gor6f/wCGcfB//QS1z/v/AA//ABqj/h
"image/png": "iVBORw0KGgoAAAANSUhEUgAAB9AAAARlCAIAAAButct1AAEAAElEQVR4Aey9d6BtNdlv/YrYQFBRmgKiolIEEaUdUEBRBEWagocmIF1A6b13ASnSBAEphyK9K6AgUkR6kSIgVaWooIAV9RuX3C83b2ZZc9W99tpj/3FOVmaSmYyZWfLLkyf/8z/+SUACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCUhAAhKQgAQkIAEJSEACEpCABCQgAQlIQAISkIAEJCABCXRG4HWdZTOXBCQgAQlIQAISkIAEJCABCUhAAhIYdwTmnHPOcVdnKywBCUhgYhJ45ZVXnn/++XHXdgX3cXfJrLAEJCABCUhAAsNIYOqpp55nnnnmn3/+ueaaa+655548efIw1tI6SUACEpDABCBwzTXXIE9cdtllTzzxxK9//evxKFX05CpNO+20Cy644Mc+9rFJkybNN998CyywQE+KtRAJSEACEhg8gfBq4412yy23XH311UP+alNwH3wP8YwSkIAEJCABCYwOART2pZZaaqWVVlp22WVHp1W2RAISkIAERojAM88888PX/hApXn311RFqWXlT0Nl5NW+99da+mssBGSsBCUhg/BPg1XbcccddeOGF99577xC2RsF9CC+KVZKABCQgAQlIYNgJzDjjjGusscauu+46yyyzDHtdrZ8EJCABCUjg/ydw1FFHob3feOON/3/ESP3P2/nII490kdlIXVQbIwEJSKAVgbXWWotX21DNKCu4t7poHpeABCQgAQlIQAIJARy/HnDAAQ7mEyQGJSABCUhgnBHAMHDbbbcdNnmiG4hYtdOivffeu5tCzCsBCUhAAuOUwLC91xTcx2lHstoSkIAEJCABCQyawBJLLHHsscfqAXbQ3D2fBCQgAQn0h8CwyROdtZI9VFZfffUpU6Z0lt1cEpCABCQwMgR4r335y18ehlVcCu4j06lsiAQkIAEJSEAC/SKA3dyJJ56oVXu/+FquBCQ
"text/plain": [
"<PIL.PpmImagePlugin.PpmImageFile image mode=RGB size=2000x1125>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCARlB9ADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD5/ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK
"image/png": "iVBORw0KGgoAAAANSUhEUgAAB9AAAARlCAIAAAButct1AAEAAElEQVR4AezdBbwc1dnHcZLcuLu7u7sRSAgJCa7BrdDipRQvBQq05S0Up0CLE9wSiBB3d3d3d8/7zz3JYZiZ3bt67+7e320+6eyZM2fOfEcueebsc846ix8EEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQ
"text/plain": [
"<PIL.PpmImagePlugin.PpmImageFile image mode=RGB size=2000x1125>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"for img in images:\n",
" display(img)"
]
},
{
"cell_type": "markdown",
"id": "fee39ce0",
"metadata": {},
"source": [
"### Image analysis with GPT-4V\n",
"\n",
"After converting a PDF file to multiple images, we'll use GPT-4V to analyze the content based on the images."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "66584c56",
"metadata": {},
"outputs": [],
"source": [
"# Initializing OpenAI client - see https://platform.openai.com/docs/quickstart?context=python\n",
"client = OpenAI()"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "eca3a581",
"metadata": {},
"outputs": [],
"source": [
"# Converting images to base64 encoded images in a data URI format to use with the ChatCompletions API\n",
"def get_img_uri(img):\n",
" buffer = BytesIO()\n",
" img.save(buffer, format=\"jpeg\")\n",
" base64_image = base64.b64encode(buffer.getvalue()).decode(\"utf-8\")\n",
" data_uri = f\"data:image/jpeg;base64,{base64_image}\"\n",
" return data_uri"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "facac9e1",
"metadata": {},
"outputs": [],
"source": [
"system_prompt = '''\n",
"You will be provided with an image of a pdf page or a slide. Your goal is to talk about the content that you see, in technical terms, as if you were delivering a presentation.\n",
"\n",
"If there are diagrams, describe the diagrams and explain their meaning.\n",
"For example: if there is a diagram describing a process flow, say something like \"the process flow starts with X then we have Y and Z...\"\n",
"\n",
"If there are tables, describe logically the content in the tables\n",
"For example: if there is a table listing items and prices, say something like \"the prices are the following: A for X, B for Y...\"\n",
"\n",
"DO NOT include terms referring to the content format\n",
"DO NOT mention the content type - DO focus on the content itself\n",
"For example: if there is a diagram/chart and text on the image, talk about both without mentioning that one is a chart and the other is text.\n",
"Simply describe what you see in the diagram and what you understand from the text.\n",
"\n",
"You should keep it concise, but keep in mind your audience cannot see the image so be exhaustive in describing the content.\n",
"\n",
"Exclude elements that are not relevant to the content:\n",
"DO NOT mention page numbers or the position of the elements on the image.\n",
"\n",
"------\n",
"\n",
"If there is an identifiable title, identify the title to give the output in the following format:\n",
"\n",
"{TITLE}\n",
"\n",
"{Content description}\n",
"\n",
"If there is no clear title, simply return the content description.\n",
"\n",
"'''\n",
"\n",
"def analyze_image(img_url):\n",
" response = client.chat.completions.create(\n",
" model=\"gpt-4-vision-preview\",\n",
" temperature=0,\n",
" messages=[\n",
" {\n",
" \"role\": \"system\",\n",
" \"content\": system_prompt\n",
" },\n",
" {\n",
" \"role\": \"user\",\n",
" \"content\": [\n",
" {\n",
" \"type\": \"image_url\",\n",
" \"image_url\": img_url,\n",
" },\n",
" ],\n",
" }\n",
" ],\n",
" max_tokens=300,\n",
" top_p=0.1\n",
" )\n",
"\n",
" return response.choices[0].message.content"
]
},
{
"cell_type": "markdown",
"id": "eb7924e7",
"metadata": {},
"source": [
"#### Testing with an example"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "f633b375",
"metadata": {},
"outputs": [],
"source": [
"img = images[2]\n",
"data_uri = get_img_uri(img)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "eb2af6a9",
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"What is Fine-tuning\n",
"\n",
"Fine-tuning a model consists of training the model to follow a set of given input/output examples. This will teach the model to behave in a certain way when confronted with a similar input in the future.\n",
"\n",
"We recommend using 50-100 examples even if the minimum is 10.\n",
"\n",
"The process involves starting with a public model, using training data to train the model, and resulting in a fine-tuned model.\n"
]
}
],
"source": [
"res = analyze_image(data_uri)\n",
"print(res)"
]
},
{
"cell_type": "markdown",
"id": "14f38c8c",
"metadata": {},
"source": [
"#### Processing all documents"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "1e6caabd",
"metadata": {},
"outputs": [],
"source": [
"files_path = \"data/example_pdfs\"\n",
"\n",
"all_items = os.listdir(files_path)\n",
"files = [item for item in all_items if os.path.isfile(os.path.join(files_path, item))]"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "eef714d9",
"metadata": {},
"outputs": [],
"source": [
"def analyze_doc_image(img):\n",
" img_uri = get_img_uri(img)\n",
" data = analyze_image(img_uri)\n",
" return data"
]
},
{
"cell_type": "markdown",
"id": "4ed51467",
"metadata": {},
"source": [
"We will list all files in the example folder and process them by \n",
"1. Extracting the text\n",
"2. Converting the docs to images\n",
"3. Analyzing pages with GPT-4V\n",
"\n",
"Note: This takes about ~2 mins to run. Feel free to skip and load directly the result file (see below)."
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "e4659b6e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Analyzing pages for doc rag-deck.pdf\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████████████████████████████████████████████████████████████| 19/19 [00:32<00:00, 1.72s/it]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Analyzing pages for doc models-page.pdf\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████████| 9/9 [00:25<00:00, 2.80s/it]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Analyzing pages for doc evals-decks.pdf\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████████████████████████████████████████████████████████████| 12/12 [00:29<00:00, 2.44s/it]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Analyzing pages for doc fine-tuning-deck.pdf\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████████| 6/6 [00:19<00:00, 3.32s/it]\n"
]
}
],
"source": [
"docs = []\n",
"\n",
"for f in files:\n",
" \n",
" path = f\"{files_path}/{f}\"\n",
" doc = {\n",
" \"filename\": f\n",
" }\n",
" text = extract_text_from_doc(path)\n",
" doc['text'] = text\n",
" imgs = convert_doc_to_images(path)\n",
" pages_description = []\n",
" \n",
" print(f\"Analyzing pages for doc {f}\")\n",
" \n",
" # Concurrent execution\n",
" with concurrent.futures.ThreadPoolExecutor(max_workers=8) as executor:\n",
" \n",
" # Removing 1st slide as it's usually just an intro\n",
" futures = [\n",
" executor.submit(analyze_doc_image, img)\n",
" for img in imgs[1:]\n",
" ]\n",
" \n",
" with tqdm(total=len(imgs)-1) as pbar:\n",
" for _ in concurrent.futures.as_completed(futures):\n",
" pbar.update(1)\n",
" \n",
" for f in futures:\n",
" res = f.result()\n",
" pages_description.append(res)\n",
" \n",
" doc['pages_description'] = pages_description\n",
" docs.append(doc)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "457e74cd",
"metadata": {},
"outputs": [],
"source": [
"# Saving result to file for later\n",
"json_path = \"data/parsed_pdf_docs.json\"\n",
"\n",
"with open(json_path, 'w') as f:\n",
" json.dump(docs, f)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "3d5bbdd7",
"metadata": {},
"outputs": [],
"source": [
"# Optional: load content from the saved file\n",
"with open(json_path, 'r') as f:\n",
" docs = json.load(f)"
]
},
{
"cell_type": "markdown",
"id": "b64ec4ca",
"metadata": {},
"source": [
"### Embedding content\n",
"Before embedding the content, we will chunk it logically by page.\n",
"For real-world scenarios, you could explore more advanced ways to chunk the content:\n",
"- Cutting it into smaller pieces\n",
"- Adding data - such as the slide title, deck title and/or the doc description - at the beginning of each piece of content. That way, each independent chunk can be in context\n",
"\n",
"For the sake of brevity, we will use a very simple chunking strategy and rely on separators to split the text by page."
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "2e8b57bd",
"metadata": {},
"outputs": [],
"source": [
"# Chunking content by page and merging together slides text & description if applicable\n",
"content = []\n",
"for doc in docs:\n",
" # Removing first slide as well\n",
" text = doc['text'].split('\\f')[1:]\n",
" description = doc['pages_description']\n",
" description_indexes = []\n",
" for i in range(len(text)):\n",
" slide_content = text[i] + '\\n'\n",
" # Trying to find matching slide description\n",
" slide_title = text[i].split('\\n')[0]\n",
" for j in range(len(description)):\n",
" description_title = description[j].split('\\n')[0]\n",
" if slide_title.lower() == description_title.lower():\n",
" slide_content += description[j].replace(description_title, '')\n",
" # Keeping track of the descriptions added\n",
" description_indexes.append(j)\n",
" # Adding the slide content + matching slide description to the content pieces\n",
" content.append(slide_content) \n",
" # Adding the slides descriptions that weren't used\n",
" for j in range(len(description)):\n",
" if j not in description_indexes:\n",
" content.append(description[j])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "36af40f8",
"metadata": {},
"outputs": [],
"source": [
"for c in content:\n",
" print(c)\n",
" print(\"\\n\\n-------------------------------\\n\\n\")"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "66dc8272",
"metadata": {},
"outputs": [],
"source": [
"# Cleaning up content\n",
"# Removing trailing spaces, additional line breaks, page numbers and references to the content being a slide\n",
"clean_content = []\n",
"for c in content:\n",
" text = c.replace(' \\n', '').replace('\\n\\n', '\\n').replace('\\n\\n\\n', '\\n').strip()\n",
" text = re.sub(r\"(?<=\\n)\\d{1,2}\", \"\", text)\n",
" text = re.sub(r\"\\b(?:the|this)\\s*slide\\s*\\w+\\b\", \"\", text, flags=re.IGNORECASE)\n",
" clean_content.append(text)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ed28dbb5",
"metadata": {},
"outputs": [],
"source": [
"for c in clean_content:\n",
" print(c)\n",
" print(\"\\n\\n-------------------------------\\n\\n\")"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "763ae4bd",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(64, 1)\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>content</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>Overview\\nRetrieval-Augmented Generationenhanc...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>What is RAG\\nRetrieve information to Augment t...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>When to use RAG\\nGood for ✅\\nNot good for ❌\\...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Technical patterns\\nData preparation\\nInput pr...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>Technical patterns\\nData preparation\\nchunk do...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" content\n",
"0 Overview\\nRetrieval-Augmented Generationenhanc...\n",
"1 What is RAG\\nRetrieve information to Augment t...\n",
"2 When to use RAG\\nGood for ✅\\nNot good for ❌\\...\n",
"3 Technical patterns\\nData preparation\\nInput pr...\n",
"4 Technical patterns\\nData preparation\\nchunk do..."
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Creating the embeddings\n",
"# We'll save to a csv file here for testing purposes but this is where you should load content in your vectorDB.\n",
"df = pd.DataFrame(clean_content, columns=['content'])\n",
"print(df.shape)\n",
"df.head()"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "c6a47f32",
"metadata": {},
"outputs": [],
"source": [
"embeddings_model = \"text-embedding-3-large\"\n",
"\n",
"def get_embeddings(text):\n",
" embeddings = client.embeddings.create(\n",
" model=\"text-embedding-3-small\",\n",
" input=text,\n",
" encoding_format=\"float\"\n",
" )\n",
" return embeddings.data[0].embedding"
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "8d22b06e",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>content</th>\n",
" <th>embeddings</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>Overview\\nRetrieval-Augmented Generationenhanc...</td>\n",
" <td>[-0.014744381, 0.03017278, 0.06353764, 0.02110...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>What is RAG\\nRetrieve information to Augment t...</td>\n",
" <td>[-0.024337867, 0.022921458, -0.00971687, 0.010...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>When to use RAG\\nGood for ✅\\nNot good for ❌\\...</td>\n",
" <td>[-0.011084231, 0.021158217, -0.00430421, 0.017...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Technical patterns\\nData preparation\\nInput pr...</td>\n",
" <td>[-0.0058343858, 0.0408407, 0.054318383, 0.0190...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>Technical patterns\\nData preparation\\nchunk do...</td>\n",
" <td>[-0.010359385, 0.03736894, 0.052995477, 0.0180...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" content \\\n",
"0 Overview\\nRetrieval-Augmented Generationenhanc... \n",
"1 What is RAG\\nRetrieve information to Augment t... \n",
"2 When to use RAG\\nGood for ✅\\nNot good for ❌\\... \n",
"3 Technical patterns\\nData preparation\\nInput pr... \n",
"4 Technical patterns\\nData preparation\\nchunk do... \n",
"\n",
" embeddings \n",
"0 [-0.014744381, 0.03017278, 0.06353764, 0.02110... \n",
"1 [-0.024337867, 0.022921458, -0.00971687, 0.010... \n",
"2 [-0.011084231, 0.021158217, -0.00430421, 0.017... \n",
"3 [-0.0058343858, 0.0408407, 0.054318383, 0.0190... \n",
"4 [-0.010359385, 0.03736894, 0.052995477, 0.0180... "
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['embeddings'] = df['content'].apply(lambda x: get_embeddings(x))\n",
"df.head()"
]
},
{
"cell_type": "code",
"execution_count": 23,
"id": "c0e9a5d3",
"metadata": {},
"outputs": [],
"source": [
"# Saving locally for later\n",
"data_path = \"data/parsed_pdf_docs_with_embeddings.csv\"\n",
"df.to_csv(data_path, index=False)"
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "9c1c27ff",
"metadata": {},
"outputs": [],
"source": [
"# Optional: load data from saved file\n",
"df = pd.read_csv(data_path)\n",
"df[\"embeddings\"] = df.embeddings.apply(literal_eval).apply(np.array)"
]
},
{
"cell_type": "markdown",
"id": "aafa06b6",
"metadata": {},
"source": [
"## Retrieval-augmented generation\n",
"\n",
"The last step of the process is to generate outputs in response to input queries, after retrieving content as context to reply."
]
},
{
"cell_type": "code",
"execution_count": 35,
"id": "27753ec2",
"metadata": {},
"outputs": [],
"source": [
"system_prompt = '''\n",
" You will be provided with an input prompt and content as context that can be used to reply to the prompt.\n",
" \n",
" You will do 2 things:\n",
" \n",
" 1. First, you will internally assess whether the content provided is relevant to reply to the input prompt. \n",
" \n",
" 2a. If that is the case, answer directly using this content. If the content is relevant, use elements found in the content to craft a reply to the input prompt.\n",
"\n",
" 2b. If the content is not relevant, use your own knowledge to reply or say that you don't know how to respond if your knowledge is not sufficient to answer.\n",
" \n",
" Stay concise with your answer, replying specifically to the input prompt without mentioning additional information provided in the context content.\n",
"'''\n",
"\n",
"model=\"gpt-4-turbo-preview\"\n",
"\n",
"def search_content(df, input_text, top_k):\n",
" embedded_value = get_embeddings(input_text)\n",
" df[\"similarity\"] = df.embeddings.apply(lambda x: cosine_similarity(np.array(x).reshape(1,-1), np.array(embedded_value).reshape(1, -1)))\n",
" res = df.sort_values('similarity', ascending=False).head(top_k)\n",
" return res\n",
"\n",
"def get_similarity(row):\n",
" similarity_score = row['similarity']\n",
" if isinstance(similarity_score, np.ndarray):\n",
" similarity_score = similarity_score[0][0]\n",
" return similarity_score\n",
"\n",
"def generate_output(input_prompt, similar_content, threshold = 0.5):\n",
" \n",
" content = similar_content.iloc[0]['content']\n",
" \n",
" # Adding more matching content if the similarity is above threshold\n",
" if len(similar_content) > 1:\n",
" for i, row in similar_content.iterrows():\n",
" similarity_score = get_similarity(row)\n",
" if similarity_score > threshold:\n",
" content += f\"\\n\\n{row['content']}\"\n",
" \n",
" prompt = f\"INPUT PROMPT:\\n{input_prompt}\\n-------\\nCONTENT:\\n{content}\"\n",
" \n",
" completion = client.chat.completions.create(\n",
" model=model,\n",
" temperature=0.5,\n",
" messages=[\n",
" {\n",
" \"role\": \"system\",\n",
" \"content\": system_prompt\n",
" },\n",
" {\n",
" \"role\": \"user\",\n",
" \"content\": prompt\n",
" }\n",
" ]\n",
" )\n",
"\n",
" return completion.choices[0].message.content"
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "5c1173f5",
"metadata": {},
"outputs": [],
"source": [
"# Example user queries related to the content\n",
"example_inputs = [\n",
" 'What are the main models you offer?',\n",
" 'Do you have a speech recognition model?',\n",
" 'Which embedding model should I use for non-English use cases?',\n",
" 'Can I introduce new knowledge in my LLM app using RAG?',\n",
" 'How many examples do I need to fine-tune a model?',\n",
" 'Which metric can I use to evaluate a summarization task?',\n",
" 'Give me a detailed example for an evaluation process where we are looking for a clear answer to compare to a ground truth.',\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": 45,
"id": "304cb806",
"metadata": {
"scrolled": false
},
"outputs": [
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #af005f; text-decoration-color: #af005f; font-weight: bold\">QUERY:</span><span style=\"color: #af005f; text-decoration-color: #af005f\"> What are the main models you offer?</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;125mQUERY:\u001b[0m\u001b[38;5;125m What are the main models you offer?\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">Matching content:</span>\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59mMatching content:\u001b[0m\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.43</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.43\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Models - OpenAI API</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">The content lists various API endpoints and their corresponding latest models:</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">-...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mModels - OpenAI API\u001b[0m\n",
"\u001b[38;5;59mThe content lists various API endpoints and their corresponding latest models:\u001b[0m\n",
"\u001b[38;5;59m-\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.39</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.39\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">26</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">02</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">2024</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">, </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">17:58</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Models - OpenAI API</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">The Moderation models are designed to check whether content co...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59m26\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m02\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m2024\u001b[0m\u001b[38;5;59m, \u001b[0m\u001b[1;38;5;59m17:58\u001b[0m\n",
"\u001b[38;5;59mModels - OpenAI API\u001b[0m\n",
"\u001b[38;5;59mThe Moderation models are designed to check whether content co\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.39</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.39\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">The content describes various models provided by OpenAI, focusing on moderation models and GPT base ...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mThe content describes various models provided by OpenAI, focusing on moderation models and GPT base \u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #008787; text-decoration-color: #008787; font-weight: bold\">REPLY:</span>\n",
"\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">The main models we offer include:</span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">- For completions: gpt-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">3.5</span><span style=\"color: #00875f; text-decoration-color: #00875f\">-turbo-instruct, babbage-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">002</span><span style=\"color: #00875f; text-decoration-color: #00875f\">, and davinci-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">002</span><span style=\"color: #00875f; text-decoration-color: #00875f\">.</span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">- For embeddings: text-embedding-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">3</span><span style=\"color: #00875f; text-decoration-color: #00875f\">-small, text-embedding-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">3</span><span style=\"color: #00875f; text-decoration-color: #00875f\">-large, and text-embedding-ada-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">002</span><span style=\"color: #00875f; text-decoration-color: #00875f\">.</span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">- For fine-tuning jobs: gpt-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">3.5</span><span style=\"color: #00875f; text-decoration-color: #00875f\">-turbo, babbage-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">002</span><span style=\"color: #00875f; text-decoration-color: #00875f\">, and davinci-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">002</span><span style=\"color: #00875f; text-decoration-color: #00875f\">.</span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">- For moderations: text-moderation-stable and text-moderation.</span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">Additionally, we have the latest models like gpt-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">3.5</span><span style=\"color: #00875f; text-decoration-color: #00875f\">-turbo-16k and fine-tuned versions of gpt-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">3.5</span><span style=\"color: #00875f; text-decoration-color: #00875f\">-turbo.</span>\n",
"\n",
"--------------\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;30mREPLY:\u001b[0m\n",
"\n",
"\u001b[38;5;29mThe main models we offer include:\u001b[0m\n",
"\u001b[38;5;29m- For completions: gpt-\u001b[0m\u001b[1;38;5;29m3.5\u001b[0m\u001b[38;5;29m-turbo-instruct, babbage-\u001b[0m\u001b[1;38;5;29m002\u001b[0m\u001b[38;5;29m, and davinci-\u001b[0m\u001b[1;38;5;29m002\u001b[0m\u001b[38;5;29m.\u001b[0m\n",
"\u001b[38;5;29m- For embeddings: text-embedding-\u001b[0m\u001b[1;38;5;29m3\u001b[0m\u001b[38;5;29m-small, text-embedding-\u001b[0m\u001b[1;38;5;29m3\u001b[0m\u001b[38;5;29m-large, and text-embedding-ada-\u001b[0m\u001b[1;38;5;29m002\u001b[0m\u001b[38;5;29m.\u001b[0m\n",
"\u001b[38;5;29m- For fine-tuning jobs: gpt-\u001b[0m\u001b[1;38;5;29m3.5\u001b[0m\u001b[38;5;29m-turbo, babbage-\u001b[0m\u001b[1;38;5;29m002\u001b[0m\u001b[38;5;29m, and davinci-\u001b[0m\u001b[1;38;5;29m002\u001b[0m\u001b[38;5;29m.\u001b[0m\n",
"\u001b[38;5;29m- For moderations: text-moderation-stable and text-moderation.\u001b[0m\n",
"\u001b[38;5;29mAdditionally, we have the latest models like gpt-\u001b[0m\u001b[1;38;5;29m3.5\u001b[0m\u001b[38;5;29m-turbo-16k and fine-tuned versions of gpt-\u001b[0m\u001b[1;38;5;29m3.5\u001b[0m\u001b[38;5;29m-turbo.\u001b[0m\n",
"\n",
"--------------\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #af005f; text-decoration-color: #af005f; font-weight: bold\">QUERY:</span><span style=\"color: #af005f; text-decoration-color: #af005f\"> Do you have a speech recognition model?</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;125mQUERY:\u001b[0m\u001b[38;5;125m Do you have a speech recognition model?\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">Matching content:</span>\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59mMatching content:\u001b[0m\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.53</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.53\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">The content describes various models related to text-to-speech, speech recognition, embeddings, and ...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mThe content describes various models related to text-to-speech, speech recognition, embeddings, and \u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.50</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.50\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">26</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">02</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">2024</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">, </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">17:58</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Models - OpenAI API</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">MODEL</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">DE S CRIPTION</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">tts-</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">1</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">New Text-to-speech </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">1</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">The latest tex...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59m26\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m02\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m2024\u001b[0m\u001b[38;5;59m, \u001b[0m\u001b[1;38;5;59m17:58\u001b[0m\n",
"\u001b[38;5;59mModels - OpenAI API\u001b[0m\n",
"\u001b[38;5;59mMODEL\u001b[0m\n",
"\u001b[38;5;59mDE S CRIPTION\u001b[0m\n",
"\u001b[38;5;59mtts-\u001b[0m\u001b[1;38;5;59m1\u001b[0m\n",
"\u001b[38;5;59mNew Text-to-speech \u001b[0m\u001b[1;38;5;59m1\u001b[0m\n",
"\u001b[38;5;59mThe latest tex\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.44</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.44\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">26</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">02</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">2024</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">, </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">17:58</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Models - OpenAI API</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">ENDP OINT</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">DATA USED</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">FOR TRAINING</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">DEFAULT</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">RETENTION</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">ELIGIBLE FO...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59m26\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m02\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m2024\u001b[0m\u001b[38;5;59m, \u001b[0m\u001b[1;38;5;59m17:58\u001b[0m\n",
"\u001b[38;5;59mModels - OpenAI API\u001b[0m\n",
"\u001b[38;5;59mENDP OINT\u001b[0m\n",
"\u001b[38;5;59mDATA USED\u001b[0m\n",
"\u001b[38;5;59mFOR TRAINING\u001b[0m\n",
"\u001b[38;5;59mDEFAULT\u001b[0m\n",
"\u001b[38;5;59mRETENTION\u001b[0m\n",
"\u001b[38;5;59mELIGIBLE FO\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #008787; text-decoration-color: #008787; font-weight: bold\">REPLY:</span>\n",
"\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">Yes, the Whisper model is a general-purpose speech recognition model mentioned in the content, capable of </span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">multilingual speech recognition, speech translation, and language identification. The v2-large model, referred to </span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">as </span><span style=\"color: #00875f; text-decoration-color: #00875f\">\"whisper-1\"</span><span style=\"color: #00875f; text-decoration-color: #00875f\">, is available through an API and is optimized for faster performance.</span>\n",
"\n",
"--------------\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;30mREPLY:\u001b[0m\n",
"\n",
"\u001b[38;5;29mYes, the Whisper model is a general-purpose speech recognition model mentioned in the content, capable of \u001b[0m\n",
"\u001b[38;5;29mmultilingual speech recognition, speech translation, and language identification. The v2-large model, referred to \u001b[0m\n",
"\u001b[38;5;29mas \u001b[0m\u001b[38;5;29m\"whisper-1\"\u001b[0m\u001b[38;5;29m, is available through an API and is optimized for faster performance.\u001b[0m\n",
"\n",
"--------------\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #af005f; text-decoration-color: #af005f; font-weight: bold\">QUERY:</span><span style=\"color: #af005f; text-decoration-color: #af005f\"> Which embedding model should I use for non-English use cases?</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;125mQUERY:\u001b[0m\u001b[38;5;125m Which embedding model should I use for non-English use cases?\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">Matching content:</span>\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59mMatching content:\u001b[0m\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.57</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.57\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">The content describes various models related to text-to-speech, speech recognition, embeddings, and ...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mThe content describes various models related to text-to-speech, speech recognition, embeddings, and \u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.46</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.46\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">26</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">02</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">2024</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">, </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">17:58</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Models - OpenAI API</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">MODEL</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">DE S CRIPTION</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">tts-</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">1</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">New Text-to-speech </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">1</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">The latest tex...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59m26\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m02\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m2024\u001b[0m\u001b[38;5;59m, \u001b[0m\u001b[1;38;5;59m17:58\u001b[0m\n",
"\u001b[38;5;59mModels - OpenAI API\u001b[0m\n",
"\u001b[38;5;59mMODEL\u001b[0m\n",
"\u001b[38;5;59mDE S CRIPTION\u001b[0m\n",
"\u001b[38;5;59mtts-\u001b[0m\u001b[1;38;5;59m1\u001b[0m\n",
"\u001b[38;5;59mNew Text-to-speech \u001b[0m\u001b[1;38;5;59m1\u001b[0m\n",
"\u001b[38;5;59mThe latest tex\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.40</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.40\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">26</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">02</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">2024</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">, </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">17:58</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Models - OpenAI API</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Multilingual capabilities</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">GPT-</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">4</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\"> outperforms both previous larg...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59m26\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m02\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m2024\u001b[0m\u001b[38;5;59m, \u001b[0m\u001b[1;38;5;59m17:58\u001b[0m\n",
"\u001b[38;5;59mModels - OpenAI API\u001b[0m\n",
"\u001b[38;5;59mMultilingual capabilities\u001b[0m\n",
"\u001b[38;5;59mGPT-\u001b[0m\u001b[1;38;5;59m4\u001b[0m\u001b[38;5;59m outperforms both previous larg\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #008787; text-decoration-color: #008787; font-weight: bold\">REPLY:</span>\n",
"\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">For non-English use cases, you should use the </span><span style=\"color: #00875f; text-decoration-color: #00875f\">\"V3 large\"</span><span style=\"color: #00875f; text-decoration-color: #00875f\"> embedding model, as it is described as the most capable </span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">for both English and non-English tasks, with an output dimension of </span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">3</span><span style=\"color: #00875f; text-decoration-color: #00875f\">,</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">072</span><span style=\"color: #00875f; text-decoration-color: #00875f\">.</span>\n",
"\n",
"--------------\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;30mREPLY:\u001b[0m\n",
"\n",
"\u001b[38;5;29mFor non-English use cases, you should use the \u001b[0m\u001b[38;5;29m\"V3 large\"\u001b[0m\u001b[38;5;29m embedding model, as it is described as the most capable \u001b[0m\n",
"\u001b[38;5;29mfor both English and non-English tasks, with an output dimension of \u001b[0m\u001b[1;38;5;29m3\u001b[0m\u001b[38;5;29m,\u001b[0m\u001b[1;38;5;29m072\u001b[0m\u001b[38;5;29m.\u001b[0m\n",
"\n",
"--------------\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #af005f; text-decoration-color: #af005f; font-weight: bold\">QUERY:</span><span style=\"color: #af005f; text-decoration-color: #af005f\"> Can I introduce new knowledge in my LLM app using RAG?</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;125mQUERY:\u001b[0m\u001b[38;5;125m Can I introduce new knowledge in my LLM app using RAG?\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">Matching content:</span>\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59mMatching content:\u001b[0m\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.50</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.50\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">What is RAG</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Retrieve information to Augment the models knowledge and Generate the output</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">“What is y...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mWhat is RAG\u001b[0m\n",
"\u001b[38;5;59mRetrieve information to Augment the models knowledge and Generate the output\u001b[0m\n",
"\u001b[38;5;59m“What is y\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.49</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.49\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">When to use RAG</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Good for ✅</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Not good for ❌</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">●</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">●</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Introducing new information to the model</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">●</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Teaching ...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mWhen to use RAG\u001b[0m\n",
"\u001b[38;5;59mGood for ✅\u001b[0m\n",
"\u001b[38;5;59mNot good for ❌\u001b[0m\n",
"\u001b[38;5;59m●\u001b[0m\n",
"\u001b[38;5;59m●\u001b[0m\n",
"\u001b[38;5;59mIntroducing new information to the model\u001b[0m\n",
"\u001b[38;5;59m●\u001b[0m\n",
"\u001b[38;5;59mTeaching \u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.43</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.43\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Technical patterns</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Data preparation: augmenting content</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">What does “Augmentingcontent” mean?</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Augmenti...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mTechnical patterns\u001b[0m\n",
"\u001b[38;5;59mData preparation: augmenting content\u001b[0m\n",
"\u001b[38;5;59mWhat does “Augmentingcontent” mean?\u001b[0m\n",
"\u001b[38;5;59mAugmenti\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #008787; text-decoration-color: #008787; font-weight: bold\">REPLY:</span>\n",
"\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">Yes, you can introduce new knowledge in your LLM app using RAG by retrieving information from a knowledge base or </span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">external sources to augment the model's knowledge and generate outputs relevant to the queries posed.</span>\n",
"\n",
"--------------\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;30mREPLY:\u001b[0m\n",
"\n",
"\u001b[38;5;29mYes, you can introduce new knowledge in your LLM app using RAG by retrieving information from a knowledge base or \u001b[0m\n",
"\u001b[38;5;29mexternal sources to augment the model's knowledge and generate outputs relevant to the queries posed.\u001b[0m\n",
"\n",
"--------------\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #af005f; text-decoration-color: #af005f; font-weight: bold\">QUERY:</span><span style=\"color: #af005f; text-decoration-color: #af005f\"> How many examples do I need to fine-tune a model?</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;125mQUERY:\u001b[0m\u001b[38;5;125m How many examples do I need to fine-tune a model?\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">Matching content:</span>\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59mMatching content:\u001b[0m\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.68</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.68\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">What is Fine-tuning</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Public Model</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Training data</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Training</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Fine-tunedmodel</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Fine-tuning a model consists...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mWhat is Fine-tuning\u001b[0m\n",
"\u001b[38;5;59mPublic Model\u001b[0m\n",
"\u001b[38;5;59mTraining data\u001b[0m\n",
"\u001b[38;5;59mTraining\u001b[0m\n",
"\u001b[38;5;59mFine-tunedmodel\u001b[0m\n",
"\u001b[38;5;59mFine-tuning a model consists\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.62</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.62\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">When to fine-tune</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Fine-tuning is good for:</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">- Following a given format or tone for the output</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">- Proce...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mWhen to fine-tune\u001b[0m\n",
"\u001b[38;5;59mFine-tuning is good for:\u001b[0m\n",
"\u001b[38;5;59m- Following a given format or tone for the output\u001b[0m\n",
"\u001b[38;5;59m- Proce\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.57</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.57\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Overview</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Fine-tuning involves adjusting theparameters of pre-trained models on aspecific dataset or t...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mOverview\u001b[0m\n",
"\u001b[38;5;59mFine-tuning involves adjusting theparameters of pre-trained models on aspecific dataset or t\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #008787; text-decoration-color: #008787; font-weight: bold\">REPLY:</span>\n",
"\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">We recommend using </span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">50</span><span style=\"color: #00875f; text-decoration-color: #00875f\">-</span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">100</span><span style=\"color: #00875f; text-decoration-color: #00875f\"> examples for fine-tuning a model, even though the minimum is </span><span style=\"color: #00875f; text-decoration-color: #00875f; font-weight: bold\">10</span><span style=\"color: #00875f; text-decoration-color: #00875f\">.</span>\n",
"\n",
"--------------\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;30mREPLY:\u001b[0m\n",
"\n",
"\u001b[38;5;29mWe recommend using \u001b[0m\u001b[1;38;5;29m50\u001b[0m\u001b[38;5;29m-\u001b[0m\u001b[1;38;5;29m100\u001b[0m\u001b[38;5;29m examples for fine-tuning a model, even though the minimum is \u001b[0m\u001b[1;38;5;29m10\u001b[0m\u001b[38;5;29m.\u001b[0m\n",
"\n",
"--------------\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #af005f; text-decoration-color: #af005f; font-weight: bold\">QUERY:</span><span style=\"color: #af005f; text-decoration-color: #af005f\"> Which metric can I use to evaluate a summarization task?</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;125mQUERY:\u001b[0m\u001b[38;5;125m Which metric can I use to evaluate a summarization task?\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">Matching content:</span>\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59mMatching content:\u001b[0m\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.53</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.53\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Technical patterns</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Metric-based evaluations</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">ROUGE is a common metric for evaluating machine summariz...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mTechnical patterns\u001b[0m\n",
"\u001b[38;5;59mMetric-based evaluations\u001b[0m\n",
"\u001b[38;5;59mROUGE is a common metric for evaluating machine summariz\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.49</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.49\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Technical patterns</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Metric-based evaluations</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Component evaluations</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Subjective evaluations</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">●</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">●</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Compari...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mTechnical patterns\u001b[0m\n",
"\u001b[38;5;59mMetric-based evaluations\u001b[0m\n",
"\u001b[38;5;59mComponent evaluations\u001b[0m\n",
"\u001b[38;5;59mSubjective evaluations\u001b[0m\n",
"\u001b[38;5;59m●\u001b[0m\n",
"\u001b[38;5;59m●\u001b[0m\n",
"\u001b[38;5;59mCompari\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.48</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.48\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Technical patterns</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Metric-based evaluations</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">BLEU score is another standard metric, this time focusin...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mTechnical patterns\u001b[0m\n",
"\u001b[38;5;59mMetric-based evaluations\u001b[0m\n",
"\u001b[38;5;59mBLEU score is another standard metric, this time focusin\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #008787; text-decoration-color: #008787; font-weight: bold\">REPLY:</span>\n",
"\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">ROUGE is a common metric you can use to evaluate a summarization task.</span>\n",
"\n",
"--------------\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;30mREPLY:\u001b[0m\n",
"\n",
"\u001b[38;5;29mROUGE is a common metric you can use to evaluate a summarization task.\u001b[0m\n",
"\n",
"--------------\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #af005f; text-decoration-color: #af005f; font-weight: bold\">QUERY:</span><span style=\"color: #af005f; text-decoration-color: #af005f\"> Give me a detailed example for an evaluation process where we are looking for a clear answer to compare to a</span>\n",
"<span style=\"color: #af005f; text-decoration-color: #af005f\">ground truth.</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;125mQUERY:\u001b[0m\u001b[38;5;125m Give me a detailed example for an evaluation process where we are looking for a clear answer to compare to a\u001b[0m\n",
"\u001b[38;5;125mground truth.\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">Matching content:</span>\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;59mMatching content:\u001b[0m\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.60</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.60\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">What are evals</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Example</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Our ground truth matches the predicted answer, so the evaluation passes!</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Eval...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mWhat are evals\u001b[0m\n",
"\u001b[38;5;59mExample\u001b[0m\n",
"\u001b[38;5;59mOur ground truth matches the predicted answer, so the evaluation passes!\u001b[0m\n",
"\u001b[38;5;59mEval\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.59</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.59\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">What are evals</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Example</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">An evaluation contains a question and a correct answer. We call this the grou...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mWhat are evals\u001b[0m\n",
"\u001b[38;5;59mExample\u001b[0m\n",
"\u001b[38;5;59mAn evaluation contains a question and a correct answer. We call this the grou\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-style: italic\">Similarity: </span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold; font-style: italic\">0.50</span>\n",
"</pre>\n"
],
"text/plain": [
"\u001b[3;38;5;59mSimilarity: \u001b[0m\u001b[1;3;38;5;59m0.50\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Technical patterns</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">Metric-based evaluations</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">What theyre good for</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">What to be aware of</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">●</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">●</span>\n",
"<span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">A good sta...</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">[</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f\">/</span><span style=\"color: #5f5f5f; text-decoration-color: #5f5f5f; font-weight: bold\">]</span>\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[38;5;59mTechnical patterns\u001b[0m\n",
"\u001b[38;5;59mMetric-based evaluations\u001b[0m\n",
"\u001b[38;5;59mWhat theyre good for\u001b[0m\n",
"\u001b[38;5;59mWhat to be aware of\u001b[0m\n",
"\u001b[38;5;59m●\u001b[0m\n",
"\u001b[38;5;59m●\u001b[0m\n",
"\u001b[38;5;59mA good sta\u001b[0m\u001b[38;5;59m...\u001b[0m\u001b[1;38;5;59m[\u001b[0m\u001b[38;5;59m/\u001b[0m\u001b[1;38;5;59m]\u001b[0m\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #008787; text-decoration-color: #008787; font-weight: bold\">REPLY:</span>\n",
"\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">The content provided is relevant and offers a detailed example for an evaluation process comparing to a ground </span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">truth. Here's a concise explanation based on the content:</span>\n",
"\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">In the given example, the evaluation process involves a question-and-answer scenario to verify the accuracy of </span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">information retrieved by a tool or system in response to a query. The question posed is, </span><span style=\"color: #00875f; text-decoration-color: #00875f\">\"What is the population of</span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">Canada?\"</span><span style=\"color: #00875f; text-decoration-color: #00875f\"> The ground truth, or the correct answer, is established as </span><span style=\"color: #00875f; text-decoration-color: #00875f\">\"The population of Canada in 2023 is 39,566,248</span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">people.\"</span><span style=\"color: #00875f; text-decoration-color: #00875f\"> A tool labeled </span><span style=\"color: #00875f; text-decoration-color: #00875f\">\"LLM\"</span><span style=\"color: #00875f; text-decoration-color: #00875f\"> is then used to search for the answer, which predicts </span><span style=\"color: #00875f; text-decoration-color: #00875f\">\"The current population of </span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">Canada is 39,566,248 as of Tuesday, May 23, 2023.\"</span><span style=\"color: #00875f; text-decoration-color: #00875f\"> This predicted answer matches the ground truth exactly, </span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">indicating that the evaluation passes. This process demonstrates how an evaluation can be used to verify the </span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">accuracy of information retrieved by a tool, comparing the predicted answer to the ground truth to ensure </span>\n",
"<span style=\"color: #00875f; text-decoration-color: #00875f\">correctness.</span>\n",
"\n",
"--------------\n",
"\n",
"\n",
"</pre>\n"
],
"text/plain": [
"\u001b[1;38;5;30mREPLY:\u001b[0m\n",
"\n",
"\u001b[38;5;29mThe content provided is relevant and offers a detailed example for an evaluation process comparing to a ground \u001b[0m\n",
"\u001b[38;5;29mtruth. Here's a concise explanation based on the content:\u001b[0m\n",
"\n",
"\u001b[38;5;29mIn the given example, the evaluation process involves a question-and-answer scenario to verify the accuracy of \u001b[0m\n",
"\u001b[38;5;29minformation retrieved by a tool or system in response to a query. The question posed is, \u001b[0m\u001b[38;5;29m\"What is the population of\u001b[0m\n",
"\u001b[38;5;29mCanada?\"\u001b[0m\u001b[38;5;29m The ground truth, or the correct answer, is established as \u001b[0m\u001b[38;5;29m\"The population of Canada in 2023 is 39,566,248\u001b[0m\n",
"\u001b[38;5;29mpeople.\"\u001b[0m\u001b[38;5;29m A tool labeled \u001b[0m\u001b[38;5;29m\"LLM\"\u001b[0m\u001b[38;5;29m is then used to search for the answer, which predicts \u001b[0m\u001b[38;5;29m\"The current population of \u001b[0m\n",
"\u001b[38;5;29mCanada is 39,566,248 as of Tuesday, May 23, 2023.\"\u001b[0m\u001b[38;5;29m This predicted answer matches the ground truth exactly, \u001b[0m\n",
"\u001b[38;5;29mindicating that the evaluation passes. This process demonstrates how an evaluation can be used to verify the \u001b[0m\n",
"\u001b[38;5;29maccuracy of information retrieved by a tool, comparing the predicted answer to the ground truth to ensure \u001b[0m\n",
"\u001b[38;5;29mcorrectness.\u001b[0m\n",
"\n",
"--------------\n",
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Running the RAG pipeline on each example\n",
"for ex in example_inputs:\n",
" print(f\"[deep_pink4][bold]QUERY:[/bold] {ex}[/deep_pink4]\\n\\n\")\n",
" matching_content = search_content(df, ex, 3)\n",
" print(f\"[grey37][b]Matching content:[/b][/grey37]\\n\")\n",
" for i, match in matching_content.iterrows():\n",
" print(f\"[grey37][i]Similarity: {get_similarity(match):.2f}[/i][/grey37]\")\n",
" print(f\"[grey37]{match['content'][:100]}{'...' if len(match['content']) > 100 else ''}[/[grey37]]\\n\\n\")\n",
" reply = generate_output(ex, matching_content)\n",
" print(f\"[turquoise4][b]REPLY:[/b][/turquoise4]\\n\\n[spring_green4]{reply}[/spring_green4]\\n\\n--------------\\n\\n\")"
]
},
{
"cell_type": "markdown",
"id": "a4dedf9c",
"metadata": {},
"source": [
"## Wrapping up\n",
"\n",
"In this notebook, we have learned how to develop a basic RAG pipeline based on PDF documents. This includes:\n",
"\n",
"- How to parse pdf documents, taking slide decks and an export from an HTML page as examples, using a python library as well as GPT-4V to interpret the visuals\n",
"- How to process the extracted content, clean it and chunk it into several pieces\n",
"- How to embed the processed content using OpenAI embeddings\n",
"- How to retrieve content that is relevant to an input query\n",
"- How to use GPT-4-turbo to generate an answer using the retrieved content as context\n",
"\n",
"If you want to explore further, consider these optimisations:\n",
"\n",
"- Playing around with the prompts provided as examples\n",
"- Chunking the content further and adding metadata as context to each chunk\n",
"- Adding rule-based filtering on the retrieval results or re-ranking results to surface to most relevant content\n",
"\n",
"You can apply the techniques covered in this notebook to multiple use cases, such as assistants that can access your proprietary data, customer service or FAQ bots that can read from your internal policies, or anything that requires leveraging rich documents that would be better understood as images."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.7"
}
},
"nbformat": 4,
"nbformat_minor": 5
}