Merge branch 'main' into patch-1

articles/how_to_work_with_large_language_models.md

@@ -6,14 +6,14 @@
 
 The magic of large language models is that by being trained to minimize this prediction error over vast quantities of text, the models end up learning concepts useful for these predictions. For example, they learn:
 
-* how to spell
-* how grammar works
-* how to paraphrase
-* how to answer questions
-* how to hold a conversation
-* how to write in many languages
-* how to code
-* etc.
+- how to spell
+- how grammar works
+- how to paraphrase
+- how to answer questions
+- how to hold a conversation
+- how to write in many languages
+- how to code
+- etc.
 
 They do this by “reading” a large amount of existing text and learning how words tend to appear in context with other words, and uses what it has learned to predict the next most likely word that might appear in response to a user request, and each subsequent word after that.
 
@@ -25,12 +25,12 @@ Of all the inputs to a large language model, by far the most influential is the
 
 Large language models can be prompted to produce output in a few ways:
 
-* **Instruction**: Tell the model what you want
-* **Completion**: Induce the model to complete the beginning of what you want
-* **Scenario**: Give the model a situation to play out
-* **Demonstration**: Show the model what you want, with either:
-  * A few examples in the prompt
-  * Many hundreds or thousands of examples in a fine-tuning training dataset
+- **Instruction**: Tell the model what you want
+- **Completion**: Induce the model to complete the beginning of what you want
+- **Scenario**: Give the model a situation to play out
+- **Demonstration**: Show the model what you want, with either:
+  - A few examples in the prompt
+  - Many hundreds or thousands of examples in a fine-tuning training dataset
 
 An example of each is shown below.
 
@@ -77,6 +77,7 @@ Output:
 Giving the model a scenario to follow or role to play out can be helpful for complex queries or when seeking imaginative responses. When using a hypothetical prompt, you set up a situation, problem, or story, and then ask the model to respond as if it were a character in that scenario or an expert on the topic.
 
 Example scenario prompt:
+
 ```text
 Your role is to extract the name of the author from any given text
 
@@ -141,11 +142,11 @@ Large language models aren't only great at text - they can be great at code too.
 
 GPT-4 powers [numerous innovative products][OpenAI Customer Stories], including:
 
-* [GitHub Copilot] (autocompletes code in Visual Studio and other IDEs)
-* [Replit](https://replit.com/) (can complete, explain, edit and generate code)
-* [Cursor](https://cursor.sh/) (build software faster in an editor designed for pair-programming with AI)
+- [GitHub Copilot] (autocompletes code in Visual Studio and other IDEs)
+- [Replit](https://replit.com/) (can complete, explain, edit and generate code)
+- [Cursor](https://cursor.sh/) (build software faster in an editor designed for pair-programming with AI)
 
-GPT-4 is more advanced than previous models like `text-davinci-002`. But, to get the best out of GPT-4 for coding tasks, it's still important to give clear and specific instructions. As a result, designing good prompts can take more care.
+GPT-4 is more advanced than previous models like `gpt-3.5-turbo-instruct`. But, to get the best out of GPT-4 for coding tasks, it's still important to give clear and specific instructions. As a result, designing good prompts can take more care.
 
 ### More prompt advice
 
@@ -153,12 +154,10 @@ For more prompt examples, visit [OpenAI Examples][OpenAI Examples].
 
 In general, the input prompt is the best lever for improving model outputs. You can try tricks like:
 
-* **Be more specific** E.g., if you want the output to be a comma separated list, ask it to return a comma separated list. If you want it to say "I don't know" when it doesn't know the answer, tell it 'Say "I don't know" if you do not know the answer.' The more specific your instructions, the better the model can respond.
-* **Provide Context**: Help the model understand the bigger picture of your request. This could be background information, examples/demonstrations of what you want or explaining the purpose of your task.
-* **Ask the model to answer as if it was an expert.** Explicitly asking the model to produce high quality output or output as if it was written by an expert can induce the model to give higher quality answers that it thinks an expert would write. Phrases like "Explain in detail" or "Describe step-by-step" can be effective.
-* **Prompt the model to write down the series of steps explaining its reasoning.** If understanding the 'why' behind an answer is important, prompt the model to include its reasoning. This can be done by simply adding a line like "[Let's think step by step](https://arxiv.org/abs/2205.11916)" before each answer.
-
-
+- **Be more specific** E.g., if you want the output to be a comma separated list, ask it to return a comma separated list. If you want it to say "I don't know" when it doesn't know the answer, tell it 'Say "I don't know" if you do not know the answer.' The more specific your instructions, the better the model can respond.
+- **Provide Context**: Help the model understand the bigger picture of your request. This could be background information, examples/demonstrations of what you want or explaining the purpose of your task.
+- **Ask the model to answer as if it was an expert.** Explicitly asking the model to produce high quality output or output as if it was written by an expert can induce the model to give higher quality answers that it thinks an expert would write. Phrases like "Explain in detail" or "Describe step-by-step" can be effective.
+- **Prompt the model to write down the series of steps explaining its reasoning.** If understanding the 'why' behind an answer is important, prompt the model to include its reasoning. This can be done by simply adding a line like "[Let's think step by step](https://arxiv.org/abs/2205.11916)" before each answer.
 
 [Fine Tuning Docs]: https://platform.openai.com/docs/guides/fine-tuning
 [OpenAI Customer Stories]: https://openai.com/customer-stories

articles/techniques_to_improve_reliability.md

@@ -14,25 +14,25 @@ If you were asked to multiply 13 by 17, would the answer pop immediately into yo
 
 Similarly, if you give GPT-3 a task that's too complex to do in the time it takes to calculate its next token, it may confabulate an incorrect guess. Yet, akin to humans, that doesn't necessarily mean the model is incapable of the task. With some time and space to reason things out, the model still may be able to answer reliably.
 
-As an example, if you ask `text-davinci-002` the following math problem about juggling balls, it answers incorrectly:
+As an example, if you ask `gpt-3.5-turbo-instruct` the following math problem about juggling balls, it answers incorrectly:
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 Q: A juggler has 16 balls. Half of the balls are golf balls and half of the golf balls are blue. How many blue golf balls are there?
 A:
 ```
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 There are 8 blue golf balls.
 ```
 
 Does this mean that GPT-3 cannot do simple math problems? No; in fact, it turns out that by prompting the model with `Let's think step by step`, the model solves the problem reliably:
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 Q: A juggler has 16 balls. Half of the balls are golf balls and half of the golf balls are blue. How many blue golf balls are there?
 A: Let's think step by step.
 ```
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 There are 16 balls in total.
 Half of the balls are golf balls.
 That means that there are 8 golf balls.
@@ -64,9 +64,9 @@ The rest of this article shares techniques for improving reliability of large la
 
 One way to give a model more time and space to think is to break tasks into simpler pieces.
 
-As an example, consider a task where we ask the model a multiple-choice question about some text - in this case, a game of Clue. When asked directly, `text-davinci-002` isn't able to put clues 3 & 5 together, and answers incorrectly:
+As an example, consider a task where we ask the model a multiple-choice question about some text - in this case, a game of Clue. When asked directly, `gpt-3.5-turbo-instruct` isn't able to put clues 3 & 5 together, and answers incorrectly:
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 Use the following clues to answer the following multiple-choice question.
 
 Clues:
@@ -84,7 +84,7 @@ Question: Was Colonel Mustard in the observatory with the candlestick?
 Solution:
 ```
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 (c) Unknown; there is not enough information to determine whether Colonel Mustard was in the observatory with the candlestick
 ```
 
@@ -96,7 +96,7 @@ However, instead of asking for the answer directly, we can split the task into t
 - Second, combine the relevant clues to reason out the answer to the question
 - Third, write the final answer: either (a), (b), or (c)
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 Use the following clues to answer the following multiple-choice question, using the following procedure:
 (1) First, go through the clues one by one and consider whether the clue is potentially relevant
 (2) Second, combine the relevant clues to reason out the answer to the question
@@ -118,7 +118,7 @@ Solution:
 (1) First, go through the clues one by one and consider whether the clue is potentially relevant:
 ```
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 Clue 1: Miss Scarlett was the only person in the lounge. This clue is not relevant.
 Clue 2: The person with the pipe was in the kitchen. This clue is not relevant.
 Clue 3: Colonel Mustard was the only person in the observatory. This clue is relevant.
@@ -136,9 +136,9 @@ By giving the model more time and space to think, and guiding it along a reasoni
 
 Another benefit of splitting complex instructions into smaller subtasks is that it can help keep the model focused on each subtask.
 
-For example, if we ask `text-davinci-002` to summarize a text in its original language, the model can lapse back into English:
+For example, if we ask `gpt-3.5-turbo-instruct` to summarize a text in its original language, the model can lapse back into English:
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 Summarize the text using the original language of the text. The summary should be one sentence long.
 
 Text:
@@ -149,13 +149,13 @@ La estadística (la forma femenina del término alemán Statistik, derivado a su
 Summary:
 ```
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 The text explains that statistics is a science that studies the variability, collection, organization, analysis, interpretation, and presentation of data, as well as the random process that generates them following the laws of probability.
 ```
 
 However, if we first ask the model to identify the language of the text, and then summarize the text, it becomes more reliable:
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 First, identify the language of the text. Second, summarize the text using the original language of the text. The summary should be one sentence long.
 
 Text:
@@ -166,7 +166,7 @@ La estadística (la forma femenina del término alemán Statistik, derivado a su
 Language:
 ```
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 Spanish
 
 La estadística es una ciencia que estudia la variabilidad, colección, organización, análisis, interpretación, y presentación de los datos, así como el proceso aleatorio que los genera siguiendo las leyes de la probabilidad.
@@ -203,7 +203,7 @@ To learn more, read the [full paper](https://arxiv.org/abs/2205.11916).
 
 If you apply this technique to your own tasks, don't be afraid to experiment with customizing the instruction. `Let's think step by step` is rather generic, so you may find better performance with instructions that hew to a stricter format customized to your use case. For example, you can try more structured variants like `First, think step by step about why X might be true. Second, think step by step about why Y might be true. Third, think step by step about whether X or Y makes more sense.`. And you can even give the model an example format to help keep it on track, e.g.:
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
 Using the IRS guidance below, answer the following questions using this format:
 (1) For each criterion, determine whether it is met by the vehicle purchase
 - {Criterion} Let's think step by step. {explanation} {yes or no, or if the question does not apply then N/A}.
@@ -229,7 +229,7 @@ Solution:
 - Does the vehicle have at least four wheels? Let's think step by step.
 ```
 
-```text-davinci-002
+```gpt-3.5-turbo-instruct
  The Toyota Prius Prime has four wheels, so the answer is yes.
 - Does the vehicle weigh less than 14,000 pounds? Let's think step by step. The Toyota Prius Prime weighs less than 14,000 pounds, so the answer is yes.
 - Does the vehicle draw energy from a battery with at least 4 kilowatt hours that may be recharged from an external source? Let's think step by step. The Toyota Prius Prime has a battery with at least 4 kilowatt hours that may be recharged from an external source, so the answer is yes.

articles/text_comparison_examples.md

@@ -8,8 +8,8 @@ Embeddings can be used for semantic search, recommendations, cluster analysis, n
 
 For more information, read OpenAI's blog post announcements:
 
-* [Introducing Text and Code Embeddings (Jan 2022)](https://openai.com/blog/introducing-text-and-code-embeddings/)
-* [New and Improved Embedding Model (Dec 2022)](https://openai.com/blog/new-and-improved-embedding-model/)
+- [Introducing Text and Code Embeddings (Jan 2022)](https://openai.com/blog/introducing-text-and-code-embeddings/)
+- [New and Improved Embedding Model (Dec 2022)](https://openai.com/blog/new-and-improved-embedding-model/)
 
 For comparison with other embedding models, see [Massive Text Embedding Benchmark (MTEB) Leaderboard](https://huggingface.co/spaces/mteb/leaderboard)
 
@@ -19,14 +19,14 @@ Embeddings can be used for search either by themselves or as a feature in a larg
 
 The simplest way to use embeddings for search is as follows:
 
-* Before the search (precompute):
-  * Split your text corpus into chunks smaller than the token limit (8,191 tokens for `text-embedding-ada-002`)
-  * Embed each chunk of text
-  * Store those embeddings in your own database or in a vector search provider like [Pinecone](https://www.pinecone.io), [Weaviate](https://weaviate.io) or [Qdrant](https://qdrant.tech)
-* At the time of the search (live compute):
-  * Embed the search query
-  * Find the closest embeddings in your database
-  * Return the top results
+- Before the search (precompute):
+  - Split your text corpus into chunks smaller than the token limit (8,191 tokens for `text-embedding-3-small`)
+  - Embed each chunk of text
+  - Store those embeddings in your own database or in a vector search provider like [Pinecone](https://www.pinecone.io), [Weaviate](https://weaviate.io) or [Qdrant](https://qdrant.tech)
+- At the time of the search (live compute):
+  - Embed the search query
+  - Find the closest embeddings in your database
+  - Return the top results
 
 An example of how to use embeddings for search is shown in [Semantic_text_search_using_embeddings.ipynb](../examples/Semantic_text_search_using_embeddings.ipynb).
 

authors.yaml

@@ -53,6 +53,11 @@ prakul:
   website: "https://www.linkedin.com/in/prakulagarwal"
   avatar: "https://media.licdn.com/dms/image/D5603AQEUug83qKgRBg/profile-displayphoto-shrink_800_800/0/1675384960197?e=1706140800&v=beta&t=qxkDbBr-Bk2ASpcwbR5JVPD6yS-vzmIwNHAa8ApyDq4"
 
+gaborcselle:
+  name: "Gabor Cselle"
+  website: "https://www.linkedin.com/in/gaborcselle"
+  avatar: "https://avatars.githubusercontent.com/u/54096?s=96&v=4"
+
 nghiauet:
   name: "Nghia Pham"
   website: "https://www.linkedin.com/in/deptraicucmanh/"
@@ -62,3 +67,18 @@ ggrn:
   name: "Greg Richardson"
   website: "https://twitter.com/ggrdson"
   avatar: "https://pbs.twimg.com/profile_images/1371549909820657664/ZG-HDNlI_400x400.jpg"
+
+teomusatoiu:
+  name: "Teodora Musatoiu"
+  website: "https://www.linkedin.com/in/teodora-musatoiu/"
+  avatar: "https://avatars.githubusercontent.com/u/156829031?s=400&u=af40fe04d9255139eb3bbf8dc83422cc694e862b&v=4"
+
+katiagg:
+  name: "Katia Gil Guzman"
+  website: "https://katia.gg"
+  avatar: "https://avatars.githubusercontent.com/u/16519462?v=4"
+
+jbeutler-openai:
+  name: "Joe Beutler"
+  website: "https://joebeutler.com"
+  avatar: "https://avatars.githubusercontent.com/u/156261485?v=4"

examples/Assistants_API_overview_python.ipynb

@@ -191,8 +191,10 @@
    ],
    "source": [
     "from openai import OpenAI\n",
+    "import os\n",
+    "\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
     "\n",
-    "client = OpenAI()\n",
     "\n",
     "assistant = client.beta.assistants.create(\n",
     "    name=\"Math Tutor\",\n",
@@ -586,7 +588,7 @@
     "\n",
     "MATH_ASSISTANT_ID = assistant.id  # or a hard-coded ID like \"asst-...\"\n",
     "\n",
-    "client = OpenAI()\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
     "\n",
     "def submit_message(assistant_id, thread, user_message):\n",
     "    client.beta.threads.messages.create(\n",
@@ -1154,7 +1156,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Once again, let's update out Assistant either through the Dashboard or the API.\n"
+    "Once again, let's update our Assistant either through the Dashboard or the API.\n"
    ]
   },
   {

examples/Backtranslation_of_SQL_queries.py

@@ -1,189 +0,0 @@
-from typing import List, Union
-
-from smokey import Smokey
-
-import openai
-
-
-def get_candidates(
-    prompt: str,
-    stop: List[str],
-    temperature: float,
-    priming_prefix: str,
-    engine: str,
-    n: int = 5,
-) -> List[str]:
-    """
-    Generate N candidate completions based on the prompt, generated with a specific temperature.
-
-    :param prompt: The prompt to start the conversation with.
-    :param stop: A list of tokens that indicate the end of the generation.
-    :param temperature: The temperature of the generation.
-    :param priming_prefix: The prefix to use for the priming.
-    :param engine: The engine to use for the generation.
-    :param n: The number of completions to generate.
-    :return: A list of completions.
-    """
-    response = openai.Completion.create(
-        engine=engine,
-        prompt=prompt,
-        temperature=temperature,
-        max_tokens=150,
-        top_p=1,
-        frequency_penalty=0,
-        presence_penalty=0,
-        stop=stop,
-        n=n,
-    )
-    responses = [priming_prefix + choice.text for choice in response.choices]
-    return responses
-
-
-def rindex(lst: List, value: str) -> int:
-    """
-    Return the index of the last occurrence of a value in a list.
-
-    :param lst: The list to search in.
-    :param value: The value to search for.
-    :return: The index of the last occurrence of the value.
-    """
-    try:
-        return len(lst) - lst[::-1].index(value) - 1
-    except ValueError:
-        raise ValueError(f"Answer start token `{value}` not found in the eval template")
-
-
-def eval_candidate(
-    candidate_answer: str,
-    original_instruction: str,
-    eval_template: str,
-    answer_start_token: str,
-    engine: str,
-) -> float:
-    """
-    Evaluate a candidate answer by calculating the average log probability
-    of the original instruction, given the candidate answer with a specific
-    evaluation template, aimed at reconstructing the original instruction.
-
-    :param candidate_answer: The candidate answer to evaluate.
-    :param original_instruction: The original instruction.
-    :param eval_template: The template to use for the evaluation.
-    :param answer_start_token: The token to use to indicate the start of the answer.
-    :param engine: The engine to use for the evaluation.
-    :return: The evaluation of the candidate answer.
-    """
-    response = openai.Completion.create(
-        engine=engine,
-        prompt=eval_template.format(candidate_answer, original_instruction),
-        temperature=0,
-        max_tokens=0,
-        top_p=1,
-        frequency_penalty=0,
-        presence_penalty=0,
-        logprobs=1,
-        echo=True,
-    )
-
-    answer_start = rindex(
-        response["choices"][0]["logprobs"]["tokens"], answer_start_token
-    )
-    logprobs = response["choices"][0]["logprobs"]["token_logprobs"][answer_start + 1 :]
-    return sum(logprobs) / len(logprobs)
-
-
-def backtranslation(
-    prompt_template: str,
-    additional_info: str,
-    instruction: str,
-    eval_template: str,
-    priming_prefix: str = "SELECT",
-    stop1: List[str] = ["#", ";"],
-    answer_start_token: str = "--",
-    n: int = 5,
-    temperature: float = 0.5,
-    return_all_results: bool = False,
-    engine: str = "davinci-codex",
-) -> Union[str, List[str, float]]:
-    """
-    Generate a number of SQL queries given a natural language instruction,
-    and pick the best one based on the average log probability of explaining the
-    candidate SQL query with the exact original instruction, when prompted for
-    a natural language explanation of the candidate SQL query.
-
-    :param prompt_template: The template to use for the prompt to generate SQL.
-    :param additional_info: Additional information to include in the prompt
-                            (SQL Tables, and their properties).
-    :param instruction: The instruction in natural language.
-    :param eval_template: The template to use for the evaluation.
-    :param priming_prefix: The prefix to use for the priming of the SQL query.
-    :param stop1: A list of tokens that indicate the end of the generation.
-    :param answer_start_token: The token to use to indicate the start of the
-                               natural answer.
-    :param n: The number of candidates to generate.
-    :param temperature: The temperature of the generation.
-    :param return_all_results: Whether to return all results or just the best one.
-    :param engine: The engine to use for the generation and evaluation.
-    :return: The best SQL query, or a list of all scored generated SQL queries.
-    """
-    prompt_template = prompt_template.format(
-        additional_info, instruction, priming_prefix
-    )
-
-    candidates = []
-    responses = get_candidates(
-        prompt_template, stop1, temperature, priming_prefix, engine=engine, n=n
-    )
-    for i in range(n):
-        quality = eval_candidate(
-            responses[i],
-            instruction,
-            eval_template,
-            answer_start_token,
-            engine=engine,
-        )
-        candidates.append((responses[i], quality))
-
-    candidates.sort(key=lambda x: x[1], reverse=True)
-    if return_all_results:
-        return candidates
-    return candidates[0][0]
-
-
-def main(
-    nl_query: str = "Return the name of each department that had more than 10 employees in June 2021",
-    eval_template: str = "{};\n-- Explanation of the above query in human readable format\n-- {}",
-    table_definitions: str = "# Employee(id, name, department_id)\n# Department(id, name, address)\n# Salary_Payments(id, employee_id, amount, date)\n",
-    prompt_template: str = "### Postgres SQL tables, with their properties:\n#\n{}#\n### {}\n{}",
-    n: int = 3,
-    temperature: float = 0.3,
-    engine: str = "davinci-codex",
-):
-    """
-    Generate a number of SQL queries given a natural language instruction,
-    and pick the best one based on the highest backtranslation score.
-
-    :param nl_query: The natural language query.
-    :param eval_template: The template to use for the evaluation.
-    :param table_definitions: The definitions of the tables used in the query.
-    :param prompt_template: The template to use for the prompt to generate SQL.
-    :param n: The number of candidates to generate.
-    :param temperature: The temperature of the generation.
-    :param engine: The engine to use for the generation and evaluation.
-    :return: The best SQL query, or a list of all scored generated SQL queries.
-    """
-
-    result = backtranslation(
-        prompt_template,
-        table_definitions,
-        nl_query,
-        eval_template,
-        priming_prefix="SELECT",
-        temperature=temperature,
-        n=n,
-        engine=engine,
-    )
-    print(result)
-
-
-if __name__ == "__main__":
-    Smokey(main)

examples/Chat_finetuning_data_prep.ipynb

@@ -83,7 +83,7 @@
     "1. **Data Type Check**: Checks whether each entry in the dataset is a dictionary (`dict`). Error type: `data_type`.\n",
     "2. **Presence of Message List**: Checks if a `messages` list is present in each entry. Error type: `missing_messages_list`.\n",
     "3. **Message Keys Check**: Validates that each message in the `messages` list contains the keys `role` and `content`. Error type: `message_missing_key`.\n",
-    "4. **Unrecognized Keys in Messages**: Logs if a message has keys other than `role`, `content`, and `name`. Error type: `message_unrecognized_key`.\n",
+    "4. **Unrecognized Keys in Messages**: Logs if a message has keys other than `role`, `content`, `weight`, `function_call`, and `name`. Error type: `message_unrecognized_key`.\n",
     "5. **Role Validation**: Ensures the `role` is one of \"system\", \"user\", or \"assistant\". Error type: `unrecognized_role`.\n",
     "6. **Content Validation**: Verifies that `content` has textual data and is a string. Error type: `missing_content`.\n",
     "7. **Assistant Message Presence**: Checks that each conversation has at least one message from the assistant. Error type: `example_missing_assistant_message`.\n",
@@ -123,7 +123,7 @@
     "        if \"role\" not in message or \"content\" not in message:\n",
     "            format_errors[\"message_missing_key\"] += 1\n",
     "        \n",
-    "        if any(k not in (\"role\", \"content\", \"name\", \"function_call\") for k in message):\n",
+    "        if any(k not in (\"role\", \"content\", \"name\", \"function_call\", \"weight\") for k in message):\n",
     "            format_errors[\"message_unrecognized_key\"] += 1\n",
     "        \n",
     "        if message.get(\"role\", None) not in (\"system\", \"user\", \"assistant\", \"function\"):\n",

examples/Clustering_for_transaction_classification.ipynb

@@ -58,8 +58,7 @@
     "import os\n",
     "from ast import literal_eval\n",
     "\n",
-    "#openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n",
-    "client = OpenAI()\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
     "COMPLETIONS_MODEL = \"gpt-3.5-turbo\"\n",
     "\n",
     "# This path leads to a file with data and precomputed embeddings\n",
@@ -399,7 +398,7 @@
     "        .sample(transactions_per_cluster, random_state=42)\n",
     "        .values\n",
     "    )\n",
-    "    completion = client.chat.completions.create(\n",
+    "    response = client.chat.completions.create(\n",
     "        model=COMPLETIONS_MODEL,\n",
     "        # We'll include a prompt to instruct the model what sort of description we're looking for\n",
     "        messages=[\n",
@@ -413,7 +412,7 @@
     "        frequency_penalty=0,\n",
     "        presence_penalty=0,\n",
     "    )\n",
-    "    print(completion.choices[0].message.content.replace(\"\\n\", \"\"))\n",
+    "    print(response.choices[0].message.content.replace(\"\\n\", \"\"))\n",
     "    print(\"\\n\")\n",
     "\n",
     "    sample_cluster_rows = embedding_df[embedding_df.Cluster == i].sample(transactions_per_cluster, random_state=42)\n",

examples/Code_search_using_embeddings.ipynb

@@ -22,7 +22,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -110,15 +110,15 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Total number of .py files: 57\n",
-      "Total number of functions extracted: 118\n"
+      "Total number of .py files: 51\n",
+      "Total number of functions extracted: 97\n"
      ]
     }
    ],
@@ -138,12 +138,12 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Now that we have our content, we can pass the data to the text-embedding-ada-002 endpoint to get back our vector embeddings."
+    "Now that we have our content, we can pass the data to the `text-embedding-3-small` model and get back our vector embeddings."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [
     {
@@ -179,35 +179,35 @@
        "      <td>def _console_log_level():\\n    if openai.log i...</td>\n",
        "      <td>_console_log_level</td>\n",
        "      <td>openai/util.py</td>\n",
-       "      <td>[0.033906757831573486, -0.00418944051489234, 0...</td>\n",
+       "      <td>[0.005937571171671152, 0.05450401455163956, 0....</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>1</th>\n",
        "      <td>def log_debug(message, **params):\\n    msg = l...</td>\n",
        "      <td>log_debug</td>\n",
        "      <td>openai/util.py</td>\n",
-       "      <td>[-0.004059609025716782, 0.004895503632724285, ...</td>\n",
+       "      <td>[0.017557814717292786, 0.05647840350866318, -0...</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2</th>\n",
        "      <td>def log_info(message, **params):\\n    msg = lo...</td>\n",
        "      <td>log_info</td>\n",
        "      <td>openai/util.py</td>\n",
-       "      <td>[0.0048639848828315735, 0.0033139237202703953,...</td>\n",
+       "      <td>[0.022524144500494003, 0.06219055876135826, -0...</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>3</th>\n",
        "      <td>def log_warn(message, **params):\\n    msg = lo...</td>\n",
        "      <td>log_warn</td>\n",
        "      <td>openai/util.py</td>\n",
-       "      <td>[0.0024026145692914724, -0.010721310041844845,...</td>\n",
+       "      <td>[0.030524108558893204, 0.0667714849114418, -0....</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>4</th>\n",
        "      <td>def logfmt(props):\\n    def fmt(key, val):\\n  ...</td>\n",
        "      <td>logfmt</td>\n",
        "      <td>openai/util.py</td>\n",
-       "      <td>[0.01664826273918152, 0.01730910874903202, 0.0...</td>\n",
+       "      <td>[0.05337328091263771, 0.03697286546230316, -0....</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
@@ -222,14 +222,14 @@
        "4  def logfmt(props):\\n    def fmt(key, val):\\n  ...              logfmt   \n",
        "\n",
        "         filepath                                     code_embedding  \n",
-       "0  openai/util.py  [0.033906757831573486, -0.00418944051489234, 0...  \n",
-       "1  openai/util.py  [-0.004059609025716782, 0.004895503632724285, ...  \n",
-       "2  openai/util.py  [0.0048639848828315735, 0.0033139237202703953,...  \n",
-       "3  openai/util.py  [0.0024026145692914724, -0.010721310041844845,...  \n",
-       "4  openai/util.py  [0.01664826273918152, 0.01730910874903202, 0.0...  "
+       "0  openai/util.py  [0.005937571171671152, 0.05450401455163956, 0....  \n",
+       "1  openai/util.py  [0.017557814717292786, 0.05647840350866318, -0...  \n",
+       "2  openai/util.py  [0.022524144500494003, 0.06219055876135826, -0...  \n",
+       "3  openai/util.py  [0.030524108558893204, 0.0667714849114418, -0....  \n",
+       "4  openai/util.py  [0.05337328091263771, 0.03697286546230316, -0....  "
       ]
      },
-     "execution_count": 11,
+     "execution_count": 3,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -238,7 +238,7 @@
     "from utils.embeddings_utils import get_embedding\n",
     "\n",
     "df = pd.DataFrame(all_funcs)\n",
-    "df['code_embedding'] = df['code'].apply(lambda x: get_embedding(x, model='text-embedding-ada-002'))\n",
+    "df['code_embedding'] = df['code'].apply(lambda x: get_embedding(x, model='text-embedding-3-small'))\n",
     "df['filepath'] = df['filepath'].map(lambda x: Path(x).relative_to(code_root))\n",
     "df.to_csv(\"data/code_search_openai-python.csv\", index=False)\n",
     "df.head()"
@@ -255,21 +255,21 @@
     "\n",
     "We define a search_functions method that takes our data that contains our embeddings, a query string, and some other configuration options. The process of searching our database works like such:\n",
     "\n",
-    "1. We first embed our query string (code_query) with text-embedding-ada-002. The reasoning here is that a query string like 'a function that reverses a string' and a function like 'def reverse(string): return string[::-1]' will be very similar when embedded.\n",
+    "1. We first embed our query string (code_query) with `text-embedding-3-small`. The reasoning here is that a query string like 'a function that reverses a string' and a function like 'def reverse(string): return string[::-1]' will be very similar when embedded.\n",
     "2. We then calculate the cosine similarity between our query string embedding and all data points in our database. This gives a distance between each point and our query.\n",
     "3. We finally sort all of our data points by their distance to our query string and return the number of results requested in the function parameters. "
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [],
    "source": [
     "from utils.embeddings_utils import cosine_similarity\n",
     "\n",
     "def search_functions(df, code_query, n=3, pprint=True, n_lines=7):\n",
-    "    embedding = get_embedding(code_query, model='text-embedding-ada-002')\n",
+    "    embedding = get_embedding(code_query, model='text-embedding-3-small')\n",
     "    df['similarities'] = df.code_embedding.apply(lambda x: cosine_similarity(x, embedding))\n",
     "\n",
     "    res = df.sort_values('similarities', ascending=False).head(n)\n",
@@ -285,14 +285,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "openai/validators.py:format_inferrer_validator  score=0.751\n",
+      "openai/validators.py:format_inferrer_validator  score=0.453\n",
       "def format_inferrer_validator(df):\n",
       "    \"\"\"\n",
       "    This validator will infer the likely fine-tuning format of the data, and display it to the user if it is classification.\n",
@@ -301,16 +301,7 @@
       "    ft_type = infer_task_type(df)\n",
       "    immediate_msg = None\n",
       "----------------------------------------------------------------------\n",
-      "openai/validators.py:get_validators  score=0.748\n",
-      "def get_validators():\n",
-      "    return [\n",
-      "        num_examples_validator,\n",
-      "        lambda x: necessary_column_validator(x, \"prompt\"),\n",
-      "        lambda x: necessary_column_validator(x, \"completion\"),\n",
-      "        additional_column_validator,\n",
-      "        non_empty_field_validator,\n",
-      "----------------------------------------------------------------------\n",
-      "openai/validators.py:infer_task_type  score=0.739\n",
+      "openai/validators.py:infer_task_type  score=0.37\n",
       "def infer_task_type(df):\n",
       "    \"\"\"\n",
       "    Infer the likely fine-tuning task type from the data\n",
@@ -318,6 +309,15 @@
       "    CLASSIFICATION_THRESHOLD = 3  # min_average instances of each class\n",
       "    if sum(df.prompt.str.len()) == 0:\n",
       "        return \"open-ended generation\"\n",
+      "----------------------------------------------------------------------\n",
+      "openai/validators.py:apply_validators  score=0.369\n",
+      "def apply_validators(\n",
+      "    df,\n",
+      "    fname,\n",
+      "    remediation,\n",
+      "    validators,\n",
+      "    auto_accept,\n",
+      "    write_out_file_func,\n",
       "----------------------------------------------------------------------\n"
      ]
     }
@@ -328,14 +328,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "openai/validators.py:get_common_xfix  score=0.794\n",
+      "openai/validators.py:get_common_xfix  score=0.487\n",
       "def get_common_xfix(series, xfix=\"suffix\"):\n",
       "    \"\"\"\n",
       "    Finds the longest common suffix or prefix of all the values in a series\n",
@@ -347,7 +347,7 @@
       "            if xfix == \"suffix\"\n",
       "            else series.str[: len(common_xfix) + 1]\n",
       "----------------------------------------------------------------------\n",
-      "openai/validators.py:common_completion_suffix_validator  score=0.778\n",
+      "openai/validators.py:common_completion_suffix_validator  score=0.449\n",
       "def common_completion_suffix_validator(df):\n",
       "    \"\"\"\n",
       "    This validator will suggest to add a common suffix to the completion if one doesn't already exist in case of classification or conditional generation.\n",
@@ -368,14 +368,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 7,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "openai/cli.py:tools_register  score=0.78\n",
+      "openai/cli.py:tools_register  score=0.391\n",
       "def tools_register(parser):\n",
       "    subparsers = parser.add_subparsers(\n",
       "        title=\"Tools\", help=\"Convenience client side tools\"\n",
@@ -421,7 +421,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.9"
+   "version": "3.11.5"
   },
   "orig_nbformat": 4
  },

examples/Creating_slides_with_Assistants_API_and_DALL-E3.ipynb

@@ -25,13 +25,14 @@
    "source": [
     "from IPython.display import display, Image\n",
     "from openai import OpenAI\n",
+    "import os\n",
     "import pandas as pd\n",
     "import json\n",
     "import io\n",
     "from PIL import Image\n",
     "import requests\n",
     "\n",
-    "client = OpenAI()\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
     "\n",
     "#Lets import some helper functions for assistants from https://cookbook.openai.com/examples/assistants_api_overview_python\n",
     "def show_json(obj):\n",

examples/Customizing_embeddings.ipynb

@@ -78,7 +78,7 @@
       "source": [
         "# input parameters\n",
         "embedding_cache_path = \"data/snli_embedding_cache.pkl\"  # embeddings will be saved/loaded here\n",
-        "default_embedding_engine = \"babbage-similarity\"  # text-embedding-ada-002 is recommended\n",
+        "default_embedding_engine = \"text-embedding-3-small\"\n",
         "num_pairs_to_embed = 1000  # 1000 is arbitrary\n",
         "local_dataset_path = \"data/snli_1.0_train_2k.csv\"  # download from: https://nlp.stanford.edu/projects/snli/\n",
         "\n",

examples/Embedding_Wikipedia_articles_for_search.ipynb

@@ -38,9 +38,12 @@
     "import mwclient  # for downloading example Wikipedia articles\n",
     "import mwparserfromhell  # for splitting Wikipedia articles into sections\n",
     "import openai  # for generating embeddings\n",
+    "import os  # for environment variables\n",
     "import pandas as pd  # for DataFrames to store article sections and embeddings\n",
     "import re  # for cutting <ref> links out of Wikipedia articles\n",
-    "import tiktoken  # for counting tokens\n"
+    "import tiktoken  # for counting tokens\n",
+    "\n",
+    "client = openai.OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {
@@ -603,8 +606,7 @@
     }
    ],
    "source": [
-    "# calculate embeddings\n",
-    "EMBEDDING_MODEL = \"text-embedding-ada-002\"  # OpenAI's best embeddings as of Apr 2023\n",
+    "EMBEDDING_MODEL = \"text-embedding-3-small\"\n",
     "BATCH_SIZE = 1000  # you can submit up to 2048 embedding inputs per request\n",
     "\n",
     "embeddings = []\n",
@@ -612,10 +614,10 @@
     "    batch_end = batch_start + BATCH_SIZE\n",
     "    batch = wikipedia_strings[batch_start:batch_end]\n",
     "    print(f\"Batch {batch_start} to {batch_end-1}\")\n",
-    "    response = openai.Embedding.create(model=EMBEDDING_MODEL, input=batch)\n",
-    "    for i, be in enumerate(response[\"data\"]):\n",
-    "        assert i == be[\"index\"]  # double check embeddings are in same order as input\n",
-    "    batch_embeddings = [e[\"embedding\"] for e in response[\"data\"]]\n",
+    "    response = client.embeddings.create(model=EMBEDDING_MODEL, input=batch)\n",
+    "    for i, be in enumerate(response.data):\n",
+    "        assert i == be.index  # double check embeddings are in same order as input\n",
+    "    batch_embeddings = [e.embedding for e in response.data]\n",
     "    embeddings.extend(batch_embeddings)\n",
     "\n",
     "df = pd.DataFrame({\"text\": wikipedia_strings, \"embedding\": embeddings})\n"

examples/Embedding_long_inputs.ipynb

@@ -9,7 +9,7 @@
     "\n",
     "OpenAI's embedding models cannot embed text that exceeds a maximum length. The maximum length varies by model, and is measured by _tokens_, not string length. If you are unfamiliar with tokenization, check out [How to count tokens with tiktoken](How_to_count_tokens_with_tiktoken.ipynb).\n",
     "\n",
-    "This notebook shows how to handle texts that are longer than a model's maximum context length. We'll demonstrate using embeddings from `text-embedding-ada-002`, but the same ideas can be applied to other models and tasks. To learn more about embeddings, check out the [OpenAI Embeddings Guide](https://beta.openai.com/docs/guides/embeddings).\n"
+    "This notebook shows how to handle texts that are longer than a model's maximum context length. We'll demonstrate using embeddings from `text-embedding-3-small`, but the same ideas can be applied to other models and tasks. To learn more about embeddings, check out the [OpenAI Embeddings Guide](https://beta.openai.com/docs/guides/embeddings).\n"
    ]
   },
   {
@@ -24,41 +24,44 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
+    "from openai import OpenAI\n",
+    "import os\n",
     "import openai\n",
     "from tenacity import retry, wait_random_exponential, stop_after_attempt, retry_if_not_exception_type\n",
     "\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
     "\n",
-    "EMBEDDING_MODEL = 'text-embedding-ada-002'\n",
+    "EMBEDDING_MODEL = 'text-embedding-3-small'\n",
     "EMBEDDING_CTX_LENGTH = 8191\n",
     "EMBEDDING_ENCODING = 'cl100k_base'\n",
     "\n",
     "# let's make sure to not retry on an invalid request, because that is what we want to demonstrate\n",
-    "@retry(wait=wait_random_exponential(min=1, max=20), stop=stop_after_attempt(6), retry=retry_if_not_exception_type(openai.InvalidRequestError))\n",
+    "@retry(wait=wait_random_exponential(min=1, max=20), stop=stop_after_attempt(6), retry=retry_if_not_exception_type(openai.BadRequestError))\n",
     "def get_embedding(text_or_tokens, model=EMBEDDING_MODEL):\n",
-    "    return openai.Embedding.create(input=text_or_tokens, model=model)[\"data\"][0][\"embedding\"]"
+    "    return client.embeddings.create(input=text_or_tokens, model=model).data[0].embedding"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The `text-embedding-ada-002` model has a context length of 8191 tokens with the `cl100k_base` encoding, and we can see that going over that limit causes an error."
+    "The `text-embedding-3-small` model has a context length of 8191 tokens with the `cl100k_base` encoding, and we can see that going over that limit causes an error."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "This model's maximum context length is 8191 tokens, however you requested 10001 tokens (10001 in your prompt; 0 for the completion). Please reduce your prompt; or completion length.\n"
+      "Error code: 400 - {'error': {'message': \"This model's maximum context length is 8192 tokens, however you requested 10001 tokens (10001 in your prompt; 0 for the completion). Please reduce your prompt; or completion length.\", 'type': 'invalid_request_error', 'param': None, 'code': None}}\n"
      ]
     }
    ],
@@ -66,7 +69,7 @@
     "long_text = 'AGI ' * 5000\n",
     "try:\n",
     "    get_embedding(long_text)\n",
-    "except openai.InvalidRequestError as e:\n",
+    "except openai.BadRequestError as e:\n",
     "    print(e)"
    ]
   },
@@ -267,7 +270,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.9"
+   "version": "3.11.5"
   },
   "vscode": {
    "interpreter": {

examples/Entity_extraction_for_long_documents.ipynb

@@ -44,7 +44,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 11,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -53,6 +53,8 @@
     "import openai\n",
     "import tiktoken\n",
     "\n",
+    "client = openai.OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
+    "\n",
     "# Extract the raw text from each PDF using textract\n",
     "text = textract.process('data/fia_f1_power_unit_financial_regulations_issue_1_-_2022-08-16.pdf', method='pdfminer').decode('utf-8')\n",
     "clean_text = text.replace(\"  \", \" \").replace(\"\\n\", \"; \").replace(';',' ')"
@@ -67,7 +69,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 28,
+   "execution_count": 20,
    "metadata": {},
    "outputs": [
     {
@@ -96,13 +98,13 @@
     "template_prompt=f'''Extract key pieces of information from this regulation document.\n",
     "If a particular piece of information is not present, output \\\"Not specified\\\".\n",
     "When you extract a key piece of information, include the closest page number.\n",
-    "Use the following format:\\n0. Who is the author\\n1. What is the amount of the \"Power Unit Cost Cap\" in USD, GBP and EUR\\n2. What is the value of External Manufacturing Costs in USD\\n3. What is the Capital Expenditure Limit in USD\\n\\nDocument: \\\"\\\"\\\"{document}\\\"\\\"\\\"\\n\\n0. Who is the author: Tom Anderson (Page 1)\\n1.'''\n",
+    "Use the following format:\\n0. Who is the author\\n1. What is the amount of the \"Power Unit Cost Cap\" in USD, GBP and EUR\\n2. What is the value of External Manufacturing Costs in USD\\n3. What is the Capital Expenditure Limit in USD\\n\\nDocument: \\\"\\\"\\\"<document>\\\"\\\"\\\"\\n\\n0. Who is the author: Tom Anderson (Page 1)\\n1.'''\n",
     "print(template_prompt)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 29,
+   "execution_count": 17,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -127,19 +129,23 @@
     "        i = j\n",
     "\n",
     "def extract_chunk(document,template_prompt):\n",
-    "    \n",
-    "    prompt=template_prompt.replace('<document>',document)\n",
+    "    prompt = template_prompt.replace('<document>',document)\n",
     "\n",
-    "    response = openai.Completion.create(\n",
-    "    model='text-davinci-003', \n",
-    "    prompt=prompt,\n",
-    "    temperature=0,\n",
-    "    max_tokens=1500,\n",
-    "    top_p=1,\n",
-    "    frequency_penalty=0,\n",
-    "    presence_penalty=0\n",
-    "    )\n",
-    "    return \"1.\" + response['choices'][0]['text']"
+    "    messages = [\n",
+    "            {\"role\": \"system\", \"content\": \"You help extract information from documents.\"},\n",
+    "            {\"role\": \"user\", \"content\": prompt}\n",
+    "            ]\n",
+    "\n",
+    "    response = client.chat.completions.create(\n",
+    "            model='gpt-4', \n",
+    "            messages=messages,\n",
+    "            temperature=0,\n",
+    "            max_tokens=1500,\n",
+    "            top_p=1,\n",
+    "            frequency_penalty=0,\n",
+    "            presence_penalty=0\n",
+    "        )\n",
+    "    return \"1.\" + response.choices[0].message.content"
    ]
   },
   {
@@ -228,7 +234,7 @@
     "template_prompt=f'''Extract key pieces of information from this regulation document.\n",
     "If a particular piece of information is not present, output \\\"Not specified\\\".\n",
     "When you extract a key piece of information, include the closest page number.\n",
-    "Use the following format:\\n0. Who is the author\\n1. How is a Minor Overspend Breach calculated\\n2. How is a Major Overspend Breach calculated\\n3. Which years do these financial regulations apply to\\n\\nDocument: \\\"\\\"\\\"{document}\\\"\\\"\\\"\\n\\n0. Who is the author: Tom Anderson (Page 1)\\n1.'''\n",
+    "Use the following format:\\n0. Who is the author\\n1. How is a Minor Overspend Breach calculated\\n2. How is a Major Overspend Breach calculated\\n3. Which years do these financial regulations apply to\\n\\nDocument: \\\"\\\"\\\"<document>\\\"\\\"\\\"\\n\\n0. Who is the author: Tom Anderson (Page 1)\\n1.'''\n",
     "print(template_prompt)"
    ]
   },
@@ -306,9 +312,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "embed_retrieve",
+   "display_name": "Python 3",
    "language": "python",
-   "name": "embed_retrieve"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -320,12 +326,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.9"
-  },
-  "vscode": {
-   "interpreter": {
-    "hash": "5997d090960a54cd76552f75eca12ec3b416cf9d01a1a5af08ae48cf90878791"
-   }
+   "version": "3.11.3"
   }
  },
  "nbformat": 4,

examples/Function_calling_finding_nearby_places.ipynb

@@ -58,12 +58,11 @@
    "outputs": [],
    "source": [
     "import json\n",
-    "import openai\n",
     "from openai import OpenAI\n",
     "import os\n",
     "import requests\n",
     "\n",
-    "client = OpenAI()\n"
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {
@@ -228,7 +227,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 7,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -252,33 +251,36 @@
     "    {\"role\": \"user\", \"content\": user_input}\n",
     "],\n",
     "        temperature=0,\n",
-    "        functions=[\n",
+    "        tools=[\n",
     "            {\n",
-    "                \"name\": \"call_google_places_api\",\n",
-    "                \"description\": \"This function calls the Google Places API to find the top places of a specified type near a specific location. It can be used when a user expresses a need (e.g., feeling hungry or tired) or wants to find a certain type of place (e.g., restaurant or hotel).\",\n",
-    "                \"parameters\": {\n",
-    "                    \"type\": \"object\",\n",
-    "                    \"properties\": {\n",
-    "                        \"place_type\": {\n",
-    "                            \"type\": \"string\",\n",
-    "                            \"description\": \"The type of place to search for.\"\n",
+    "                \"type\": \"function\",\n",
+    "                \"function\" : {\n",
+    "                    \"name\": \"call_google_places_api\",\n",
+    "                    \"description\": \"This function calls the Google Places API to find the top places of a specified type near a specific location. It can be used when a user expresses a need (e.g., feeling hungry or tired) or wants to find a certain type of place (e.g., restaurant or hotel).\",\n",
+    "                    \"parameters\": {\n",
+    "                        \"type\": \"object\",\n",
+    "                        \"properties\": {\n",
+    "                            \"place_type\": {\n",
+    "                                \"type\": \"string\",\n",
+    "                                \"description\": \"The type of place to search for.\"\n",
+    "                            }\n",
+    "                        }\n",
+    "                    },\n",
+    "                    \"result\": {\n",
+    "                        \"type\": \"array\",\n",
+    "                        \"items\": {\n",
+    "                            \"type\": \"string\"\n",
     "                        }\n",
-    "                    }\n",
-    "                },\n",
-    "                \"result\": {\n",
-    "                    \"type\": \"array\",\n",
-    "                    \"items\": {\n",
-    "                        \"type\": \"string\"\n",
     "                    }\n",
     "                }\n",
     "            }\n",
     "        ],\n",
     "    )\n",
     "\n",
-    "    print(response.choices[0].message.function_call)\n",
+    "    print(response.choices[0].message.tool_calls)\n",
     "\n",
-    "    if response.choices[0].finish_reason=='function_call':\n",
-    "        function_call = response.choices[0].message.function_call\n",
+    "    if response.choices[0].finish_reason=='tool_calls':\n",
+    "        function_call = response.choices[0].message.tool_calls[0].function\n",
     "        if function_call.name == \"call_google_places_api\":\n",
     "            place_type = json.loads(function_call.arguments)[\"place_type\"]\n",
     "            places = call_google_places_api(user_id, place_type, food_preference)\n",
@@ -302,15 +304,15 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 8,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "FunctionCall(arguments='{\\n  \"place_type\": \"restaurant\"\\n}', name='call_google_places_api')\n",
-      "I couldn't find any places of interest nearby.\n"
+      "[ChatCompletionMessageToolCall(id='call_Q1mXIi7D6GhobfE4tkruX7nB', function=Function(arguments='{\\n  \"place_type\": \"restaurant\"\\n}', name='call_google_places_api'), type='function')]\n",
+      "Here are some places you might be interested in: Sotto Mare is a restaurant located at 552 Green Street. It has a rating of 4.6 based on 3765 user reviews. Mona Lisa Restaurant is a restaurant located at 353 Columbus Avenue #3907. It has a rating of 4.4 based on 1888 user reviews.\n"
      ]
     }
    ],
@@ -338,7 +340,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.16"
+   "version": "3.11.5"
   },
   "orig_nbformat": 4
  },

examples/GPT_with_vision_for_video_understanding.ipynb

@@ -27,7 +27,7 @@
     "import os\n",
     "import requests\n",
     "\n",
-    "client = OpenAI()"
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {

examples/Get_embeddings_from_dataset.ipynb

@@ -26,32 +26,30 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 20,
    "metadata": {},
    "outputs": [],
    "source": [
-    "# imports\n",
     "import pandas as pd\n",
     "import tiktoken\n",
     "\n",
-    "from utils.embeddings_utils import get_embedding\n"
+    "from utils.embeddings_utils import get_embedding"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 16,
    "metadata": {},
    "outputs": [],
    "source": [
-    "# embedding model parameters\n",
-    "embedding_model = \"text-embedding-ada-002\"\n",
-    "embedding_encoding = \"cl100k_base\"  # this the encoding for text-embedding-ada-002\n",
-    "max_tokens = 8000  # the maximum for text-embedding-ada-002 is 8191\n"
+    "embedding_model = \"text-embedding-3-small\"\n",
+    "embedding_encoding = \"cl100k_base\"\n",
+    "max_tokens = 8000  # the maximum for text-embedding-3-small is 8191"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 17,
    "metadata": {},
    "outputs": [
     {
@@ -127,7 +125,7 @@
        "1  Title: Arrived in pieces; Content: Not pleased...  "
       ]
      },
-     "execution_count": 8,
+     "execution_count": 17,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -141,12 +139,12 @@
     "df[\"combined\"] = (\n",
     "    \"Title: \" + df.Summary.str.strip() + \"; Content: \" + df.Text.str.strip()\n",
     ")\n",
-    "df.head(2)\n"
+    "df.head(2)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 18,
    "metadata": {},
    "outputs": [
     {
@@ -155,7 +153,7 @@
        "1000"
       ]
      },
-     "execution_count": 9,
+     "execution_count": 18,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -171,7 +169,7 @@
     "# omit reviews that are too long to embed\n",
     "df[\"n_tokens\"] = df.combined.apply(lambda x: len(encoding.encode(x)))\n",
     "df = df[df.n_tokens <= max_tokens].tail(top_n)\n",
-    "len(df)\n"
+    "len(df)"
    ]
   },
   {
@@ -184,7 +182,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 21,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -192,7 +190,16 @@
     "\n",
     "# This may take a few minutes\n",
     "df[\"embedding\"] = df.combined.apply(lambda x: get_embedding(x, model=embedding_model))\n",
-    "df.to_csv(\"data/fine_food_reviews_with_embeddings_1k.csv\")\n"
+    "df.to_csv(\"data/fine_food_reviews_with_embeddings_1k.csv\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "a = get_embedding(\"hi\", model=embedding_model)"
    ]
   }
  ],
@@ -212,7 +219,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.16"
+   "version": "3.11.5"
   },
   "orig_nbformat": 4,
   "vscode": {

examples/How_to_build_a_tool-using_agent_with_Langchain.ipynb

@@ -662,7 +662,7 @@
     "import wget\n",
     "\n",
     "# Here is a URL to a zip archive containing the transcribed podcasts\n",
-    "# Note that this data has already been split into chunks and embeddings from OpenAI's text-embedding-ada-002 embedding model are included\n",
+    "# Note that this data has already been split into chunks and embeddings from OpenAI's `text-embedding-3-small` embedding model are included\n",
     "content_url = 'https://cdn.openai.com/API/examples/data/sysk_podcast_transcripts_embedded.json.zip'\n",
     "\n",
     "# Download the file (it is ~541 MB so this will take some time)\n",

examples/How_to_call_functions_for_knowledge_retrieval.ipynb

@@ -34,113 +34,17 @@
      "is_executing": true
     }
    },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0mRequirement already satisfied: scipy in /opt/homebrew/lib/python3.11/site-packages (1.10.1)\n",
-      "Requirement already satisfied: numpy<1.27.0,>=1.19.5 in /opt/homebrew/lib/python3.11/site-packages (from scipy) (1.24.3)\n",
-      "\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0mRequirement already satisfied: tenacity in /opt/homebrew/lib/python3.11/site-packages (8.2.2)\n",
-      "\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0mRequirement already satisfied: tiktoken in /opt/homebrew/lib/python3.11/site-packages (0.4.0)\n",
-      "Requirement already satisfied: regex>=2022.1.18 in /opt/homebrew/lib/python3.11/site-packages (from tiktoken) (2023.6.3)\n",
-      "Requirement already satisfied: requests>=2.26.0 in /opt/homebrew/lib/python3.11/site-packages (from tiktoken) (2.30.0)\n",
-      "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/homebrew/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (3.1.0)\n",
-      "Requirement already satisfied: idna<4,>=2.5 in /opt/homebrew/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (3.4)\n",
-      "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/homebrew/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (1.25.11)\n",
-      "Requirement already satisfied: certifi>=2017.4.17 in /opt/homebrew/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (2023.5.7)\n",
-      "\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0mRequirement already satisfied: termcolor in /opt/homebrew/lib/python3.11/site-packages (2.3.0)\n",
-      "\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0mRequirement already satisfied: openai in /opt/homebrew/lib/python3.11/site-packages (0.27.6)\n",
-      "Requirement already satisfied: requests>=2.20 in /opt/homebrew/lib/python3.11/site-packages (from openai) (2.30.0)\n",
-      "Requirement already satisfied: tqdm in /opt/homebrew/lib/python3.11/site-packages (from openai) (4.65.0)\n",
-      "Requirement already satisfied: aiohttp in /opt/homebrew/lib/python3.11/site-packages (from openai) (3.8.4)\n",
-      "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/homebrew/lib/python3.11/site-packages (from requests>=2.20->openai) (3.1.0)\n",
-      "Requirement already satisfied: idna<4,>=2.5 in /opt/homebrew/lib/python3.11/site-packages (from requests>=2.20->openai) (3.4)\n",
-      "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/homebrew/lib/python3.11/site-packages (from requests>=2.20->openai) (1.25.11)\n",
-      "Requirement already satisfied: certifi>=2017.4.17 in /opt/homebrew/lib/python3.11/site-packages (from requests>=2.20->openai) (2023.5.7)\n",
-      "Requirement already satisfied: attrs>=17.3.0 in /opt/homebrew/lib/python3.11/site-packages (from aiohttp->openai) (23.1.0)\n",
-      "Requirement already satisfied: multidict<7.0,>=4.5 in /opt/homebrew/lib/python3.11/site-packages (from aiohttp->openai) (6.0.4)\n",
-      "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /opt/homebrew/lib/python3.11/site-packages (from aiohttp->openai) (4.0.2)\n",
-      "Requirement already satisfied: yarl<2.0,>=1.0 in /opt/homebrew/lib/python3.11/site-packages (from aiohttp->openai) (1.9.2)\n",
-      "Requirement already satisfied: frozenlist>=1.1.1 in /opt/homebrew/lib/python3.11/site-packages (from aiohttp->openai) (1.3.3)\n",
-      "Requirement already satisfied: aiosignal>=1.1.2 in /opt/homebrew/lib/python3.11/site-packages (from aiohttp->openai) (1.3.1)\n",
-      "\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0mRequirement already satisfied: requests in /opt/homebrew/lib/python3.11/site-packages (2.30.0)\n",
-      "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/homebrew/lib/python3.11/site-packages (from requests) (3.1.0)\n",
-      "Requirement already satisfied: idna<4,>=2.5 in /opt/homebrew/lib/python3.11/site-packages (from requests) (3.4)\n",
-      "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/homebrew/lib/python3.11/site-packages (from requests) (1.25.11)\n",
-      "Requirement already satisfied: certifi>=2017.4.17 in /opt/homebrew/lib/python3.11/site-packages (from requests) (2023.5.7)\n",
-      "\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0mRequirement already satisfied: arxiv in /opt/homebrew/lib/python3.11/site-packages (1.4.7)\n",
-      "Requirement already satisfied: feedparser in /opt/homebrew/lib/python3.11/site-packages (from arxiv) (6.0.10)\n",
-      "Requirement already satisfied: sgmllib3k in /opt/homebrew/lib/python3.11/site-packages (from feedparser->arxiv) (1.0.0)\n",
-      "\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\u001b[0mRequirement already satisfied: pandas in /opt/homebrew/lib/python3.11/site-packages (2.0.1)\n",
-      "Requirement already satisfied: python-dateutil>=2.8.2 in /opt/homebrew/lib/python3.11/site-packages (from pandas) (2.8.2)\n",
-      "Requirement already satisfied: pytz>=2020.1 in /opt/homebrew/lib/python3.11/site-packages (from pandas) (2023.3)\n",
-      "Requirement already satisfied: tzdata>=2022.1 in /opt/homebrew/lib/python3.11/site-packages (from pandas) (2023.3)\n",
-      "Requirement already satisfied: numpy>=1.21.0 in /opt/homebrew/lib/python3.11/site-packages (from pandas) (1.24.3)\n",
-      "Requirement already satisfied: six>=1.5 in /opt/homebrew/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0)\n",
-      "\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0mRequirement already satisfied: PyPDF2 in /opt/homebrew/lib/python3.11/site-packages (3.0.1)\n",
-      "\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0mRequirement already satisfied: tqdm in /opt/homebrew/lib/python3.11/site-packages (4.65.0)\n",
-      "\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m\u001b[33mWARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'\u001b[0m\u001b[33m\n",
-      "\u001b[0m"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
-    "!pip install scipy\n",
-    "!pip install tenacity\n",
-    "!pip install tiktoken==0.3.3\n",
-    "!pip install termcolor \n",
-    "!pip install openai\n",
-    "!pip install requests\n",
-    "!pip install arxiv\n",
-    "!pip install pandas\n",
-    "!pip install PyPDF2\n",
-    "!pip install tqdm"
+    "!pip install scipy --quiet\n",
+    "!pip install tenacity --quiet\n",
+    "!pip install tiktoken==0.3.3 --quiet\n",
+    "!pip install termcolor --quiet\n",
+    "!pip install openai --quiet\n",
+    "!pip install arxiv --quiet\n",
+    "!pip install pandas --quiet\n",
+    "!pip install PyPDF2 --quiet\n",
+    "!pip install tqdm --quiet"
    ]
   },
   {
@@ -151,26 +55,25 @@
    "outputs": [],
    "source": [
     "import os\n",
-    "\n",
     "import arxiv\n",
     "import ast\n",
     "import concurrent\n",
-    "from csv import writer\n",
-    "from IPython.display import display, Markdown, Latex\n",
     "import json\n",
-    "import openai\n",
     "import os\n",
     "import pandas as pd\n",
+    "import tiktoken\n",
+    "from csv import writer\n",
+    "from IPython.display import display, Markdown, Latex\n",
+    "from openai import OpenAI\n",
     "from PyPDF2 import PdfReader\n",
-    "import requests\n",
     "from scipy import spatial\n",
     "from tenacity import retry, wait_random_exponential, stop_after_attempt\n",
-    "import tiktoken\n",
     "from tqdm import tqdm\n",
     "from termcolor import colored\n",
     "\n",
     "GPT_MODEL = \"gpt-3.5-turbo-0613\"\n",
-    "EMBEDDING_MODEL = \"text-embedding-ada-002\"\n"
+    "EMBEDDING_MODEL = \"text-embedding-ada-002\"\n",
+    "client = OpenAI()"
    ]
   },
   {
@@ -191,7 +94,15 @@
    "execution_count": 3,
    "id": "2de5d32d",
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Directory './data/papers' already exists.\n"
+     ]
+    }
+   ],
    "source": [
     "directory = './data/papers'\n",
     "\n",
@@ -207,7 +118,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 4,
    "id": "ae5cb7a1",
    "metadata": {},
    "outputs": [],
@@ -223,26 +134,30 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 5,
    "id": "57217b9d",
    "metadata": {},
    "outputs": [],
    "source": [
     "@retry(wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(3))\n",
     "def embedding_request(text):\n",
-    "    response = openai.Embedding.create(input=text, model=EMBEDDING_MODEL)\n",
+    "    response = client.embeddings.create(input=text, model=EMBEDDING_MODEL)\n",
     "    return response\n",
     "\n",
     "\n",
+    "@retry(wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(3))\n",
     "def get_articles(query, library=paper_dir_filepath, top_k=5):\n",
     "    \"\"\"This function gets the top_k articles based on a user's query, sorted by relevance.\n",
     "    It also downloads the files and stores them in arxiv_library.csv to be retrieved by the read_article_and_summarize.\n",
     "    \"\"\"\n",
+    "    client = arxiv.Client()\n",
     "    search = arxiv.Search(\n",
-    "        query=query, max_results=top_k, sort_by=arxiv.SortCriterion.Relevance\n",
+    "        query = \"quantum\",\n",
+    "        max_results = 10,\n",
+    "        sort_by = arxiv.SortCriterion.SubmittedDate\n",
     "    )\n",
     "    result_list = []\n",
-    "    for result in search.results():\n",
+    "    for result in client.results(search):\n",
     "        result_dict = {}\n",
     "        result_dict.update({\"title\": result.title})\n",
     "        result_dict.update({\"summary\": result.summary})\n",
@@ -257,7 +172,7 @@
     "        file_reference = [\n",
     "            result.title,\n",
     "            result.download_pdf(data_dir),\n",
-    "            response[\"data\"][0][\"embedding\"],\n",
+    "            response.data[0].embedding,\n",
     "        ]\n",
     "\n",
     "        # Write to file\n",
@@ -270,20 +185,20 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 6,
    "id": "dda02bdb",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "{'title': 'Proximal Policy Optimization and its Dynamic Version for Sequence Generation',\n",
-       " 'summary': 'In sequence generation task, many works use policy gradient for model\\noptimization to tackle the intractable backpropagation issue when maximizing\\nthe non-differentiable evaluation metrics or fooling the discriminator in\\nadversarial learning. In this paper, we replace policy gradient with proximal\\npolicy optimization (PPO), which is a proved more efficient reinforcement\\nlearning algorithm, and propose a dynamic approach for PPO (PPO-dynamic). We\\ndemonstrate the efficacy of PPO and PPO-dynamic on conditional sequence\\ngeneration tasks including synthetic experiment and chit-chat chatbot. The\\nresults show that PPO and PPO-dynamic can beat policy gradient by stability and\\nperformance.',\n",
-       " 'article_url': 'http://arxiv.org/abs/1808.07982v1',\n",
-       " 'pdf_url': 'http://arxiv.org/pdf/1808.07982v1'}"
+       "{'title': 'Quantum types: going beyond qubits and quantum gates',\n",
+       " 'summary': 'Quantum computing is a growing field with significant potential applications.\\nLearning how to code quantum programs means understanding how qubits work and\\nlearning to use quantum gates. This is analogous to creating classical\\nalgorithms using logic gates and bits. Even after learning all concepts, it is\\ndifficult to create new algorithms, which hinders the acceptance of quantum\\nprogramming by most developers. This article outlines the need for higher-level\\nabstractions and proposes some of them in a developer-friendly programming\\nlanguage called Rhyme. The new quantum types are extensions of classical types,\\nincluding bits, integers, floats, characters, arrays, and strings. We show how\\nto use such types with code snippets.',\n",
+       " 'article_url': 'http://arxiv.org/abs/2401.15073v1',\n",
+       " 'pdf_url': 'http://arxiv.org/pdf/2401.15073v1'}"
       ]
      },
-     "execution_count": 5,
+     "execution_count": 6,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -296,7 +211,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 7,
    "id": "11675627",
    "metadata": {},
    "outputs": [],
@@ -309,7 +224,7 @@
     ") -> list[str]:\n",
     "    \"\"\"Returns a list of strings and relatednesses, sorted from most related to least.\"\"\"\n",
     "    query_embedding_response = embedding_request(query)\n",
-    "    query_embedding = query_embedding_response[\"data\"][0][\"embedding\"]\n",
+    "    query_embedding = query_embedding_response.data[0].embedding\n",
     "    strings_and_relatednesses = [\n",
     "        (row[\"filepath\"], relatedness_fn(query_embedding, row[\"embedding\"]))\n",
     "        for i, row in df.iterrows()\n",
@@ -321,7 +236,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 8,
    "id": "7211df2c",
    "metadata": {},
    "outputs": [],
@@ -362,10 +277,10 @@
     "def extract_chunk(content, template_prompt):\n",
     "    \"\"\"This function applies a prompt to some input content. In this case it returns a summarized chunk of text\"\"\"\n",
     "    prompt = template_prompt + content\n",
-    "    response = openai.ChatCompletion.create(\n",
+    "    response = client.chat.completions.create(\n",
     "        model=GPT_MODEL, messages=[{\"role\": \"user\", \"content\": prompt}], temperature=0\n",
     "    )\n",
-    "    return response[\"choices\"][0][\"message\"][\"content\"]\n",
+    "    return response.choices[0].message.content\n",
     "\n",
     "\n",
     "def summarize_text(query):\n",
@@ -418,7 +333,7 @@
     "\n",
     "    # Final summary\n",
     "    print(\"Summarizing into overall summary\")\n",
-    "    response = openai.ChatCompletion.create(\n",
+    "    response = client.chat.completions.create(\n",
     "        model=GPT_MODEL,\n",
     "        messages=[\n",
     "            {\n",
@@ -437,7 +352,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 9,
    "id": "898b94d4",
    "metadata": {},
    "outputs": [
@@ -453,7 +368,7 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "100%|████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:04<00:00,  1.19s/it]\n"
+      "100%|██████████| 6/6 [00:06<00:00,  1.08s/it]\n"
      ]
     },
     {
@@ -471,7 +386,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 10,
    "id": "c715f60d",
    "metadata": {},
    "outputs": [
@@ -480,25 +395,29 @@
      "output_type": "stream",
      "text": [
       "Core Argument:\n",
-      "- The paper discusses the use of Proximal Policy Optimization (PPO) in sequence generation tasks, specifically in the context of chit-chat chatbots.\n",
-      "- The authors argue that PPO is a more efficient reinforcement learning algorithm compared to policy gradient, commonly used in text generation tasks.\n",
-      "- They propose a dynamic approach for PPO (PPO-dynamic) and demonstrate its efficacy in synthetic experiments and chit-chat chatbot tasks.\n",
+      "- The academic paper explores the connection between the transverse field Ising (TFI) model and the ϕ4 model, highlighting the analogy between topological solitary waves in the ϕ4 model and the effect of the transverse field on spin flips in the TFI model.\n",
+      "- The study reveals regimes of memory/loss of memory and coherence/decoherence in the classical ϕ4 model subjected to periodic perturbations, which are essential in annealing phenomena.\n",
+      "- The exploration of the analogy between lower-dimensional linear quantum systems and higher-dimensional classical nonlinear systems can lead to a deeper understanding of information processing in these systems.\n",
       "\n",
       "Evidence:\n",
-      "- PPO-dynamic achieves high precision scores comparable to other algorithms in a synthetic counting task.\n",
-      "- PPO-dynamic shows faster progress and more stable learning curves compared to PPO in the synthetic counting task.\n",
-      "- In the chit-chat chatbot task, PPO-dynamic achieves a slightly higher BLEU-2 score than other algorithms.\n",
-      "- PPO and PPO-dynamic have more stable learning curves and converge faster than policy gradient.\n",
+      "- The authors analyze the dynamics and relaxation of weakly coupled ϕ4 chains through numerical simulations, observing kink and breather excitations and investigating the structural phase transition associated with the double well potential.\n",
+      "- The critical temperature (Tc) approaches zero as the inter-chain coupling strength (C⊥) approaches zero, but there is a finite Tc for C⊥>0.\n",
+      "- The spectral function shows peaks corresponding to particle motion across the double-well potential at higher temperatures and oscillations in a single well at lower temperatures.\n",
+      "- The soft-mode frequency (ωs) decreases as temperature approaches Ts, the dynamical crossover temperature.\n",
+      "- The relaxation process of the average displacement (QD) is controlled by spatially extended vibrations and large kink densities.\n",
+      "- The mean domain size (⟨DS⟩) exhibits an algebraic decay for finite C⊥>0.\n",
+      "- The probability of larger domain sizes is higher before a kick compared to after a kick for C⊥>0.\n",
       "\n",
       "Conclusions:\n",
-      "- PPO is a better optimization method for sequence learning compared to policy gradient.\n",
-      "- PPO-dynamic further improves the optimization process by dynamically adjusting hyperparameters.\n",
-      "- PPO can be used as a new optimization method for GAN-based sequence learning for better performance.\n"
+      "- The authors suggest further exploration of the crossover between decoherence and finite coherence in periodic-kick strength space.\n",
+      "- They propose extending the study to different kick profiles, introducing kink defects, and studying weakly-coupled chains in higher dimensions.\n",
+      "- Recognizing similarities between classical nonlinear equations and quantum linear ones in information processing is important.\n",
+      "- Future research directions include investigating the dynamics of quantum annealing, measurement and memory in the periodically driven complex Ginzburg-Landau equation, and the behavior of solitons and domain walls in various systems.\n"
      ]
     }
    ],
    "source": [
-    "print(chat_test_response[\"choices\"][0][\"message\"][\"content\"])\n"
+    "print(chat_test_response.choices[0].message.content)\n"
    ]
   },
   {
@@ -514,25 +433,18 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 11,
    "id": "77a6fb4f",
    "metadata": {},
    "outputs": [],
    "source": [
     "@retry(wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(3))\n",
     "def chat_completion_request(messages, functions=None, model=GPT_MODEL):\n",
-    "    headers = {\n",
-    "        \"Content-Type\": \"application/json\",\n",
-    "        \"Authorization\": \"Bearer \" + openai.api_key,\n",
-    "    }\n",
-    "    json_data = {\"model\": model, \"messages\": messages}\n",
-    "    if functions is not None:\n",
-    "        json_data.update({\"functions\": functions})\n",
     "    try:\n",
-    "        response = requests.post(\n",
-    "            \"https://api.openai.com/v1/chat/completions\",\n",
-    "            headers=headers,\n",
-    "            json=json_data,\n",
+    "        response = client.chat.completions.create(\n",
+    "            model=model,\n",
+    "            messages=messages,\n",
+    "            functions=functions,\n",
     "        )\n",
     "        return response\n",
     "    except Exception as e:\n",
@@ -543,7 +455,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 12,
    "id": "73f7672d",
    "metadata": {},
    "outputs": [],
@@ -574,7 +486,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 13,
    "id": "978b7877",
    "metadata": {},
    "outputs": [],
@@ -619,7 +531,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 14,
    "id": "0c88ae15",
    "metadata": {},
    "outputs": [],
@@ -627,23 +539,23 @@
     "def chat_completion_with_function_execution(messages, functions=[None]):\n",
     "    \"\"\"This function makes a ChatCompletion API call with the option of adding functions\"\"\"\n",
     "    response = chat_completion_request(messages, functions)\n",
-    "    full_message = response.json()[\"choices\"][0]\n",
-    "    if full_message[\"finish_reason\"] == \"function_call\":\n",
+    "    full_message = response.choices[0]\n",
+    "    if full_message.finish_reason == \"function_call\":\n",
     "        print(f\"Function generation requested, calling function\")\n",
     "        return call_arxiv_function(messages, full_message)\n",
     "    else:\n",
     "        print(f\"Function not required, responding to user\")\n",
-    "        return response.json()\n",
+    "        return response\n",
     "\n",
     "\n",
     "def call_arxiv_function(messages, full_message):\n",
     "    \"\"\"Function calling function which executes function calls when the model believes it is necessary.\n",
     "    Currently extended by adding clauses to this if statement.\"\"\"\n",
     "\n",
-    "    if full_message[\"message\"][\"function_call\"][\"name\"] == \"get_articles\":\n",
+    "    if full_message.message.function_call.name == \"get_articles\":\n",
     "        try:\n",
     "            parsed_output = json.loads(\n",
-    "                full_message[\"message\"][\"function_call\"][\"arguments\"]\n",
+    "                full_message.message.function_call.arguments\n",
     "            )\n",
     "            print(\"Getting search results\")\n",
     "            results = get_articles(parsed_output[\"query\"])\n",
@@ -654,23 +566,23 @@
     "        messages.append(\n",
     "            {\n",
     "                \"role\": \"function\",\n",
-    "                \"name\": full_message[\"message\"][\"function_call\"][\"name\"],\n",
+    "                \"name\": full_message.message.function_call.name,\n",
     "                \"content\": str(results),\n",
     "            }\n",
     "        )\n",
     "        try:\n",
     "            print(\"Got search results, summarizing content\")\n",
     "            response = chat_completion_request(messages)\n",
-    "            return response.json()\n",
+    "            return response\n",
     "        except Exception as e:\n",
     "            print(type(e))\n",
     "            raise Exception(\"Function chat request failed\")\n",
     "\n",
     "    elif (\n",
-    "        full_message[\"message\"][\"function_call\"][\"name\"] == \"read_article_and_summarize\"\n",
+    "        full_message.message.function_call.name == \"read_article_and_summarize\"\n",
     "    ):\n",
     "        parsed_output = json.loads(\n",
-    "            full_message[\"message\"][\"function_call\"][\"arguments\"]\n",
+    "            full_message.message.function_call.arguments\n",
     "        )\n",
     "        print(\"Finding and reading paper\")\n",
     "        summary = summarize_text(parsed_output[\"query\"])\n",
@@ -693,7 +605,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 15,
    "id": "c39a1d80",
    "metadata": {},
    "outputs": [],
@@ -709,7 +621,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 16,
    "id": "253fd0f7",
    "metadata": {},
    "outputs": [
@@ -718,43 +630,28 @@
      "output_type": "stream",
      "text": [
       "Function generation requested, calling function\n",
-      "Finding and reading paper\n",
-      "Chunking text from paper\n",
-      "Summarizing each chunk of text\n"
-     ]
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "100%|██████████████████████████████████████████████████████████████████████████████████| 17/17 [00:06<00:00,  2.65it/s]\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Summarizing into overall summary\n"
+      "Getting search results\n",
+      "Got search results, summarizing content\n"
      ]
     },
     {
      "data": {
       "text/markdown": [
-       "Core Argument:\n",
-       "- The paper focuses on the theoretical analysis of the PPO-Clip algorithm in the context of deep reinforcement learning.\n",
-       "- The authors propose two core ideas: reinterpreting PPO-Clip from the perspective of hinge loss and introducing a two-step policy improvement scheme.\n",
-       "- The paper establishes the global convergence of PPO-Clip and characterizes its convergence rate.\n",
+       "PPO (Proximal Policy Optimization) is a reinforcement learning algorithm that aims to find the optimal policy for an agent by optimizing the policy parameters in an iterative manner. Here are a few papers that discuss PPO in more detail:\n",
        "\n",
-       "Evidence:\n",
-       "- The paper addresses the challenges posed by the clipping mechanism and neural function approximation.\n",
-       "- The authors provide theoretical proofs, lemmas, and mathematical analysis to support their arguments.\n",
-       "- The paper presents empirical experiments on various reinforcement learning benchmark tasks to validate the effectiveness of PPO-Clip.\n",
+       "1. Title: \"Proximal Policy Optimization Algorithms\"\n",
+       "   Article URL: [arxiv.org/abs/1707.06347v2](http://arxiv.org/abs/1707.06347v2)\n",
+       "   Summary: This paper introduces two algorithms, PPO (Proximal Policy Optimization) and TRPO (Trust Region Policy Optimization), that address the issue of sample efficiency and stability in reinforcement learning. PPO uses a surrogate objective function that makes smaller updates to the policy parameters, resulting in more stable and efficient learning.\n",
        "\n",
-       "Conclusions:\n",
-       "- The paper offers theoretical insights into the performance of PPO-Clip and provides a framework for analyzing its convergence properties.\n",
-       "- PPO-Clip is shown to have a global convergence rate of O(1/sqrt(T)), where T is the number of iterations.\n",
-       "- The hinge loss reinterpretation of PPO-Clip allows for variants with comparable empirical performance.\n",
-       "- The paper contributes to a better understanding of PPO-Clip in the reinforcement learning community."
+       "2. Title: \"Emergence of Locomotion Behaviours in Rich Environments with PPO\"\n",
+       "   Article URL: [arxiv.org/abs/1707.02286v3](http://arxiv.org/abs/1707.02286v3)\n",
+       "   Summary: This paper explores the use of PPO in training agents to learn locomotion behaviors in complex and dynamic environments. The authors demonstrate the effectiveness of PPO in learning a variety of locomotion skills, such as walking, jumping, and climbing.\n",
+       "\n",
+       "3. Title: \"Proximal Policy Optimization for Multi-Agent Systems\"\n",
+       "   Article URL: [arxiv.org/abs/2006.14171v2](http://arxiv.org/abs/2006.14171v2)\n",
+       "   Summary: This paper extends PPO to the domain of multi-agent systems, where multiple agents interact and learn together. The authors propose a decentralized version of PPO that allows each agent to update its policy independently based on its local observations, resulting in more scalable and efficient learning in multi-agent environments.\n",
+       "\n",
+       "These papers provide detailed explanations of the PPO algorithm, its advantages, and its applications in different scenarios. Reading them can give you a deeper understanding of how PPO reinforcement learning works."
       ],
       "text/plain": [
        "<IPython.core.display.Markdown object>"
@@ -770,14 +667,14 @@
     "chat_response = chat_completion_with_function_execution(\n",
     "    paper_conversation.conversation_history, functions=arxiv_functions\n",
     ")\n",
-    "assistant_message = chat_response[\"choices\"][0][\"message\"][\"content\"]\n",
+    "assistant_message = chat_response.choices[0].message.content\n",
     "paper_conversation.add_message(\"assistant\", assistant_message)\n",
     "display(Markdown(assistant_message))\n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 17,
    "id": "3ca3e18a",
    "metadata": {},
    "outputs": [
@@ -795,7 +692,7 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "100%|████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:04<00:00,  1.08s/it]\n"
+      "100%|██████████| 6/6 [00:07<00:00,  1.19s/it]\n"
      ]
     },
     {
@@ -809,20 +706,24 @@
      "data": {
       "text/markdown": [
        "Core Argument:\n",
-       "- The paper discusses the use of proximal policy optimization (PPO) in sequence generation tasks, specifically in the context of chit-chat chatbots.\n",
-       "- The authors argue that PPO is a more efficient reinforcement learning algorithm compared to policy gradient, which is commonly used in text generation tasks.\n",
-       "- They propose a dynamic approach for PPO (PPO-dynamic) and demonstrate its efficacy in synthetic experiments and chit-chat chatbot tasks.\n",
+       "- The academic paper explores the connection between the transverse field Ising (TFI) model and the ϕ4 model, highlighting the analogy between the coupling of topological solitary waves in the ϕ4 model and the effect of the transverse field on spin flips in the TFI model.\n",
+       "- The study reveals regimes of memory/loss of memory and coherence/decoherence in the classical ϕ4 model subjected to periodic perturbations, which are essential in annealing phenomena.\n",
+       "- The exploration of the analogy between lower-dimensional linear quantum systems and higher-dimensional classical nonlinear systems can lead to a deeper understanding of information processing in these systems.\n",
        "\n",
        "Evidence:\n",
-       "- The authors derive the constraints for PPO-dynamic and provide the pseudo code for both PPO and PPO-dynamic.\n",
-       "- They compare the performance of PPO-dynamic with other algorithms, including REINFORCE, MIXER, and SeqGAN, on a synthetic counting task and a chit-chat chatbot task using the OpenSubtitles dataset.\n",
-       "- In the synthetic counting task, PPO-dynamic achieves a high precision score comparable to REINFORCE and MIXER, with a faster learning curve compared to PPO.\n",
-       "- In the chit-chat chatbot task, PPO-dynamic achieves a slightly higher BLEU-2 score than REINFORCE and PPO, with a more stable and faster learning curve than policy gradient.\n",
+       "- The authors analyze the dynamics and relaxation of weakly coupled ϕ4 chains through numerical simulations, studying the behavior of kink and breather excitations and the structural phase transition associated with the double well potential.\n",
+       "- The critical temperature (Tc) approaches zero as the inter-chain coupling strength (C⊥) approaches zero, but there is a finite Tc for C⊥>0.\n",
+       "- The spectral function shows peaks corresponding to particle motion across the double-well potential at higher temperatures and oscillations in a single well at lower temperatures.\n",
+       "- The soft-mode frequency (ωs) decreases as temperature approaches Ts, the dynamical crossover temperature.\n",
+       "- The relaxation process of the average displacement (QD) is controlled by spatially extended vibrations and large kink densities.\n",
+       "- The mean domain size (⟨DS⟩) exhibits an algebraic decay for finite C⊥>0.\n",
+       "- The probability of larger domain sizes is higher before a kick compared to after a kick for C⊥>0.\n",
        "\n",
        "Conclusions:\n",
-       "- The results suggest that PPO is a better optimization method for sequence learning compared to policy gradient.\n",
-       "- PPO-dynamic further improves the optimization process by dynamically adjusting the hyperparameters.\n",
-       "- The authors conclude that PPO can be used as a new optimization method for GAN-based sequence learning for better performance."
+       "- The study of weakly-coupled classical ϕ4 chains provides insights into quantum annealing architectures and the role of topological excitations in these systems.\n",
+       "- The equilibration of the system is faster for higher kick strengths, and the mean domain size increases with higher final temperatures.\n",
+       "- Further exploration of the crossover between decoherence and finite coherence in periodic-kick strength space is suggested.\n",
+       "- The paper highlights the importance of recognizing similarities between classical nonlinear equations and quantum linear ones in information processing and suggests future research directions in this area."
       ],
       "text/plain": [
        "<IPython.core.display.Markdown object>"
@@ -841,15 +742,15 @@
     "updated_response = chat_completion_with_function_execution(\n",
     "    paper_conversation.conversation_history, functions=arxiv_functions\n",
     ")\n",
-    "display(Markdown(updated_response[\"choices\"][0][\"message\"][\"content\"]))\n"
+    "display(Markdown(updated_response.choices[0].message.content))\n"
    ]
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "tua_test",
+   "display_name": "Python 3",
    "language": "python",
-   "name": "tua_test"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -861,7 +762,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.11"
+   "version": "3.12.1"
   }
  },
  "nbformat": 4,

examples/How_to_call_functions_with_chat_models.ipynb

@@ -12,7 +12,7 @@
     "\n",
     "`tools` is an optional parameter in the Chat Completion API which can be used to provide function specifications. The purpose of this is to enable models to generate function arguments which adhere to the provided specifications. Note that the API will not actually execute any function calls. It is up to developers to execute function calls using model outputs.\n",
     "\n",
-    "Within the `tools` parameter, if the `functions` parameter is provided then by default the model will decide when it is appropriate to use one of the functions. The API can be forced to use a specific function by setting the `tool_choice` parameter to `{\"name\": \"<insert-function-name>\"}`. The API can also be forced to not use any function by setting the `tool_choice` parameter to `\"none\"`. If a function is used, the output will contain `\"finish_reason\": \"function_call\"` in the response, as well as a `tool_choice` object that has the name of the function and the generated function arguments.\n",
+    "Within the `tools` parameter, if the `functions` parameter is provided then by default the model will decide when it is appropriate to use one of the functions. The API can be forced to use a specific function by setting the `tool_choice` parameter to `{\"type\": \"function\", \"function\": {\"name\": \"my_function\"}}`. The API can also be forced to not use any function by setting the `tool_choice` parameter to `\"none\"`. If a function is used, the output will contain `\"finish_reason\": \"tool_calls\"` in the response, as well as a `tool_calls` object that has the name of the function and the generated function arguments.\n",
     "\n",
     "### Overview\n",
     "\n",
@@ -33,7 +33,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 1,
    "id": "80e71f33",
    "metadata": {
     "pycharm": {
@@ -42,28 +42,27 @@
    },
    "outputs": [],
    "source": [
-    "!pip install scipy\n",
-    "!pip install tenacity\n",
-    "!pip install tiktoken\n",
-    "!pip install termcolor \n",
-    "!pip install openai\n",
-    "!pip install requests"
+    "!pip install scipy --quiet\n",
+    "!pip install tenacity --quiet\n",
+    "!pip install tiktoken --quiet\n",
+    "!pip install termcolor --quiet\n",
+    "!pip install openai --quiet"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 2,
    "id": "dab872c5",
    "metadata": {},
    "outputs": [],
    "source": [
     "import json\n",
-    "import openai\n",
-    "import requests\n",
+    "from openai import OpenAI\n",
     "from tenacity import retry, wait_random_exponential, stop_after_attempt\n",
-    "from termcolor import colored\n",
+    "from termcolor import colored  \n",
     "\n",
-    "GPT_MODEL = \"gpt-3.5-turbo-0613\""
+    "GPT_MODEL = \"gpt-3.5-turbo-0613\"\n",
+    "client = OpenAI()"
    ]
   },
   {
@@ -79,27 +78,19 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 3,
    "id": "745ceec5",
    "metadata": {},
    "outputs": [],
    "source": [
     "@retry(wait=wait_random_exponential(multiplier=1, max=40), stop=stop_after_attempt(3))\n",
     "def chat_completion_request(messages, tools=None, tool_choice=None, model=GPT_MODEL):\n",
-    "    headers = {\n",
-    "        \"Content-Type\": \"application/json\",\n",
-    "        \"Authorization\": \"Bearer \" + openai.api_key,\n",
-    "    }\n",
-    "    json_data = {\"model\": model, \"messages\": messages}\n",
-    "    if tools is not None:\n",
-    "        json_data.update({\"tools\": tools})\n",
-    "    if tool_choice is not None:\n",
-    "        json_data.update({\"tool_choice\": tool_choice})\n",
     "    try:\n",
-    "        response = requests.post(\n",
-    "            \"https://api.openai.com/v1/chat/completions\",\n",
-    "            headers=headers,\n",
-    "            json=json_data,\n",
+    "        response = client.chat.completions.create(\n",
+    "            model=model,\n",
+    "            messages=messages,\n",
+    "            tools=tools,\n",
+    "            tool_choice=tool_choice,\n",
     "        )\n",
     "        return response\n",
     "    except Exception as e:\n",
@@ -110,7 +101,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 4,
    "id": "c4d1c99f",
    "metadata": {},
    "outputs": [],
@@ -120,7 +111,7 @@
     "        \"system\": \"red\",\n",
     "        \"user\": \"green\",\n",
     "        \"assistant\": \"blue\",\n",
-    "        \"tool\": \"magenta\",\n",
+    "        \"function\": \"magenta\",\n",
     "    }\n",
     "    \n",
     "    for message in messages:\n",
@@ -132,7 +123,7 @@
     "            print(colored(f\"assistant: {message['function_call']}\\n\", role_to_color[message[\"role\"]]))\n",
     "        elif message[\"role\"] == \"assistant\" and not message.get(\"function_call\"):\n",
     "            print(colored(f\"assistant: {message['content']}\\n\", role_to_color[message[\"role\"]]))\n",
-    "        elif message[\"role\"] == \"tool\":\n",
+    "        elif message[\"role\"] == \"function\":\n",
     "            print(colored(f\"function ({message['name']}): {message['content']}\\n\", role_to_color[message[\"role\"]]))\n"
    ]
   },
@@ -149,7 +140,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 5,
    "id": "d2e25069",
    "metadata": {},
    "outputs": [],
@@ -217,18 +208,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 6,
    "id": "518d6827",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "{'role': 'assistant',\n",
-       " 'content': 'Sure, I can help you with that. Could you please tell me the city and state you are in or the location you want to know the weather for?'}"
+       "ChatCompletionMessage(content='Sure, could you please tell me the location for which you would like to know the weather?', role='assistant', function_call=None, tool_calls=None)"
       ]
      },
-     "execution_count": 5,
+     "execution_count": 6,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -240,7 +230,7 @@
     "chat_response = chat_completion_request(\n",
     "    messages, tools=tools\n",
     ")\n",
-    "assistant_message = chat_response.json()[\"choices\"][0][\"message\"]\n",
+    "assistant_message = chat_response.choices[0].message\n",
     "messages.append(assistant_message)\n",
     "assistant_message\n"
    ]
@@ -256,22 +246,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 7,
    "id": "23c42a6e",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "{'role': 'assistant',\n",
-       " 'content': None,\n",
-       " 'tool_calls': [{'id': 'call_o7uyztQLeVIoRdjcDkDJY3ni',\n",
-       "   'type': 'function',\n",
-       "   'function': {'name': 'get_current_weather',\n",
-       "    'arguments': '{\\n  \"location\": \"Glasgow, Scotland\",\\n  \"format\": \"celsius\"\\n}'}}]}"
+       "ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_2PArU89L2uf4uIzRqnph4SrN', function=Function(arguments='{\\n  \"location\": \"Glasgow, Scotland\",\\n  \"format\": \"celsius\"\\n}', name='get_current_weather'), type='function')])"
       ]
      },
-     "execution_count": 6,
+     "execution_count": 7,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -281,7 +266,7 @@
     "chat_response = chat_completion_request(\n",
     "    messages, tools=tools\n",
     ")\n",
-    "assistant_message = chat_response.json()[\"choices\"][0][\"message\"]\n",
+    "assistant_message = chat_response.choices[0].message\n",
     "messages.append(assistant_message)\n",
     "assistant_message\n"
    ]
@@ -297,18 +282,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 8,
    "id": "fa232e54",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "{'role': 'assistant',\n",
-       " 'content': 'Sure, I can help you with that. Please let me know the value for x.'}"
+       "ChatCompletionMessage(content='Sure, I can help you with that. How many days would you like to get the weather forecast for?', role='assistant', function_call=None, tool_calls=None)"
       ]
      },
-     "execution_count": 7,
+     "execution_count": 8,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -320,7 +304,7 @@
     "chat_response = chat_completion_request(\n",
     "    messages, tools=tools\n",
     ")\n",
-    "assistant_message = chat_response.json()[\"choices\"][0][\"message\"]\n",
+    "assistant_message = chat_response.choices[0].message\n",
     "messages.append(assistant_message)\n",
     "assistant_message\n"
    ]
@@ -336,24 +320,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 9,
    "id": "c7d8a543",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "{'index': 0,\n",
-       " 'message': {'role': 'assistant',\n",
-       "  'content': None,\n",
-       "  'tool_calls': [{'id': 'call_drz2YpGPWEMVySzYgsWYY249',\n",
-       "    'type': 'function',\n",
-       "    'function': {'name': 'get_n_day_weather_forecast',\n",
-       "     'arguments': '{\\n  \"location\": \"Glasgow, Scotland\",\\n  \"format\": \"celsius\",\\n  \"num_days\": 5\\n}'}}]},\n",
-       " 'finish_reason': 'tool_calls'}"
+       "Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_ujD1NwPxzeOSCbgw2NOabOin', function=Function(arguments='{\\n  \"location\": \"Glasgow, Scotland\",\\n  \"format\": \"celsius\",\\n  \"num_days\": 5\\n}', name='get_n_day_weather_forecast'), type='function')]), internal_metrics=[{'cached_prompt_tokens': 128, 'total_accepted_tokens': 0, 'total_batched_tokens': 273, 'total_predicted_tokens': 0, 'total_rejected_tokens': 0, 'total_tokens_in_completion': 274, 'cached_embeddings_bytes': 0, 'cached_embeddings_n': 0, 'uncached_embeddings_bytes': 0, 'uncached_embeddings_n': 0, 'fetched_embeddings_bytes': 0, 'fetched_embeddings_n': 0, 'n_evictions': 0, 'sampling_steps': 40, 'sampling_steps_with_predictions': 0, 'batcher_ttft': 0.035738229751586914, 'batcher_initial_queue_time': 0.0007979869842529297}])"
       ]
      },
-     "execution_count": 8,
+     "execution_count": 9,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -363,7 +340,7 @@
     "chat_response = chat_completion_request(\n",
     "    messages, tools=tools\n",
     ")\n",
-    "chat_response.json()[\"choices\"][0]\n"
+    "chat_response.choices[0]\n"
    ]
   },
   {
@@ -386,22 +363,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 10,
    "id": "559371b7",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "{'role': 'assistant',\n",
-       " 'content': None,\n",
-       " 'tool_calls': [{'id': 'call_jdmoJQ4lqsu4mBWcVBYtt5cU',\n",
-       "   'type': 'function',\n",
-       "   'function': {'name': 'get_n_day_weather_forecast',\n",
-       "    'arguments': '{\\n  \"location\": \"Toronto, Canada\",\\n  \"format\": \"celsius\",\\n  \"num_days\": 1\\n}'}}]}"
+       "ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_MapM0kaNZBR046H4tAB2UGVu', function=Function(arguments='{\\n  \"location\": \"Toronto, Canada\",\\n  \"format\": \"celsius\",\\n  \"num_days\": 1\\n}', name='get_n_day_weather_forecast'), type='function')])"
       ]
      },
-     "execution_count": 9,
+     "execution_count": 10,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -414,27 +386,22 @@
     "chat_response = chat_completion_request(\n",
     "    messages, tools=tools, tool_choice={\"type\": \"function\", \"function\": {\"name\": \"get_n_day_weather_forecast\"}}\n",
     ")\n",
-    "chat_response.json()[\"choices\"][0][\"message\"]\n"
+    "chat_response.choices[0].message"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 11,
    "id": "a7ab0f58",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "{'role': 'assistant',\n",
-       " 'content': None,\n",
-       " 'tool_calls': [{'id': 'call_RYXaDjxpUCfWmpXU7BZEYVqS',\n",
-       "   'type': 'function',\n",
-       "   'function': {'name': 'get_current_weather',\n",
-       "    'arguments': '{\\n  \"location\": \"Toronto, Canada\",\\n  \"format\": \"celsius\"\\n}'}}]}"
+       "ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_z8ijGSoMLS7xcaU7MjLmpRL8', function=Function(arguments='{\\n  \"location\": \"Toronto, Canada\",\\n  \"format\": \"celsius\"\\n}', name='get_current_weather'), type='function')])"
       ]
      },
-     "execution_count": 10,
+     "execution_count": 11,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -447,7 +414,7 @@
     "chat_response = chat_completion_request(\n",
     "    messages, tools=tools\n",
     ")\n",
-    "chat_response.json()[\"choices\"][0][\"message\"]\n"
+    "chat_response.choices[0].message"
    ]
   },
   {
@@ -461,18 +428,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 12,
    "id": "acfe54e6",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "{'role': 'assistant',\n",
-       " 'content': '{ \"location\": \"Toronto, Canada\", \"format\": \"celsius\" }'}"
+       "ChatCompletionMessage(content='{\\n  \"location\": \"Toronto, Canada\",\\n  \"format\": \"celsius\"\\n}', role='assistant', function_call=None, tool_calls=None)"
       ]
      },
-     "execution_count": 11,
+     "execution_count": 12,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -484,11 +450,12 @@
     "chat_response = chat_completion_request(\n",
     "    messages, tools=tools, tool_choice=\"none\"\n",
     ")\n",
-    "chat_response.json()[\"choices\"][0][\"message\"]\n"
+    "chat_response.choices[0].message\n"
    ]
   },
   {
    "cell_type": "markdown",
+   "id": "b616353b",
    "metadata": {},
    "source": [
     "### Parallel Function Calling\n",
@@ -498,23 +465,18 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 13,
+   "id": "380eeb68",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "[{'id': 'call_fLsKR5vGllhbWxvpqsDT3jBj',\n",
-       "  'type': 'function',\n",
-       "  'function': {'name': 'get_n_day_weather_forecast',\n",
-       "   'arguments': '{\"location\": \"San Francisco, CA\", \"format\": \"celsius\", \"num_days\": 4}'}},\n",
-       " {'id': 'call_CchlsGE8OE03QmeyFbg7pkDz',\n",
-       "  'type': 'function',\n",
-       "  'function': {'name': 'get_n_day_weather_forecast',\n",
-       "   'arguments': '{\"location\": \"Glasgow\", \"format\": \"celsius\", \"num_days\": 4}'}}]"
+       "[ChatCompletionMessageToolCall(id='call_8BlkS2yvbkkpL3V1Yxc6zR6u', function=Function(arguments='{\"location\": \"San Francisco, CA\", \"format\": \"celsius\", \"num_days\": 4}', name='get_n_day_weather_forecast'), type='function'),\n",
+       " ChatCompletionMessageToolCall(id='call_vSZMy3f24wb3vtNXucpFfAbG', function=Function(arguments='{\"location\": \"Glasgow\", \"format\": \"celsius\", \"num_days\": 4}', name='get_n_day_weather_forecast'), type='function')]"
       ]
      },
-     "execution_count": 12,
+     "execution_count": 13,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -527,8 +489,7 @@
     "    messages, tools=tools, model='gpt-3.5-turbo-1106'\n",
     ")\n",
     "\n",
-    "chat_response.json()\n",
-    "assistant_message = chat_response.json()[\"choices\"][0][\"message\"]['tool_calls']\n",
+    "assistant_message = chat_response.choices[0].message.tool_calls\n",
     "assistant_message"
    ]
   },
@@ -558,7 +519,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 14,
    "id": "30f6b60e",
    "metadata": {},
    "outputs": [
@@ -579,7 +540,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 15,
    "id": "abec0214",
    "metadata": {},
    "outputs": [],
@@ -622,7 +583,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 16,
    "id": "0c0104cd",
    "metadata": {},
    "outputs": [],
@@ -647,7 +608,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 17,
    "id": "0258813a",
    "metadata": {},
    "outputs": [],
@@ -691,7 +652,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": 18,
    "id": "65585e74",
    "metadata": {},
    "outputs": [],
@@ -705,17 +666,17 @@
     "    return results\n",
     "\n",
     "def execute_function_call(message):\n",
-    "    if message[\"tool_calls\"][0][\"function\"][\"name\"] == \"ask_database\":\n",
-    "        query = json.loads(message[\"tool_calls\"][0][\"function\"][\"arguments\"])[\"query\"]\n",
+    "    if message.tool_calls[0].function.name == \"ask_database\":\n",
+    "        query = json.loads(message.tool_calls[0].function.arguments)[\"query\"]\n",
     "        results = ask_database(conn, query)\n",
     "    else:\n",
-    "        results = f\"Error: function {message['tool_calls'][0]['function']['name']} does not exist\"\n",
+    "        results = f\"Error: function {message.tool_calls[0].function.name} does not exist\"\n",
     "    return results"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 19,
    "id": "38c55083",
    "metadata": {},
    "outputs": [
@@ -727,7 +688,7 @@
       "\u001b[0m\n",
       "\u001b[32muser: Hi, who are the top 5 artists by number of tracks?\n",
       "\u001b[0m\n",
-      "\u001b[34massistant: {'name': 'ask_database', 'arguments': '{\\n  \"query\": \"SELECT Artist.Name, COUNT(Track.TrackId) AS TrackCount FROM Artist JOIN Album ON Artist.ArtistId = Album.ArtistId JOIN Track ON Album.AlbumId = Track.AlbumId GROUP BY Artist.Name ORDER BY TrackCount DESC LIMIT 5\"\\n}'}\n",
+      "\u001b[34massistant: Function(arguments='{\\n  \"query\": \"SELECT Artist.Name, COUNT(Track.TrackId) AS TrackCount FROM Artist JOIN Album ON Artist.ArtistId = Album.ArtistId JOIN Track ON Album.AlbumId = Track.AlbumId GROUP BY Artist.ArtistId ORDER BY TrackCount DESC LIMIT 5;\"\\n}', name='ask_database')\n",
       "\u001b[0m\n",
       "\u001b[35mfunction (ask_database): [('Iron Maiden', 213), ('U2', 135), ('Led Zeppelin', 114), ('Metallica', 112), ('Lost', 92)]\n",
       "\u001b[0m\n"
@@ -739,18 +700,18 @@
     "messages.append({\"role\": \"system\", \"content\": \"Answer user questions by generating SQL queries against the Chinook Music Database.\"})\n",
     "messages.append({\"role\": \"user\", \"content\": \"Hi, who are the top 5 artists by number of tracks?\"})\n",
     "chat_response = chat_completion_request(messages, tools)\n",
-    "assistant_message = chat_response.json()[\"choices\"][0][\"message\"]\n",
-    "assistant_message['content'] = str(assistant_message[\"tool_calls\"][0][\"function\"])\n",
-    "messages.append(assistant_message)\n",
-    "if assistant_message.get(\"tool_calls\"):\n",
+    "assistant_message = chat_response.choices[0].message\n",
+    "assistant_message.content = str(assistant_message.tool_calls[0].function)\n",
+    "messages.append({\"role\": assistant_message.role, \"content\": assistant_message.content})\n",
+    "if assistant_message.tool_calls:\n",
     "    results = execute_function_call(assistant_message)\n",
-    "    messages.append({\"role\": \"tool\", \"tool_call_id\": assistant_message[\"tool_calls\"][0]['id'], \"name\": assistant_message[\"tool_calls\"][0][\"function\"][\"name\"], \"content\": results})\n",
+    "    messages.append({\"role\": \"function\", \"tool_call_id\": assistant_message.tool_calls[0].id, \"name\": assistant_message.tool_calls[0].function.name, \"content\": results})\n",
     "pretty_print_conversation(messages)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 20,
    "id": "710481dc",
    "metadata": {
     "scrolled": true
@@ -764,13 +725,13 @@
       "\u001b[0m\n",
       "\u001b[32muser: Hi, who are the top 5 artists by number of tracks?\n",
       "\u001b[0m\n",
-      "\u001b[34massistant: {'name': 'ask_database', 'arguments': '{\\n  \"query\": \"SELECT Artist.Name, COUNT(Track.TrackId) AS TrackCount FROM Artist JOIN Album ON Artist.ArtistId = Album.ArtistId JOIN Track ON Album.AlbumId = Track.AlbumId GROUP BY Artist.Name ORDER BY TrackCount DESC LIMIT 5\"\\n}'}\n",
+      "\u001b[34massistant: Function(arguments='{\\n  \"query\": \"SELECT Artist.Name, COUNT(Track.TrackId) AS TrackCount FROM Artist JOIN Album ON Artist.ArtistId = Album.ArtistId JOIN Track ON Album.AlbumId = Track.AlbumId GROUP BY Artist.ArtistId ORDER BY TrackCount DESC LIMIT 5;\"\\n}', name='ask_database')\n",
       "\u001b[0m\n",
       "\u001b[35mfunction (ask_database): [('Iron Maiden', 213), ('U2', 135), ('Led Zeppelin', 114), ('Metallica', 112), ('Lost', 92)]\n",
       "\u001b[0m\n",
       "\u001b[32muser: What is the name of the album with the most tracks?\n",
       "\u001b[0m\n",
-      "\u001b[34massistant: {'name': 'ask_database', 'arguments': '{\\n  \"query\": \"SELECT Album.Title, COUNT(Track.TrackId) AS TrackCount FROM Album JOIN Track ON Album.AlbumId = Track.AlbumId GROUP BY Album.Title ORDER BY TrackCount DESC LIMIT 1\"\\n}'}\n",
+      "\u001b[34massistant: Function(arguments='{\\n  \"query\": \"SELECT Album.Title, COUNT(Track.TrackId) AS TrackCount FROM Album JOIN Track ON Album.AlbumId = Track.AlbumId GROUP BY Album.AlbumId ORDER BY TrackCount DESC LIMIT 1;\"\\n}', name='ask_database')\n",
       "\u001b[0m\n",
       "\u001b[35mfunction (ask_database): [('Greatest Hits', 57)]\n",
       "\u001b[0m\n"
@@ -780,12 +741,12 @@
    "source": [
     "messages.append({\"role\": \"user\", \"content\": \"What is the name of the album with the most tracks?\"})\n",
     "chat_response = chat_completion_request(messages, tools)\n",
-    "assistant_message = chat_response.json()[\"choices\"][0][\"message\"]\n",
-    "assistant_message['content'] = str(assistant_message[\"tool_calls\"][0][\"function\"])\n",
-    "messages.append(assistant_message)\n",
-    "if assistant_message.get(\"tool_calls\"):\n",
+    "assistant_message = chat_response.choices[0].message\n",
+    "assistant_message.content = str(assistant_message.tool_calls[0].function)\n",
+    "messages.append({\"role\": assistant_message.role, \"content\": assistant_message.content})\n",
+    "if assistant_message.tool_calls:\n",
     "    results = execute_function_call(assistant_message)\n",
-    "    messages.append({\"role\": \"tool\", \"tool_call_id\": assistant_message[\"tool_calls\"][0]['id'], \"name\": assistant_message[\"tool_calls\"][0][\"function\"][\"name\"], \"content\": results})\n",
+    "    messages.append({\"role\": \"function\", \"tool_call_id\": assistant_message.tool_calls[0].id, \"name\": assistant_message.tool_calls[0].function.name, \"content\": results})\n",
     "pretty_print_conversation(messages)"
    ]
   },
@@ -817,7 +778,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.6"
+   "version": "3.12.1"
   }
  },
  "nbformat": 4,

examples/How_to_count_tokens_with_tiktoken.ipynb

@@ -22,7 +22,7 @@
     "\n",
     "| Encoding name           | OpenAI models                                       |\n",
     "|-------------------------|-----------------------------------------------------|\n",
-    "| `cl100k_base`           | `gpt-4`, `gpt-3.5-turbo`, `text-embedding-ada-002`  |\n",
+    "| `cl100k_base`           | `gpt-4`, `gpt-3.5-turbo`, `text-embedding-ada-002`, `text-embedding-3-small`, `text-embedding-3-large`  |\n",
     "| `p50k_base`             | Codex models, `text-davinci-002`, `text-davinci-003`|\n",
     "| `r50k_base` (or `gpt2`) | GPT-3 models like `davinci`                         |\n",
     "\n",
@@ -89,7 +89,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -433,7 +433,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -480,7 +480,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
@@ -519,7 +519,10 @@
    "source": [
     "# let's verify the function above matches the OpenAI API response\n",
     "\n",
-    "import openai\n",
+    "from openai import OpenAI\n",
+    "import os\n",
+    "\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
     "\n",
     "example_messages = [\n",
     "    {\n",
@@ -564,15 +567,20 @@
     "    # example token count from the function defined above\n",
     "    print(f\"{num_tokens_from_messages(example_messages, model)} prompt tokens counted by num_tokens_from_messages().\")\n",
     "    # example token count from the OpenAI API\n",
-    "    response = openai.ChatCompletion.create(\n",
-    "        model=model,\n",
-    "        messages=example_messages,\n",
-    "        temperature=0,\n",
-    "        max_tokens=1,  # we're only counting input tokens here, so let's not waste tokens on the output\n",
-    "    )\n",
-    "    print(f'{response[\"usage\"][\"prompt_tokens\"]} prompt tokens counted by the OpenAI API.')\n",
+    "    response = client.chat.completions.create(model=model,\n",
+    "    messages=example_messages,\n",
+    "    temperature=0,\n",
+    "    max_tokens=1)\n",
+    "    print(f'{response.usage.prompt_tokens} prompt tokens counted by the OpenAI API.')\n",
     "    print()\n"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -591,7 +599,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.13"
+   "version": "3.11.5"
   },
   "vscode": {
    "interpreter": {

examples/How_to_finetune_chat_models.ipynb

@@ -53,8 +53,7 @@
     "import pandas as pd\n",
     "from pprint import pprint\n",
     "\n",
-    "OPENAI_API_KEY = os.getenv(\"OPENAI_API_KEY\", \"\")\n",
-    "openai.api_key = OPENAI_API_KEY"
+    "client = openai.OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {
@@ -421,14 +420,14 @@
    ],
    "source": [
     "with open(training_file_name, \"rb\") as training_fd:\n",
-    "    training_response = openai.files.create(\n",
+    "    training_response = client.files.create(\n",
     "        file=training_fd, purpose=\"fine-tune\"\n",
     "    )\n",
     "\n",
     "training_file_id = training_response.id\n",
     "\n",
     "with open(validation_file_name, \"rb\") as validation_fd:\n",
-    "    validation_response = openai.files.create(\n",
+    "    validation_response = client.files.create(\n",
     "        file=validation_fd, purpose=\"fine-tune\"\n",
     "    )\n",
     "validation_file_id = validation_response.id\n",
@@ -465,7 +464,7 @@
     }
    ],
    "source": [
-    "response = openai.fine_tuning.jobs.create(\n",
+    "response = client.fine_tuning.jobs.create(\n",
     "    training_file=training_file_id,\n",
     "    validation_file=validation_file_id,\n",
     "    model=\"gpt-3.5-turbo\",\n",
@@ -507,7 +506,7 @@
     }
    ],
    "source": [
-    "response = openai.fine_tuning.jobs.retrieve(job_id)\n",
+    "response = client.fine_tuning.jobs.retrieve(job_id)\n",
     "\n",
     "print(\"Job ID:\", response.id)\n",
     "print(\"Status:\", response.status)\n",
@@ -556,7 +555,7 @@
     }
    ],
    "source": [
-    "response = openai.fine_tuning.jobs.list_events(job_id)\n",
+    "response = client.fine_tuning.jobs.list_events(job_id)\n",
     "\n",
     "events = response.data\n",
     "events.reverse()\n",
@@ -588,7 +587,7 @@
     }
    ],
    "source": [
-    "response = openai.fine_tuning.jobs.retrieve(job_id)\n",
+    "response = client.fine_tuning.jobs.retrieve(job_id)\n",
     "fine_tuned_model_id = response.fine_tuned_model\n",
     "\n",
     "if fine_tuned_model_id is None: \n",
@@ -644,7 +643,7 @@
     "test_messages = []\n",
     "test_messages.append({\"role\": \"system\", \"content\": system_message})\n",
     "user_message = create_user_message(test_row)\n",
-    "test_messages.append({\"role\": \"user\", \"content\": create_user_message(test_row)})\n",
+    "test_messages.append({\"role\": \"user\", \"content\": user_message})\n",
     "\n",
     "pprint(test_messages)"
    ]
@@ -664,7 +663,7 @@
     }
    ],
    "source": [
-    "response = openai.chat.completions.create(\n",
+    "response = client.chat.completions.create(\n",
     "    model=fine_tuned_model_id, messages=test_messages, temperature=0, max_tokens=500\n",
     ")\n",
     "print(response.choices[0].message.content)"

examples/How_to_format_inputs_to_ChatGPT_models.ipynb

@@ -36,12 +36,15 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 22,
    "metadata": {},
    "outputs": [],
    "source": [
     "# import the OpenAI Python library for calling the OpenAI API\n",
-    "import openai\n"
+    "from openai import OpenAI\n",
+    "import os\n",
+    "\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {
@@ -49,17 +52,37 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 2. An example chat API call\n",
+    "## 2. An example chat completion API call\n",
     "\n",
-    "A chat API call has two required inputs:\n",
-    "- `model`: the name of the model you want to use (e.g., `gpt-3.5-turbo`, `gpt-4`, `gpt-3.5-turbo-0613`, `gpt-3.5-turbo-16k-0613`)\n",
+    "A chat completion API call parameters,\n",
+    "**Required**\n",
+    "- `model`: the name of the model you want to use (e.g., `gpt-3.5-turbo`, `gpt-4`, `gpt-3.5-turbo-16k-1106`)\n",
     "- `messages`: a list of message objects, where each object has two required fields:\n",
-    "    - `role`: the role of the messenger (either `system`, `user`, or `assistant`)\n",
+    "    - `role`: the role of the messenger (either `system`, `user`, `assistant` or `tool`)\n",
     "    - `content`: the content of the message (e.g., `Write me a beautiful poem`)\n",
     "\n",
     "Messages can also contain an optional `name` field, which give the messenger a name. E.g., `example-user`, `Alice`, `BlackbeardBot`. Names may not contain spaces.\n",
     "\n",
-    "As of June 2023, you can also optionally submit a list of `functions` that tell GPT whether it can generate JSON to feed into a function. For details, see the [documentation](https://platform.openai.com/docs/guides/gpt/function-calling), [API reference](https://platform.openai.com/docs/api-reference/chat), or the Cookbook guide [How to call functions with chat models](How_to_call_functions_with_chat_models.ipynb).\n",
+    "**Optional**\n",
+    "- `frequency_penalty`: Penalizes tokens based on their frequency, reducing repetition.\n",
+    "- `logit_bias`: Modifies likelihood of specified tokens with bias values.\n",
+    "- `logprobs`: Returns log probabilities of output tokens if true.\n",
+    "- `top_logprobs`: Specifies the number of most likely tokens to return at each position.\n",
+    "- `max_tokens`: Sets the maximum number of generated tokens in chat completion.\n",
+    "- `n`: Generates a specified number of chat completion choices for each input.\n",
+    "- `presence_penalty`: Penalizes new tokens based on their presence in the text.\n",
+    "- `response_format`: Specifies the output format, e.g., JSON mode.\n",
+    "- `seed`: Ensures deterministic sampling with a specified seed.\n",
+    "- `stop`: Specifies up to 4 sequences where the API should stop generating tokens.\n",
+    "- `stream`: Sends partial message deltas as tokens become available.\n",
+    "- `temperature`: Sets the sampling temperature between 0 and 2.\n",
+    "- `top_p`: Uses nucleus sampling; considers tokens with top_p probability mass.\n",
+    "- `tools`: Lists functions the model may call.\n",
+    "- `tool_choice`: Controls the model's function calls (none/auto/function).\n",
+    "- `user`: Unique identifier for end-user monitoring and abuse detection.\n",
+    "\n",
+    "\n",
+    "As of January 2024, you can also optionally submit a list of `functions` that tell GPT whether it can generate JSON to feed into a function. For details, see the [documentation](https://platform.openai.com/docs/guides/function-calling), [API reference](https://platform.openai.com/docs/api-reference/chat), or the Cookbook guide [How to call functions with chat models](How_to_call_functions_with_chat_models.ipynb).\n",
     "\n",
     "Typically, a conversation will start with a system message that tells the assistant how to behave, followed by alternating user and assistant messages, but you are not required to follow this format.\n",
     "\n",
@@ -68,44 +91,13 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 2,
    "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "<OpenAIObject chat.completion id=chatcmpl-7UkgnSDzlevZxiy0YjZcLYdUMz5yZ at 0x118e394f0> JSON: {\n",
-       "  \"id\": \"chatcmpl-7UkgnSDzlevZxiy0YjZcLYdUMz5yZ\",\n",
-       "  \"object\": \"chat.completion\",\n",
-       "  \"created\": 1687563669,\n",
-       "  \"model\": \"gpt-3.5-turbo-0301\",\n",
-       "  \"choices\": [\n",
-       "    {\n",
-       "      \"index\": 0,\n",
-       "      \"message\": {\n",
-       "        \"role\": \"assistant\",\n",
-       "        \"content\": \"Orange who?\"\n",
-       "      },\n",
-       "      \"finish_reason\": \"stop\"\n",
-       "    }\n",
-       "  ],\n",
-       "  \"usage\": {\n",
-       "    \"prompt_tokens\": 39,\n",
-       "    \"completion_tokens\": 3,\n",
-       "    \"total_tokens\": 42\n",
-       "  }\n",
-       "}"
-      ]
-     },
-     "execution_count": 3,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
     "# Example OpenAI Python library request\n",
     "MODEL = \"gpt-3.5-turbo\"\n",
-    "response = openai.ChatCompletion.create(\n",
+    "response = client.chat.completions.create(\n",
     "    model=MODEL,\n",
     "    messages=[\n",
     "        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
@@ -114,9 +106,48 @@
     "        {\"role\": \"user\", \"content\": \"Orange.\"},\n",
     "    ],\n",
     "    temperature=0,\n",
-    ")\n",
-    "\n",
-    "response\n"
+    ")\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{\n",
+      "    \"id\": \"chatcmpl-8dee9DuEFcg2QILtT2a6EBXZnpirM\",\n",
+      "    \"choices\": [\n",
+      "        {\n",
+      "            \"finish_reason\": \"stop\",\n",
+      "            \"index\": 0,\n",
+      "            \"logprobs\": null,\n",
+      "            \"message\": {\n",
+      "                \"content\": \"Orange who?\",\n",
+      "                \"role\": \"assistant\",\n",
+      "                \"function_call\": null,\n",
+      "                \"tool_calls\": null\n",
+      "            }\n",
+      "        }\n",
+      "    ],\n",
+      "    \"created\": 1704461729,\n",
+      "    \"model\": \"gpt-3.5-turbo-0613\",\n",
+      "    \"object\": \"chat.completion\",\n",
+      "    \"system_fingerprint\": null,\n",
+      "    \"usage\": {\n",
+      "        \"completion_tokens\": 3,\n",
+      "        \"prompt_tokens\": 35,\n",
+      "        \"total_tokens\": 38\n",
+      "    }\n",
+      "}\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(json.dumps(json.loads(response.model_dump_json()), indent=4))"
    ]
   },
   {
@@ -126,14 +157,19 @@
    "source": [
     "As you can see, the response object has a few fields:\n",
     "- `id`: the ID of the request\n",
-    "- `object`: the type of object returned (e.g., `chat.completion`)\n",
-    "- `created`: the timestamp of the request\n",
-    "- `model`: the full name of the model used to generate the response\n",
-    "- `usage`: the number of tokens used to generate the replies, counting prompt, completion, and total\n",
     "- `choices`: a list of completion objects (only one, unless you set `n` greater than 1)\n",
-    "    - `message`: the message object generated by the model, with `role` and `content`\n",
     "    - `finish_reason`: the reason the model stopped generating text (either `stop`, or `length` if `max_tokens` limit was reached)\n",
-    "    - `index`: the index of the completion in the list of choices"
+    "    - `index`: The index of the choice in the list of choices.\n",
+    "    - `logprobs`: Log probability information for the choice.\n",
+    "    - `message`: the message object generated by the model\n",
+    "        - `content`: content of message\n",
+    "        - `role`: The role of the author of this message.\n",
+    "        - `tool_calls`: The tool calls generated by the model, such as function calls. if the tools is given\n",
+    "- `created`: the timestamp of the request\n",
+    "- `model`: the full name of the model used to generate the response\n",
+    "- `object`: the type of object returned (e.g., `chat.completion`)\n",
+    "- `system_fingerprint`: This fingerprint represents the backend configuration that the model runs with.\n",
+    "- `usage`: the number of tokens used to generate the replies, counting prompt, completion, and total"
    ]
   },
   {
@@ -146,7 +182,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 9,
    "metadata": {},
    "outputs": [
     {
@@ -155,13 +191,13 @@
        "'Orange who?'"
       ]
      },
-     "execution_count": 4,
+     "execution_count": 9,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "response['choices'][0]['message']['content']\n"
+    "response.choices[0].message.content\n"
    ]
   },
   {
@@ -176,20 +212,36 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 10,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Ahoy matey! Asynchronous programming be like havin' a crew o' pirates workin' on different tasks at the same time. Ye see, instead o' waitin' for one task to be completed before startin' the next, ye can assign tasks to yer crew and let 'em work on 'em simultaneously. This way, ye can get more done in less time and keep yer ship sailin' smoothly. It be like havin' a bunch o' pirates rowin' the ship at different speeds, but still gettin' us to our destination. Arrr!\n"
+      "Arr, me matey! Let me tell ye a tale of asynchronous programming, in the style of the fearsome pirate Blackbeard!\n",
+      "\n",
+      "Picture this, me hearties. In the vast ocean of programming, there be times when ye need to perform multiple tasks at once. But fear not, for asynchronous programming be here to save the day!\n",
+      "\n",
+      "Ye see, in traditional programming, ye be waitin' for one task to be done before movin' on to the next. But with asynchronous programming, ye can be takin' care of multiple tasks at the same time, just like a pirate multitaskin' on the high seas!\n",
+      "\n",
+      "Instead of waitin' for a task to be completed, ye can be sendin' it off on its own journey, while ye move on to the next task. It be like havin' a crew of trusty sailors, each takin' care of their own duties, without waitin' for the others.\n",
+      "\n",
+      "Now, ye may be wonderin', how does this sorcery work? Well, me matey, it be all about callbacks and promises. When ye be sendin' off a task, ye be attachin' a callback function to it. This be like leavin' a message in a bottle, tellin' the task what to do when it be finished.\n",
+      "\n",
+      "While the task be sailin' on its own, ye can be movin' on to the next task, without wastin' any precious time. And when the first task be done, it be sendin' a signal back to ye, lettin' ye know it be finished. Then ye can be takin' care of the callback function, like openin' the bottle and readin' the message inside.\n",
+      "\n",
+      "But wait, there be more! With promises, ye can be makin' even fancier arrangements. Instead of callbacks, ye be makin' a promise that the task will be completed. It be like a contract between ye and the task, swearin' that it will be done.\n",
+      "\n",
+      "Ye can be attachin' multiple promises to a task, promisin' different outcomes. And when the task be finished, it be fulfillin' the promises, lettin' ye know it be done. Then ye can be handlin' the fulfillments, like collectin' the rewards of yer pirate adventures!\n",
+      "\n",
+      "So, me hearties, that be the tale of asynchronous programming, told in the style of the fearsome pirate Blackbeard! With callbacks and promises, ye can be takin' care of multiple tasks at once, just like a pirate conquerin' the seven seas!\n"
      ]
     }
    ],
    "source": [
     "# example with a system message\n",
-    "response = openai.ChatCompletion.create(\n",
+    "response = client.chat.completions.create(\n",
     "    model=MODEL,\n",
     "    messages=[\n",
     "        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
@@ -198,31 +250,43 @@
     "    temperature=0,\n",
     ")\n",
     "\n",
-    "print(response['choices'][0]['message']['content'])\n"
+    "print(response.choices[0].message.content)\n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 11,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Ahoy mateys! Let me tell ye about asynchronous programming, arrr! It be like havin' a crew of sailors workin' on different tasks at the same time, without waitin' for each other to finish. Ye see, in traditional programming, ye have to wait for one task to be completed before movin' on to the next. But with asynchronous programming, ye can start multiple tasks at once and let them run in the background while ye focus on other things.\n",
+      "Arr, me hearties! Gather 'round and listen up, for I be tellin' ye about the mysterious art of asynchronous programming, in the style of the fearsome pirate Blackbeard!\n",
       "\n",
-      "It be like havin' a lookout keepin' watch for enemy ships while the rest of the crew be busy with their own tasks. They don't have to stop what they're doin' to keep an eye out, because the lookout be doin' it for them. And when the lookout spots an enemy ship, they can alert the crew and everyone can work together to defend the ship.\n",
+      "Now, ye see, in the world of programming, there be times when we need to perform tasks that take a mighty long time to complete. These tasks might involve fetchin' data from the depths of the internet, or performin' complex calculations that would make even Davy Jones scratch his head.\n",
       "\n",
-      "In the same way, asynchronous programming allows different parts of yer code to work together without gettin' in each other's way. It be especially useful for tasks that take a long time to complete, like loadin' large files or connectin' to a server. Instead of makin' yer program wait for these tasks to finish, ye can let them run in the background while yer program continues to do other things.\n",
+      "In the olden days, we pirates used to wait patiently for each task to finish afore movin' on to the next one. But that be a waste of precious time, me hearties! We be pirates, always lookin' for ways to be more efficient and plunder more booty!\n",
       "\n",
-      "So there ye have it, me hearties! Asynchronous programming be like havin' a crew of sailors workin' together without gettin' in each other's way. It be a powerful tool for any programmer, and one that can help ye sail the seas of code with ease!\n"
+      "That be where asynchronous programming comes in, me mateys. It be a way to tackle multiple tasks at once, without waitin' for each one to finish afore movin' on. It be like havin' a crew of scallywags workin' on different tasks simultaneously, while ye be overseein' the whole operation.\n",
+      "\n",
+      "Ye see, in asynchronous programming, we be breakin' down our tasks into smaller chunks called \"coroutines.\" Each coroutine be like a separate pirate, workin' on its own task. When a coroutine be startin' its work, it don't wait for the task to finish afore movin' on to the next one. Instead, it be movin' on to the next task, lettin' the first one continue in the background.\n",
+      "\n",
+      "Now, ye might be wonderin', \"But Blackbeard, how be we know when a task be finished if we don't wait for it?\" Ah, me hearties, that be where the magic of callbacks and promises come in!\n",
+      "\n",
+      "When a coroutine be startin' its work, it be attachin' a callback or a promise to it. This be like leavin' a message in a bottle, tellin' the coroutine what to do when it be finished. So, while the coroutine be workin' away, the rest of the crew be movin' on to other tasks, plunderin' more booty along the way.\n",
+      "\n",
+      "When a coroutine be finished with its task, it be sendin' a signal to the callback or fulfillin' the promise, lettin' the rest of the crew know that it be done. Then, the crew can gather 'round and handle the results of the completed task, celebratin' their victory and countin' their plunder.\n",
+      "\n",
+      "So, me hearties, asynchronous programming be like havin' a crew of pirates workin' on different tasks at once, without waitin' for each one to finish afore movin' on. It be a way to be more efficient, plunder more booty, and conquer the vast seas of programming!\n",
+      "\n",
+      "Now, set sail, me mateys, and embrace the power of asynchronous programming like true pirates of the digital realm! Arr!\n"
      ]
     }
    ],
    "source": [
     "# example without a system message\n",
-    "response = openai.ChatCompletion.create(\n",
+    "response = client.chat.completions.create(\n",
     "    model=MODEL,\n",
     "    messages=[\n",
     "        {\"role\": \"user\", \"content\": \"Explain asynchronous programming in the style of the pirate Blackbeard.\"},\n",
@@ -230,7 +294,7 @@
     "    temperature=0,\n",
     ")\n",
     "\n",
-    "print(response['choices'][0]['message']['content'])\n"
+    "print(response.choices[0].message.content)\n"
    ]
   },
   {
@@ -257,28 +321,28 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 12,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Sure! Fractions are a way of representing a part of a whole. The top number of a fraction is called the numerator, and it represents how many parts of the whole we are talking about. The bottom number is called the denominator, and it represents how many equal parts the whole is divided into.\n",
+      "Of course! Fractions are a way to represent parts of a whole. They are made up of two numbers: a numerator and a denominator. The numerator tells you how many parts you have, and the denominator tells you how many equal parts make up the whole.\n",
       "\n",
-      "For example, if we have a pizza that is divided into 8 equal slices, and we take 3 slices, we can represent that as the fraction 3/8. The numerator is 3 because we took 3 slices, and the denominator is 8 because the pizza was divided into 8 slices.\n",
+      "Let's take an example to understand this better. Imagine you have a pizza that is divided into 8 equal slices. If you eat 3 slices, you can represent that as the fraction 3/8. Here, the numerator is 3 because you ate 3 slices, and the denominator is 8 because the whole pizza is divided into 8 slices.\n",
       "\n",
-      "To add or subtract fractions, we need to have a common denominator. This means that the denominators of the fractions need to be the same. To do this, we can find the least common multiple (LCM) of the denominators and then convert each fraction to an equivalent fraction with the LCM as the denominator.\n",
+      "Fractions can also be used to represent numbers less than 1. For example, if you eat half of a pizza, you can write it as 1/2. Here, the numerator is 1 because you ate one slice, and the denominator is 2 because the whole pizza is divided into 2 equal parts.\n",
       "\n",
-      "To multiply fractions, we simply multiply the numerators together and the denominators together. To divide fractions, we multiply the first fraction by the reciprocal of the second fraction (flip the second fraction upside down).\n",
+      "Now, let's talk about equivalent fractions. Equivalent fractions are different fractions that represent the same amount. For example, 1/2 and 2/4 are equivalent fractions because they both represent half of something. To find equivalent fractions, you can multiply or divide both the numerator and denominator by the same number.\n",
       "\n",
-      "Now, here's a question to check for understanding: If we have a pizza that is divided into 12 equal slices, and we take 4 slices, what is the fraction that represents how much of the pizza we took?\n"
+      "Here's a question to check your understanding: If you have a cake divided into 12 equal slices and you eat 4 slices, what fraction of the cake did you eat?\n"
      ]
     }
    ],
    "source": [
     "# An example of a system message that primes the assistant to explain concepts in great depth\n",
-    "response = openai.ChatCompletion.create(\n",
+    "response = client.chat.completions.create(\n",
     "    model=MODEL,\n",
     "    messages=[\n",
     "        {\"role\": \"system\", \"content\": \"You are a friendly and helpful teaching assistant. You explain concepts in great depth using simple terms, and you give examples to help people learn. At the end of each explanation, you ask a question to check for understanding\"},\n",
@@ -287,25 +351,25 @@
     "    temperature=0,\n",
     ")\n",
     "\n",
-    "print(response[\"choices\"][0][\"message\"][\"content\"])\n"
+    "print(response.choices[0].message.content)\n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 13,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Fractions represent a part of a whole. They consist of a numerator (top number) and a denominator (bottom number) separated by a line. The numerator represents how many parts of the whole are being considered, while the denominator represents the total number of equal parts that make up the whole.\n"
+      "Fractions represent parts of a whole. They have a numerator (top number) and a denominator (bottom number).\n"
      ]
     }
    ],
    "source": [
     "# An example of a system message that primes the assistant to give brief, to-the-point answers\n",
-    "response = openai.ChatCompletion.create(\n",
+    "response = client.chat.completions.create(\n",
     "    model=MODEL,\n",
     "    messages=[\n",
     "        {\"role\": \"system\", \"content\": \"You are a laconic assistant. You reply with brief, to-the-point answers with no elaboration.\"},\n",
@@ -314,7 +378,7 @@
     "    temperature=0,\n",
     ")\n",
     "\n",
-    "print(response[\"choices\"][0][\"message\"][\"content\"])\n"
+    "print(response.choices[0].message.content)\n"
    ]
   },
   {
@@ -333,20 +397,20 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 14,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "We don't have enough time to complete the entire project perfectly.\n"
+      "This sudden change in direction means we don't have enough time to complete the entire project for the client.\n"
      ]
     }
    ],
    "source": [
     "# An example of a faked few-shot conversation to prime the model into translating business jargon to simpler speech\n",
-    "response = openai.ChatCompletion.create(\n",
+    "response = client.chat.completions.create(\n",
     "    model=MODEL,\n",
     "    messages=[\n",
     "        {\"role\": \"system\", \"content\": \"You are a helpful, pattern-following assistant.\"},\n",
@@ -361,7 +425,7 @@
     "    temperature=0,\n",
     ")\n",
     "\n",
-    "print(response[\"choices\"][0][\"message\"][\"content\"])\n"
+    "print(response.choices[0].message.content)\n"
    ]
   },
   {
@@ -376,20 +440,20 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 15,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "This sudden change in plans means we don't have enough time to do everything for the client's project.\n"
+      "This sudden change in direction means we don't have enough time to complete the entire project for the client.\n"
      ]
     }
    ],
    "source": [
     "# The business jargon translation example, but with example names for the example messages\n",
-    "response = openai.ChatCompletion.create(\n",
+    "response = client.chat.completions.create(\n",
     "    model=MODEL,\n",
     "    messages=[\n",
     "        {\"role\": \"system\", \"content\": \"You are a helpful, pattern-following assistant that translates corporate jargon into plain English.\"},\n",
@@ -402,7 +466,7 @@
     "    temperature=0,\n",
     ")\n",
     "\n",
-    "print(response[\"choices\"][0][\"message\"][\"content\"])\n"
+    "print(response.choices[0].message.content)\n"
    ]
   },
   {
@@ -416,7 +480,7 @@
     "\n",
     "As an example, one developer discovered an increase in accuracy when they inserted a user message that said \"Great job so far, these have been perfect\" to help condition the model into providing higher quality responses.\n",
     "\n",
-    "For more ideas on how to lift the reliability of the models, consider reading our guide on [techniques to increase reliability](../techniques_to_improve_reliability.md). It was written for non-chat models, but many of its principles still apply."
+    "For more ideas on how to lift the reliability of the models, consider reading our guide on [techniques to increase reliability](../techniques_to_improve_reliability). It was written for non-chat models, but many of its principles still apply."
    ]
   },
   {
@@ -444,7 +508,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 16,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -494,35 +558,29 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 23,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "gpt-3.5-turbo-0301\n",
-      "127 prompt tokens counted by num_tokens_from_messages().\n",
-      "127 prompt tokens counted by the OpenAI API.\n",
-      "\n",
-      "gpt-3.5-turbo-0613\n",
+      "gpt-3.5-turbo-1106\n",
+      "Warning: gpt-3.5-turbo may update over time. Returning num tokens assuming gpt-3.5-turbo-0613.\n",
       "129 prompt tokens counted by num_tokens_from_messages().\n",
       "129 prompt tokens counted by the OpenAI API.\n",
       "\n",
       "gpt-3.5-turbo\n",
       "Warning: gpt-3.5-turbo may update over time. Returning num tokens assuming gpt-3.5-turbo-0613.\n",
       "129 prompt tokens counted by num_tokens_from_messages().\n",
-      "127 prompt tokens counted by the OpenAI API.\n",
-      "\n",
-      "gpt-4-0314\n",
-      "129 prompt tokens counted by num_tokens_from_messages().\n",
       "129 prompt tokens counted by the OpenAI API.\n",
       "\n",
-      "gpt-4-0613\n",
+      "gpt-4\n",
+      "Warning: gpt-4 may update over time. Returning num tokens assuming gpt-4-0613.\n",
       "129 prompt tokens counted by num_tokens_from_messages().\n",
       "129 prompt tokens counted by the OpenAI API.\n",
       "\n",
-      "gpt-4\n",
+      "gpt-4-1106-preview\n",
       "Warning: gpt-4 may update over time. Returning num tokens assuming gpt-4-0613.\n",
       "129 prompt tokens counted by num_tokens_from_messages().\n",
       "129 prompt tokens counted by the OpenAI API.\n",
@@ -532,9 +590,6 @@
    ],
    "source": [
     "# let's verify the function above matches the OpenAI API response\n",
-    "\n",
-    "import openai\n",
-    "\n",
     "example_messages = [\n",
     "    {\n",
     "        \"role\": \"system\",\n",
@@ -567,26 +622,33 @@
     "]\n",
     "\n",
     "for model in [\n",
-    "    \"gpt-3.5-turbo-0301\",\n",
-    "    \"gpt-3.5-turbo-0613\",\n",
+    "    # \"gpt-3.5-turbo-0301\",\n",
+    "    # \"gpt-4-0314\",\n",
+    "    # \"gpt-4-0613\",\n",
+    "    \"gpt-3.5-turbo-1106\",\n",
     "    \"gpt-3.5-turbo\",\n",
-    "    \"gpt-4-0314\",\n",
-    "    \"gpt-4-0613\",\n",
     "    \"gpt-4\",\n",
+    "    \"gpt-4-1106-preview\",\n",
     "    ]:\n",
     "    print(model)\n",
     "    # example token count from the function defined above\n",
     "    print(f\"{num_tokens_from_messages(example_messages, model)} prompt tokens counted by num_tokens_from_messages().\")\n",
     "    # example token count from the OpenAI API\n",
-    "    response = openai.ChatCompletion.create(\n",
-    "        model=model,\n",
-    "        messages=example_messages,\n",
-    "        temperature=0,\n",
-    "        max_tokens=1,  # we're only counting input tokens here, so let's not waste tokens on the output\n",
-    "    )\n",
-    "    print(f'{response[\"usage\"][\"prompt_tokens\"]} prompt tokens counted by the OpenAI API.')\n",
+    "    response = client.chat.completions.create(model=model,\n",
+    "    messages=example_messages,\n",
+    "    temperature=0,\n",
+    "    max_tokens=1)\n",
+    "    token = response.usage.prompt_tokens\n",
+    "    print(f'{token} prompt tokens counted by the OpenAI API.')\n",
     "    print()\n"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -605,7 +667,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.9"
+   "version": "3.11.5"
   },
   "orig_nbformat": 4,
   "vscode": {

examples/How_to_handle_rate_limits.ipynb

@@ -11,7 +11,7 @@
     "\n",
     "This guide shares tips for avoiding and handling rate limit errors.\n",
     "\n",
-    "To see an example script for throttling parallel requests to avoid rate limit errors, see [api_request_parallel_processor.py](api_request_parallel_processor.py).\n",
+    "To see an example script for throttling parallel requests to avoid rate limit errors, see [api_request_parallel_processor.py](https://github.com/openai/openai-cookbook/blob/main/examples/api_request_parallel_processor.py).\n",
     "\n",
     "## Why rate limits exist\n",
     "\n",
@@ -43,9 +43,19 @@
     "\n",
     "### Requesting a rate limit increase\n",
     "\n",
-    "If you'd like your organization's rate limit increased, please fill out the following form:\n",
+    "If you'd like your organization's rate limit increased, please visit your [Limits settings page](https://platform.openai.com/account/limits) to see how you can increase your usage tier\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import openai\n",
+    "import os\n",
     "\n",
-    "- [OpenAI Rate Limit Increase Request form](https://forms.gle/56ZrwXXoxAN1yt6i9)\n"
+    "client = openai.OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {
@@ -69,9 +79,6 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from openai import OpenAI # for making OpenAI API requests\n",
-    "client = OpenAI()\n",
-    "\n",
     "# request a bunch of completions in a loop\n",
     "for _ in range(100):\n",
     "    client.chat.completions.create(\n",
@@ -132,8 +139,6 @@
     }
    ],
    "source": [
-    "from openai import OpenAI # for OpenAI API calls\n",
-    "client = OpenAI()\n",
     "from tenacity import (\n",
     "    retry,\n",
     "    stop_after_attempt,\n",
@@ -177,10 +182,6 @@
    ],
    "source": [
     "import backoff  # for exponential backoff\n",
-    "from openai import OpenAI # for OpenAI API calls\n",
-    "import openai\n",
-    "client = OpenAI()  \n",
-    "\n",
     "\n",
     "@backoff.on_exception(backoff.expo, openai.RateLimitError)\n",
     "def completions_with_backoff(**kwargs):\n",
@@ -220,10 +221,6 @@
     "import random\n",
     "import time\n",
     "\n",
-    "import openai\n",
-    "from openai import OpenAI\n",
-    "client = OpenAI()\n",
-    "\n",
     "# define a retry decorator\n",
     "def retry_with_exponential_backoff(\n",
     "    func,\n",
@@ -316,8 +313,6 @@
    "source": [
     "# imports\n",
     "import time\n",
-    "from openai import OpenAI\n",
-    "client = OpenAI()\n",
     "\n",
     "# Define a function that adds a delay to a Completion API call\n",
     "def delayed_completion(delay_in_seconds: float = 1, **kwargs):\n",
@@ -383,10 +378,6 @@
     }
    ],
    "source": [
-    "from openai import OpenAI # for making OpenAI API requests\n",
-    "client = OpenAI()  \n",
-    "\n",
-    "\n",
     "num_stories = 10\n",
     "content = \"Once upon a time,\"\n",
     "\n",
@@ -432,15 +423,11 @@
     }
    ],
    "source": [
-    "from openai import OpenAI # for making OpenAI API requests\n",
-    "client = OpenAI()  \n",
-    "\n",
-    "\n",
     "num_stories = 10\n",
     "prompts = [\"Once upon a time,\"] * num_stories\n",
     "\n",
     "# batched example, with 10 stories completions per request\n",
-    "response = client.completions.create(\n",
+    "response = client.chat.completions.create(\n",
     "    model=\"curie\",\n",
     "    prompt=prompts,\n",
     "    max_tokens=20,\n",

examples/How_to_stream_completions.ipynb

@@ -26,8 +26,7 @@
     "Below, this notebook shows:\n",
     "1. What a typical chat completion response looks like\n",
     "2. What a streaming chat completion response looks like\n",
-    "3. How much time is saved by streaming a chat completion\n",
-    "4. How to stream non-chat completions (used by older models like `text-davinci-003`)"
+    "3. How much time is saved by streaming a chat completion"
    ]
   },
   {
@@ -36,19 +35,20 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# !pip install openai==1.3.7"
+    "# !pip install openai"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [],
    "source": [
     "# imports\n",
     "import time  # for measuring time duration of API calls\n",
     "from openai import OpenAI\n",
-    "client = OpenAI()  # for OpenAI API calls"
+    "import os\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {
@@ -63,28 +63,28 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Full response received 7.90 seconds after request\n",
+      "Full response received 5.27 seconds after request\n",
       "Full response received:\n",
-      "ChatCompletion(id='chatcmpl-8SjX2G5nLRWrfnnPeFn9HAaJvSVZf', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content='1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100.', role='assistant', function_call=None, tool_calls=None))], created=1701858900, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=299, prompt_tokens=36, total_tokens=335))\n"
+      "ChatCompletion(id='chatcmpl-8ZB8ywkV5DuuJO7xktqUcNYfG8j6I', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100.', role='assistant', function_call=None, tool_calls=None))], created=1703395008, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=299, prompt_tokens=36, total_tokens=335))\n"
      ]
     }
    ],
    "source": [
     "# Example of an OpenAI ChatCompletion request\n",
-    "# https://platform.openai.com/docs/guides/chat\n",
+    "# https://platform.openai.com/docs/guides/text-generation/chat-completions-api\n",
     "\n",
     "# record the time before the request is sent\n",
     "start_time = time.time()\n",
     "\n",
     "# send a ChatCompletion request to count to 100\n",
-    "completion =  client.chat.completions.create(\n",
+    "response = client.chat.completions.create(\n",
     "    model='gpt-3.5-turbo',\n",
     "    messages=[\n",
     "        {'role': 'user', 'content': 'Count to 100, with a comma between each number and no newlines. E.g., 1, 2, 3, ...'}\n",
@@ -96,7 +96,7 @@
     "\n",
     "# print the time delay and text received\n",
     "print(f\"Full response received {response_time:.2f} seconds after request\")\n",
-    "print(f\"Full response received:\\n{completion}\")\n"
+    "print(f\"Full response received:\\n{response}\")\n"
    ]
   },
   {
@@ -104,14 +104,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The reply can be extracted with `response['choices'][0]['message']`.\n",
+    "The reply can be extracted with `response.choices[0].message`.\n",
     "\n",
-    "The content of the reply can be extracted with `response['choices'][0]['message']['content']`."
+    "The content of the reply can be extracted with `response.choices[0].message.content`."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [
     {
@@ -126,10 +126,10 @@
     }
    ],
    "source": [
-    "reply = completion.choices[0].message\n",
+    "reply = response.choices[0].message\n",
     "print(f\"Extracted reply: \\n{reply}\")\n",
     "\n",
-    "reply_content = completion.choices[0].message.content\n",
+    "reply_content = response.choices[0].message.content\n",
     "print(f\"Extracted content: \\n{reply_content}\")\n"
    ]
   },
@@ -147,20 +147,20 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "ChatCompletionChunk(id='chatcmpl-8SjXATR0aAejfQHMDHkNYYRTiUdca', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, role='assistant', tool_calls=None), finish_reason=None, index=0)], created=1701858908, model='gpt-3.5-turbo-0613', object='chat.completion.chunk', system_fingerprint=None)\n",
+      "ChatCompletionChunk(id='chatcmpl-8ZB9m2Ubv8FJs3CIb84WvYwqZCHST', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, role='assistant', tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1703395058, model='gpt-3.5-turbo-0613', object='chat.completion.chunk', system_fingerprint=None)\n",
       "\n",
       "****************\n",
-      "ChatCompletionChunk(id='chatcmpl-8SjXATR0aAejfQHMDHkNYYRTiUdca', choices=[Choice(delta=ChoiceDelta(content='2', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1701858908, model='gpt-3.5-turbo-0613', object='chat.completion.chunk', system_fingerprint=None)\n",
+      "ChatCompletionChunk(id='chatcmpl-8ZB9m2Ubv8FJs3CIb84WvYwqZCHST', choices=[Choice(delta=ChoiceDelta(content='2', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1703395058, model='gpt-3.5-turbo-0613', object='chat.completion.chunk', system_fingerprint=None)\n",
       "2\n",
       "****************\n",
-      "ChatCompletionChunk(id='chatcmpl-8SjXATR0aAejfQHMDHkNYYRTiUdca', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, role=None, tool_calls=None), finish_reason='stop', index=0)], created=1701858908, model='gpt-3.5-turbo-0613', object='chat.completion.chunk', system_fingerprint=None)\n",
+      "ChatCompletionChunk(id='chatcmpl-8ZB9m2Ubv8FJs3CIb84WvYwqZCHST', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, role=None, tool_calls=None), finish_reason='stop', index=0, logprobs=None)], created=1703395058, model='gpt-3.5-turbo-0613', object='chat.completion.chunk', system_fingerprint=None)\n",
       "None\n",
       "****************\n"
      ]
@@ -168,9 +168,10 @@
    ],
    "source": [
     "# Example of an OpenAI ChatCompletion request with stream=True\n",
-    "# https://platform.openai.com/docs/guides/chat\n",
+    "# https://platform.openai.com/docs/api-reference/streaming#chat/create-stream\n",
     "\n",
-    "completion = client.chat.completions.create(\n",
+    "# a ChatCompletion request\n",
+    "response = client.chat.completions.create(\n",
     "    model='gpt-3.5-turbo',\n",
     "    messages=[\n",
     "        {'role': 'user', 'content': \"What's 1+1? Answer in one word.\"}\n",
@@ -179,7 +180,7 @@
     "    stream=True  # this time, we set stream=True\n",
     ")\n",
     "\n",
-    "for chunk in completion:\n",
+    "for chunk in response:\n",
     "    print(chunk)\n",
     "    print(chunk.choices[0].delta.content)\n",
     "    print(\"****************\")"
@@ -208,328 +209,328 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 8,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Message received 0.99 seconds after request: \n",
-      "Message received 1.06 seconds after request: 1\n",
-      "Message received 1.06 seconds after request: ,\n",
-      "Message received 1.06 seconds after request:  \n",
-      "Message received 1.14 seconds after request: 2\n",
-      "Message received 1.14 seconds after request: ,\n",
-      "Message received 1.14 seconds after request:  \n",
-      "Message received 1.20 seconds after request: 3\n",
-      "Message received 1.20 seconds after request: ,\n",
-      "Message received 1.20 seconds after request:  \n",
-      "Message received 1.27 seconds after request: 4\n",
+      "Message received 0.31 seconds after request: \n",
+      "Message received 0.31 seconds after request: 1\n",
+      "Message received 0.34 seconds after request: ,\n",
+      "Message received 0.34 seconds after request:  \n",
+      "Message received 0.34 seconds after request: 2\n",
+      "Message received 0.39 seconds after request: ,\n",
+      "Message received 0.39 seconds after request:  \n",
+      "Message received 0.39 seconds after request: 3\n",
+      "Message received 0.42 seconds after request: ,\n",
+      "Message received 0.42 seconds after request:  \n",
+      "Message received 0.42 seconds after request: 4\n",
+      "Message received 0.47 seconds after request: ,\n",
+      "Message received 0.47 seconds after request:  \n",
+      "Message received 0.47 seconds after request: 5\n",
+      "Message received 0.51 seconds after request: ,\n",
+      "Message received 0.51 seconds after request:  \n",
+      "Message received 0.51 seconds after request: 6\n",
+      "Message received 0.55 seconds after request: ,\n",
+      "Message received 0.55 seconds after request:  \n",
+      "Message received 0.55 seconds after request: 7\n",
+      "Message received 0.59 seconds after request: ,\n",
+      "Message received 0.59 seconds after request:  \n",
+      "Message received 0.59 seconds after request: 8\n",
+      "Message received 0.63 seconds after request: ,\n",
+      "Message received 0.63 seconds after request:  \n",
+      "Message received 0.63 seconds after request: 9\n",
+      "Message received 0.67 seconds after request: ,\n",
+      "Message received 0.67 seconds after request:  \n",
+      "Message received 0.67 seconds after request: 10\n",
+      "Message received 0.71 seconds after request: ,\n",
+      "Message received 0.71 seconds after request:  \n",
+      "Message received 0.71 seconds after request: 11\n",
+      "Message received 0.75 seconds after request: ,\n",
+      "Message received 0.75 seconds after request:  \n",
+      "Message received 0.75 seconds after request: 12\n",
+      "Message received 0.98 seconds after request: ,\n",
+      "Message received 0.98 seconds after request:  \n",
+      "Message received 0.98 seconds after request: 13\n",
+      "Message received 1.02 seconds after request: ,\n",
+      "Message received 1.02 seconds after request:  \n",
+      "Message received 1.02 seconds after request: 14\n",
+      "Message received 1.04 seconds after request: ,\n",
+      "Message received 1.04 seconds after request:  \n",
+      "Message received 1.04 seconds after request: 15\n",
+      "Message received 1.08 seconds after request: ,\n",
+      "Message received 1.08 seconds after request:  \n",
+      "Message received 1.08 seconds after request: 16\n",
+      "Message received 1.12 seconds after request: ,\n",
+      "Message received 1.12 seconds after request:  \n",
+      "Message received 1.12 seconds after request: 17\n",
+      "Message received 1.16 seconds after request: ,\n",
+      "Message received 1.16 seconds after request:  \n",
+      "Message received 1.16 seconds after request: 18\n",
+      "Message received 1.19 seconds after request: ,\n",
+      "Message received 1.19 seconds after request:  \n",
+      "Message received 1.19 seconds after request: 19\n",
+      "Message received 1.23 seconds after request: ,\n",
+      "Message received 1.23 seconds after request:  \n",
+      "Message received 1.23 seconds after request: 20\n",
       "Message received 1.27 seconds after request: ,\n",
       "Message received 1.27 seconds after request:  \n",
-      "Message received 1.34 seconds after request: 5\n",
-      "Message received 1.34 seconds after request: ,\n",
-      "Message received 1.34 seconds after request:  \n",
-      "Message received 1.40 seconds after request: 6\n",
-      "Message received 1.40 seconds after request: ,\n",
-      "Message received 1.40 seconds after request:  \n",
-      "Message received 1.47 seconds after request: 7\n",
-      "Message received 1.48 seconds after request: ,\n",
-      "Message received 1.48 seconds after request:  \n",
-      "Message received 1.56 seconds after request: 8\n",
-      "Message received 1.56 seconds after request: ,\n",
-      "Message received 1.56 seconds after request:  \n",
-      "Message received 1.63 seconds after request: 9\n",
-      "Message received 1.63 seconds after request: ,\n",
-      "Message received 1.63 seconds after request:  \n",
-      "Message received 1.71 seconds after request: 10\n",
-      "Message received 1.71 seconds after request: ,\n",
-      "Message received 1.71 seconds after request:  \n",
-      "Message received 1.78 seconds after request: 11\n",
-      "Message received 1.78 seconds after request: ,\n",
-      "Message received 1.78 seconds after request:  \n",
-      "Message received 1.86 seconds after request: 12\n",
+      "Message received 1.27 seconds after request: 21\n",
+      "Message received 1.31 seconds after request: ,\n",
+      "Message received 1.31 seconds after request:  \n",
+      "Message received 1.31 seconds after request: 22\n",
+      "Message received 1.35 seconds after request: ,\n",
+      "Message received 1.35 seconds after request:  \n",
+      "Message received 1.35 seconds after request: 23\n",
+      "Message received 1.39 seconds after request: ,\n",
+      "Message received 1.39 seconds after request:  \n",
+      "Message received 1.39 seconds after request: 24\n",
+      "Message received 1.43 seconds after request: ,\n",
+      "Message received 1.43 seconds after request:  \n",
+      "Message received 1.43 seconds after request: 25\n",
+      "Message received 1.47 seconds after request: ,\n",
+      "Message received 1.47 seconds after request:  \n",
+      "Message received 1.47 seconds after request: 26\n",
+      "Message received 1.51 seconds after request: ,\n",
+      "Message received 1.51 seconds after request:  \n",
+      "Message received 1.51 seconds after request: 27\n",
+      "Message received 1.55 seconds after request: ,\n",
+      "Message received 1.55 seconds after request:  \n",
+      "Message received 1.55 seconds after request: 28\n",
+      "Message received 1.59 seconds after request: ,\n",
+      "Message received 1.59 seconds after request:  \n",
+      "Message received 1.59 seconds after request: 29\n",
+      "Message received 1.59 seconds after request: ,\n",
+      "Message received 1.59 seconds after request:  \n",
+      "Message received 1.59 seconds after request: 30\n",
+      "Message received 1.59 seconds after request: ,\n",
+      "Message received 1.59 seconds after request:  \n",
+      "Message received 1.59 seconds after request: 31\n",
+      "Message received 1.59 seconds after request: ,\n",
+      "Message received 1.59 seconds after request:  \n",
+      "Message received 1.60 seconds after request: 32\n",
+      "Message received 1.60 seconds after request: ,\n",
+      "Message received 1.60 seconds after request:  \n",
+      "Message received 1.60 seconds after request: 33\n",
+      "Message received 1.60 seconds after request: ,\n",
+      "Message received 1.60 seconds after request:  \n",
+      "Message received 1.67 seconds after request: 34\n",
+      "Message received 1.67 seconds after request: ,\n",
+      "Message received 1.67 seconds after request:  \n",
+      "Message received 1.68 seconds after request: 35\n",
+      "Message received 1.68 seconds after request: ,\n",
+      "Message received 1.68 seconds after request:  \n",
+      "Message received 1.86 seconds after request: 36\n",
       "Message received 1.86 seconds after request: ,\n",
       "Message received 1.86 seconds after request:  \n",
-      "Message received 2.20 seconds after request: 13\n",
-      "Message received 2.20 seconds after request: ,\n",
-      "Message received 2.20 seconds after request:  \n",
-      "Message received 2.31 seconds after request: 14\n",
+      "Message received 1.90 seconds after request: 37\n",
+      "Message received 1.90 seconds after request: ,\n",
+      "Message received 1.90 seconds after request:  \n",
+      "Message received 1.94 seconds after request: 38\n",
+      "Message received 1.94 seconds after request: ,\n",
+      "Message received 1.94 seconds after request:  \n",
+      "Message received 1.98 seconds after request: 39\n",
+      "Message received 1.98 seconds after request: ,\n",
+      "Message received 1.98 seconds after request:  \n",
+      "Message received 2.05 seconds after request: 40\n",
+      "Message received 2.05 seconds after request: ,\n",
+      "Message received 2.05 seconds after request:  \n",
+      "Message received 2.09 seconds after request: 41\n",
+      "Message received 2.09 seconds after request: ,\n",
+      "Message received 2.09 seconds after request:  \n",
+      "Message received 2.14 seconds after request: 42\n",
+      "Message received 2.14 seconds after request: ,\n",
+      "Message received 2.14 seconds after request:  \n",
+      "Message received 2.14 seconds after request: 43\n",
+      "Message received 2.14 seconds after request: ,\n",
+      "Message received 2.14 seconds after request:  \n",
+      "Message received 2.14 seconds after request: 44\n",
+      "Message received 2.14 seconds after request: ,\n",
+      "Message received 2.14 seconds after request:  \n",
+      "Message received 2.14 seconds after request: 45\n",
+      "Message received 2.14 seconds after request: ,\n",
+      "Message received 2.14 seconds after request:  \n",
+      "Message received 2.15 seconds after request: 46\n",
+      "Message received 2.15 seconds after request: ,\n",
+      "Message received 2.15 seconds after request:  \n",
+      "Message received 2.30 seconds after request: 47\n",
+      "Message received 2.30 seconds after request: ,\n",
+      "Message received 2.30 seconds after request:  \n",
+      "Message received 2.30 seconds after request: 48\n",
+      "Message received 2.30 seconds after request: ,\n",
+      "Message received 2.30 seconds after request:  \n",
+      "Message received 2.30 seconds after request: 49\n",
+      "Message received 2.30 seconds after request: ,\n",
+      "Message received 2.30 seconds after request:  \n",
+      "Message received 2.31 seconds after request: 50\n",
       "Message received 2.31 seconds after request: ,\n",
       "Message received 2.31 seconds after request:  \n",
-      "Message received 2.38 seconds after request: 15\n",
-      "Message received 2.38 seconds after request: ,\n",
+      "Message received 2.39 seconds after request: 51\n",
+      "Message received 2.39 seconds after request: ,\n",
       "Message received 2.39 seconds after request:  \n",
-      "Message received 2.46 seconds after request: 16\n",
-      "Message received 2.46 seconds after request: ,\n",
-      "Message received 2.46 seconds after request:  \n",
-      "Message received 2.53 seconds after request: 17\n",
-      "Message received 2.53 seconds after request: ,\n",
-      "Message received 2.53 seconds after request:  \n",
-      "Message received 2.61 seconds after request: 18\n",
-      "Message received 2.61 seconds after request: ,\n",
-      "Message received 2.61 seconds after request:  \n",
-      "Message received 2.67 seconds after request: 19\n",
-      "Message received 2.67 seconds after request: ,\n",
-      "Message received 2.67 seconds after request:  \n",
-      "Message received 2.74 seconds after request: 20\n",
-      "Message received 2.74 seconds after request: ,\n",
-      "Message received 2.74 seconds after request:  \n",
-      "Message received 2.83 seconds after request: 21\n",
-      "Message received 2.83 seconds after request: ,\n",
-      "Message received 2.83 seconds after request:  \n",
-      "Message received 2.89 seconds after request: 22\n",
+      "Message received 2.40 seconds after request: 52\n",
+      "Message received 2.40 seconds after request: ,\n",
+      "Message received 2.40 seconds after request:  \n",
+      "Message received 2.48 seconds after request: 53\n",
+      "Message received 2.48 seconds after request: ,\n",
+      "Message received 2.48 seconds after request:  \n",
+      "Message received 2.49 seconds after request: 54\n",
+      "Message received 2.49 seconds after request: ,\n",
+      "Message received 2.49 seconds after request:  \n",
+      "Message received 2.68 seconds after request: 55\n",
+      "Message received 2.68 seconds after request: ,\n",
+      "Message received 2.68 seconds after request:  \n",
+      "Message received 2.72 seconds after request: 56\n",
+      "Message received 2.72 seconds after request: ,\n",
+      "Message received 2.72 seconds after request:  \n",
+      "Message received 2.77 seconds after request: 57\n",
+      "Message received 2.77 seconds after request: ,\n",
+      "Message received 2.77 seconds after request:  \n",
+      "Message received 2.80 seconds after request: 58\n",
+      "Message received 2.80 seconds after request: ,\n",
+      "Message received 2.80 seconds after request:  \n",
+      "Message received 2.85 seconds after request: 59\n",
+      "Message received 2.85 seconds after request: ,\n",
+      "Message received 2.85 seconds after request:  \n",
+      "Message received 2.88 seconds after request: 60\n",
+      "Message received 2.88 seconds after request: ,\n",
+      "Message received 2.88 seconds after request:  \n",
+      "Message received 2.88 seconds after request: 61\n",
+      "Message received 2.88 seconds after request: ,\n",
+      "Message received 2.88 seconds after request:  \n",
+      "Message received 2.89 seconds after request: 62\n",
+      "Message received 2.89 seconds after request: ,\n",
+      "Message received 2.89 seconds after request:  \n",
+      "Message received 2.89 seconds after request: 63\n",
       "Message received 2.89 seconds after request: ,\n",
       "Message received 2.89 seconds after request:  \n",
-      "Message received 3.02 seconds after request: 23\n",
-      "Message received 3.02 seconds after request: ,\n",
-      "Message received 3.03 seconds after request:  \n",
-      "Message received 3.09 seconds after request: 24\n",
-      "Message received 3.09 seconds after request: ,\n",
-      "Message received 3.09 seconds after request:  \n",
-      "Message received 3.15 seconds after request: 25\n",
-      "Message received 3.15 seconds after request: ,\n",
-      "Message received 3.16 seconds after request:  \n",
-      "Message received 3.22 seconds after request: 26\n",
-      "Message received 3.22 seconds after request: ,\n",
-      "Message received 3.22 seconds after request:  \n",
-      "Message received 3.30 seconds after request: 27\n",
-      "Message received 3.30 seconds after request: ,\n",
-      "Message received 3.30 seconds after request:  \n",
-      "Message received 3.37 seconds after request: 28\n",
+      "Message received 2.92 seconds after request: 64\n",
+      "Message received 2.92 seconds after request: ,\n",
+      "Message received 2.92 seconds after request:  \n",
+      "Message received 3.37 seconds after request: 65\n",
       "Message received 3.37 seconds after request: ,\n",
       "Message received 3.37 seconds after request:  \n",
-      "Message received 3.55 seconds after request: 29\n",
-      "Message received 3.55 seconds after request: ,\n",
-      "Message received 3.55 seconds after request:  \n",
-      "Message received 3.55 seconds after request: 30\n",
-      "Message received 3.55 seconds after request: ,\n",
-      "Message received 3.55 seconds after request:  \n",
-      "Message received 3.55 seconds after request: 31\n",
-      "Message received 3.55 seconds after request: ,\n",
-      "Message received 3.56 seconds after request:  \n",
-      "Message received 3.56 seconds after request: 32\n",
-      "Message received 3.56 seconds after request: ,\n",
-      "Message received 3.56 seconds after request:  \n",
-      "Message received 3.56 seconds after request: 33\n",
+      "Message received 3.38 seconds after request: 66\n",
+      "Message received 3.38 seconds after request: ,\n",
+      "Message received 3.38 seconds after request:  \n",
+      "Message received 3.38 seconds after request: 67\n",
+      "Message received 3.38 seconds after request: ,\n",
+      "Message received 3.38 seconds after request:  \n",
+      "Message received 3.38 seconds after request: 68\n",
+      "Message received 3.38 seconds after request: ,\n",
+      "Message received 3.38 seconds after request:  \n",
+      "Message received 3.42 seconds after request: 69\n",
+      "Message received 3.42 seconds after request: ,\n",
+      "Message received 3.42 seconds after request:  \n",
+      "Message received 3.43 seconds after request: 70\n",
+      "Message received 3.43 seconds after request: ,\n",
+      "Message received 3.43 seconds after request:  \n",
+      "Message received 3.46 seconds after request: 71\n",
+      "Message received 3.46 seconds after request: ,\n",
+      "Message received 3.46 seconds after request:  \n",
+      "Message received 3.47 seconds after request: 72\n",
+      "Message received 3.47 seconds after request: ,\n",
+      "Message received 3.47 seconds after request:  \n",
+      "Message received 3.50 seconds after request: 73\n",
+      "Message received 3.50 seconds after request: ,\n",
+      "Message received 3.50 seconds after request:  \n",
+      "Message received 3.51 seconds after request: 74\n",
+      "Message received 3.51 seconds after request: ,\n",
+      "Message received 3.51 seconds after request:  \n",
+      "Message received 3.52 seconds after request: 75\n",
+      "Message received 3.52 seconds after request: ,\n",
+      "Message received 3.52 seconds after request:  \n",
+      "Message received 3.54 seconds after request: 76\n",
+      "Message received 3.54 seconds after request: ,\n",
+      "Message received 3.54 seconds after request:  \n",
+      "Message received 3.56 seconds after request: 77\n",
       "Message received 3.56 seconds after request: ,\n",
       "Message received 3.56 seconds after request:  \n",
-      "Message received 3.60 seconds after request: 34\n",
-      "Message received 3.60 seconds after request: ,\n",
-      "Message received 3.60 seconds after request:  \n",
-      "Message received 3.63 seconds after request: 35\n",
-      "Message received 3.63 seconds after request: ,\n",
-      "Message received 3.63 seconds after request:  \n",
-      "Message received 3.95 seconds after request: 36\n",
+      "Message received 3.59 seconds after request: 78\n",
+      "Message received 3.59 seconds after request: ,\n",
+      "Message received 3.59 seconds after request:  \n",
+      "Message received 3.59 seconds after request: 79\n",
+      "Message received 3.59 seconds after request: ,\n",
+      "Message received 3.59 seconds after request:  \n",
+      "Message received 3.59 seconds after request: 80\n",
+      "Message received 3.59 seconds after request: ,\n",
+      "Message received 3.59 seconds after request:  \n",
+      "Message received 3.61 seconds after request: 81\n",
+      "Message received 3.61 seconds after request: ,\n",
+      "Message received 3.61 seconds after request:  \n",
+      "Message received 3.65 seconds after request: 82\n",
+      "Message received 3.65 seconds after request: ,\n",
+      "Message received 3.65 seconds after request:  \n",
+      "Message received 3.85 seconds after request: 83\n",
+      "Message received 3.85 seconds after request: ,\n",
+      "Message received 3.85 seconds after request:  \n",
+      "Message received 3.90 seconds after request: 84\n",
+      "Message received 3.90 seconds after request: ,\n",
+      "Message received 3.90 seconds after request:  \n",
+      "Message received 3.95 seconds after request: 85\n",
       "Message received 3.95 seconds after request: ,\n",
       "Message received 3.95 seconds after request:  \n",
-      "Message received 4.02 seconds after request: 37\n",
-      "Message received 4.02 seconds after request: ,\n",
-      "Message received 4.02 seconds after request:  \n",
-      "Message received 4.08 seconds after request: 38\n",
+      "Message received 4.00 seconds after request: 86\n",
+      "Message received 4.00 seconds after request: ,\n",
+      "Message received 4.00 seconds after request:  \n",
+      "Message received 4.04 seconds after request: 87\n",
+      "Message received 4.04 seconds after request: ,\n",
+      "Message received 4.04 seconds after request:  \n",
+      "Message received 4.08 seconds after request: 88\n",
       "Message received 4.08 seconds after request: ,\n",
       "Message received 4.08 seconds after request:  \n",
-      "Message received 4.17 seconds after request: 39\n",
-      "Message received 4.17 seconds after request: ,\n",
-      "Message received 4.17 seconds after request:  \n",
-      "Message received 4.22 seconds after request: 40\n",
-      "Message received 4.22 seconds after request: ,\n",
-      "Message received 4.22 seconds after request:  \n",
-      "Message received 4.29 seconds after request: 41\n",
-      "Message received 4.29 seconds after request: ,\n",
-      "Message received 4.29 seconds after request:  \n",
-      "Message received 4.37 seconds after request: 42\n",
-      "Message received 4.37 seconds after request: ,\n",
-      "Message received 4.37 seconds after request:  \n",
-      "Message received 4.37 seconds after request: 43\n",
-      "Message received 4.37 seconds after request: ,\n",
-      "Message received 4.37 seconds after request:  \n",
-      "Message received 4.37 seconds after request: 44\n",
-      "Message received 4.37 seconds after request: ,\n",
-      "Message received 4.37 seconds after request:  \n",
-      "Message received 4.38 seconds after request: 45\n",
-      "Message received 4.38 seconds after request: ,\n",
-      "Message received 4.38 seconds after request:  \n",
-      "Message received 4.39 seconds after request: 46\n",
+      "Message received 4.12 seconds after request: 89\n",
+      "Message received 4.12 seconds after request: ,\n",
+      "Message received 4.12 seconds after request:  \n",
+      "Message received 4.18 seconds after request: 90\n",
+      "Message received 4.18 seconds after request: ,\n",
+      "Message received 4.18 seconds after request:  \n",
+      "Message received 4.18 seconds after request: 91\n",
+      "Message received 4.18 seconds after request: ,\n",
+      "Message received 4.18 seconds after request:  \n",
+      "Message received 4.18 seconds after request: 92\n",
+      "Message received 4.18 seconds after request: ,\n",
+      "Message received 4.18 seconds after request:  \n",
+      "Message received 4.19 seconds after request: 93\n",
+      "Message received 4.19 seconds after request: ,\n",
+      "Message received 4.19 seconds after request:  \n",
+      "Message received 4.20 seconds after request: 94\n",
+      "Message received 4.20 seconds after request: ,\n",
+      "Message received 4.20 seconds after request:  \n",
+      "Message received 4.23 seconds after request: 95\n",
+      "Message received 4.23 seconds after request: ,\n",
+      "Message received 4.23 seconds after request:  \n",
+      "Message received 4.27 seconds after request: 96\n",
+      "Message received 4.27 seconds after request: ,\n",
+      "Message received 4.27 seconds after request:  \n",
+      "Message received 4.39 seconds after request: 97\n",
+      "Message received 4.39 seconds after request: ,\n",
+      "Message received 4.39 seconds after request:  \n",
+      "Message received 4.39 seconds after request: 98\n",
       "Message received 4.39 seconds after request: ,\n",
       "Message received 4.39 seconds after request:  \n",
-      "Message received 4.66 seconds after request: 47\n",
-      "Message received 4.66 seconds after request: ,\n",
-      "Message received 4.66 seconds after request:  \n",
-      "Message received 4.66 seconds after request: 48\n",
-      "Message received 4.66 seconds after request: ,\n",
-      "Message received 4.66 seconds after request:  \n",
-      "Message received 4.66 seconds after request: 49\n",
-      "Message received 4.66 seconds after request: ,\n",
-      "Message received 4.66 seconds after request:  \n",
-      "Message received 4.69 seconds after request: 50\n",
-      "Message received 4.69 seconds after request: ,\n",
-      "Message received 4.69 seconds after request:  \n",
-      "Message received 4.80 seconds after request: 51\n",
-      "Message received 4.80 seconds after request: ,\n",
-      "Message received 4.80 seconds after request:  \n",
-      "Message received 4.83 seconds after request: 52\n",
-      "Message received 4.83 seconds after request: ,\n",
-      "Message received 4.83 seconds after request:  \n",
-      "Message received 4.95 seconds after request: 53\n",
-      "Message received 4.95 seconds after request: ,\n",
-      "Message received 4.95 seconds after request:  \n",
-      "Message received 4.97 seconds after request: 54\n",
-      "Message received 4.97 seconds after request: ,\n",
-      "Message received 4.97 seconds after request:  \n",
-      "Message received 5.31 seconds after request: 55\n",
-      "Message received 5.31 seconds after request: ,\n",
-      "Message received 5.31 seconds after request:  \n",
-      "Message received 5.37 seconds after request: 56\n",
-      "Message received 5.37 seconds after request: ,\n",
-      "Message received 5.37 seconds after request:  \n",
-      "Message received 5.47 seconds after request: 57\n",
-      "Message received 5.47 seconds after request: ,\n",
-      "Message received 5.47 seconds after request:  \n",
-      "Message received 5.54 seconds after request: 58\n",
-      "Message received 5.54 seconds after request: ,\n",
-      "Message received 5.54 seconds after request:  \n",
-      "Message received 5.60 seconds after request: 59\n",
-      "Message received 5.60 seconds after request: ,\n",
-      "Message received 5.60 seconds after request:  \n",
-      "Message received 5.66 seconds after request: 60\n",
-      "Message received 5.66 seconds after request: ,\n",
-      "Message received 5.66 seconds after request:  \n",
-      "Message received 5.67 seconds after request: 61\n",
-      "Message received 5.67 seconds after request: ,\n",
-      "Message received 5.67 seconds after request:  \n",
-      "Message received 5.67 seconds after request: 62\n",
-      "Message received 5.67 seconds after request: ,\n",
-      "Message received 5.67 seconds after request:  \n",
-      "Message received 5.67 seconds after request: 63\n",
-      "Message received 5.67 seconds after request: ,\n",
-      "Message received 5.67 seconds after request:  \n",
-      "Message received 5.69 seconds after request: 64\n",
-      "Message received 5.69 seconds after request: ,\n",
-      "Message received 5.69 seconds after request:  \n",
-      "Message received 5.95 seconds after request: 65\n",
-      "Message received 5.95 seconds after request: ,\n",
-      "Message received 5.95 seconds after request:  \n",
-      "Message received 5.95 seconds after request: 66\n",
-      "Message received 5.95 seconds after request: ,\n",
-      "Message received 5.95 seconds after request:  \n",
-      "Message received 5.95 seconds after request: 67\n",
-      "Message received 5.95 seconds after request: ,\n",
-      "Message received 5.95 seconds after request:  \n",
-      "Message received 5.97 seconds after request: 68\n",
-      "Message received 5.97 seconds after request: ,\n",
-      "Message received 5.97 seconds after request:  \n",
-      "Message received 6.08 seconds after request: 69\n",
-      "Message received 6.08 seconds after request: ,\n",
-      "Message received 6.08 seconds after request:  \n",
-      "Message received 6.10 seconds after request: 70\n",
-      "Message received 6.10 seconds after request: ,\n",
-      "Message received 6.10 seconds after request:  \n",
-      "Message received 6.17 seconds after request: 71\n",
-      "Message received 6.17 seconds after request: ,\n",
-      "Message received 6.17 seconds after request:  \n",
-      "Message received 6.24 seconds after request: 72\n",
-      "Message received 6.24 seconds after request: ,\n",
-      "Message received 6.24 seconds after request:  \n",
-      "Message received 6.35 seconds after request: 73\n",
-      "Message received 6.35 seconds after request: ,\n",
-      "Message received 6.35 seconds after request:  \n",
-      "Message received 6.37 seconds after request: 74\n",
-      "Message received 6.37 seconds after request: ,\n",
-      "Message received 6.37 seconds after request:  \n",
-      "Message received 6.45 seconds after request: 75\n",
-      "Message received 6.45 seconds after request: ,\n",
-      "Message received 6.45 seconds after request:  \n",
-      "Message received 6.52 seconds after request: 76\n",
-      "Message received 6.52 seconds after request: ,\n",
-      "Message received 6.52 seconds after request:  \n",
-      "Message received 6.59 seconds after request: 77\n",
-      "Message received 6.59 seconds after request: ,\n",
-      "Message received 6.59 seconds after request:  \n",
-      "Message received 6.83 seconds after request: 78\n",
-      "Message received 6.83 seconds after request: ,\n",
-      "Message received 6.83 seconds after request:  \n",
-      "Message received 6.83 seconds after request: 79\n",
-      "Message received 6.84 seconds after request: ,\n",
-      "Message received 6.84 seconds after request:  \n",
-      "Message received 6.84 seconds after request: 80\n",
-      "Message received 6.84 seconds after request: ,\n",
-      "Message received 6.84 seconds after request:  \n",
-      "Message received 6.85 seconds after request: 81\n",
-      "Message received 6.85 seconds after request: ,\n",
-      "Message received 6.85 seconds after request:  \n",
-      "Message received 6.92 seconds after request: 82\n",
-      "Message received 6.92 seconds after request: ,\n",
-      "Message received 6.92 seconds after request:  \n",
-      "Message received 7.23 seconds after request: 83\n",
-      "Message received 7.23 seconds after request: ,\n",
-      "Message received 7.23 seconds after request:  \n",
-      "Message received 7.30 seconds after request: 84\n",
-      "Message received 7.30 seconds after request: ,\n",
-      "Message received 7.30 seconds after request:  \n",
-      "Message received 7.37 seconds after request: 85\n",
-      "Message received 7.37 seconds after request: ,\n",
-      "Message received 7.37 seconds after request:  \n",
-      "Message received 7.43 seconds after request: 86\n",
-      "Message received 7.43 seconds after request: ,\n",
-      "Message received 7.43 seconds after request:  \n",
-      "Message received 7.70 seconds after request: 87\n",
-      "Message received 7.70 seconds after request: ,\n",
-      "Message received 7.70 seconds after request:  \n",
-      "Message received 7.71 seconds after request: 88\n",
-      "Message received 7.71 seconds after request: ,\n",
-      "Message received 7.71 seconds after request:  \n",
-      "Message received 7.73 seconds after request: 89\n",
-      "Message received 7.73 seconds after request: ,\n",
-      "Message received 7.73 seconds after request:  \n",
-      "Message received 7.80 seconds after request: 90\n",
-      "Message received 7.80 seconds after request: ,\n",
-      "Message received 7.80 seconds after request:  \n",
-      "Message received 7.80 seconds after request: 91\n",
-      "Message received 7.80 seconds after request: ,\n",
-      "Message received 7.80 seconds after request:  \n",
-      "Message received 7.80 seconds after request: 92\n",
-      "Message received 7.80 seconds after request: ,\n",
-      "Message received 7.80 seconds after request:  \n",
-      "Message received 7.80 seconds after request: 93\n",
-      "Message received 7.80 seconds after request: ,\n",
-      "Message received 7.80 seconds after request:  \n",
-      "Message received 7.82 seconds after request: 94\n",
-      "Message received 7.82 seconds after request: ,\n",
-      "Message received 7.82 seconds after request:  \n",
-      "Message received 7.90 seconds after request: 95\n",
-      "Message received 7.90 seconds after request: ,\n",
-      "Message received 7.90 seconds after request:  \n",
-      "Message received 7.96 seconds after request: 96\n",
-      "Message received 7.96 seconds after request: ,\n",
-      "Message received 7.97 seconds after request:  \n",
-      "Message received 8.13 seconds after request: 97\n",
-      "Message received 8.14 seconds after request: ,\n",
-      "Message received 8.14 seconds after request:  \n",
-      "Message received 8.14 seconds after request: 98\n",
-      "Message received 8.14 seconds after request: ,\n",
-      "Message received 8.14 seconds after request:  \n",
-      "Message received 8.16 seconds after request: 99\n",
-      "Message received 8.16 seconds after request: ,\n",
-      "Message received 8.16 seconds after request:  \n",
-      "Message received 8.16 seconds after request: 100\n",
-      "Message received 8.16 seconds after request: .\n",
-      "Message received 8.16 seconds after request: None\n",
-      "Full response received 8.16 seconds after request\n",
+      "Message received 4.41 seconds after request: 99\n",
+      "Message received 4.41 seconds after request: ,\n",
+      "Message received 4.41 seconds after request:  \n",
+      "Message received 4.41 seconds after request: 100\n",
+      "Message received 4.41 seconds after request: .\n",
+      "Message received 4.41 seconds after request: None\n",
+      "Full response received 4.41 seconds after request\n",
       "Full conversation received: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100.\n"
      ]
     }
    ],
    "source": [
     "# Example of an OpenAI ChatCompletion request with stream=True\n",
-    "# https://platform.openai.com/docs/guides/chat\n",
+    "# https://platform.openai.com/docs/api-reference/streaming#chat/create-stream\n",
     "\n",
     "# record the time before the request is sent\n",
     "start_time = time.time()\n",
     "\n",
     "# send a ChatCompletion request to count to 100\n",
-    "completion = client.chat.completions.create(\n",
+    "response = client.chat.completions.create(\n",
     "    model='gpt-3.5-turbo',\n",
     "    messages=[\n",
     "        {'role': 'user', 'content': 'Count to 100, with a comma between each number and no newlines. E.g., 1, 2, 3, ...'}\n",
@@ -541,7 +542,7 @@
     "collected_chunks = []\n",
     "collected_messages = []\n",
     "# iterate through the stream of events\n",
-    "for chunk in completion:\n",
+    "for chunk in response:\n",
     "    chunk_time = time.time() - start_time  # calculate the time delay of the chunk\n",
     "    collected_chunks.append(chunk)  # save the event response\n",
     "    chunk_message = chunk.choices[0].delta.content  # extract the message\n",
@@ -563,10 +564,15 @@
    "source": [
     "#### Time comparison\n",
     "\n",
-    "In the example above, both requests took about 3 seconds to fully complete. Request times will vary depending on load and other stochastic factors.\n",
+    "In the example above, both requests took about 4 to 5 seconds to fully complete. Request times will vary depending on load and other stochastic factors.\n",
     "\n",
     "However, with the streaming request, we received the first token after 0.1 seconds, and subsequent tokens every ~0.01-0.02 seconds."
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": []
   }
  ],
  "metadata": {
@@ -585,7 +591,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.12"
+   "version": "3.11.3"
   },
   "orig_nbformat": 4,
   "vscode": {

examples/How_to_use_moderation.ipynb

@@ -0,0 +1,498 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# How to use the moderation API\n",
+    "\n",
+    "**Note:** This guide is designed to complement our Guardrails Cookbook by providing a more focused look at moderation techniques. While there is some overlap in content and structure, this cookbook delves deeper into the nuances of tailoring moderation criteria to specific needs, offering a more granular level of control. If you're interested in a broader overview of content safety measures, including guardrails and moderation, we recommend starting with the [Guardrails Cookbook](https://cookbook.openai.com/examples/how_to_use_guardrails). Together, these resources offer a comprehensive understanding of how to effectively manage and moderate content within your applications.\n",
+    "\n",
+    "Moderation, much like guardrails in the physical world, serves as a preventative measure to ensure that your application remains within the bounds of acceptable and safe content. Moderation techniques are incredibly versatile and can be applied to a wide array of scenarios where LLMs might encounter issues. This notebook is designed to offer straightforward examples that can be adapted to suit your specific needs, while also discussing the considerations and trade-offs involved in deciding whether to implement moderation and how to go about it. This notebook will use our [Moderation API](https://platform.openai.com/docs/guides/moderation/overview), a tool you can use to check whether text is potentially harmful.\n",
+    "\n",
+    "This notebook will concentrate on:\n",
+    "\n",
+    "- **Input Moderation:** Identifying and flagging inappropriate or harmful content before it is processed by your LLM.\n",
+    "- **Output Moderation:** Reviewing and validating the content generated by your LLM before it reaches the end user.\n",
+    "- **Custom Moderation:** Tailoring moderation criteria and rules to suit the specific needs and context of your application, ensuring a personalized and effective content control mechanism."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from openai import OpenAI\n",
+    "client = OpenAI()\n",
+    "\n",
+    "GPT_MODEL = 'gpt-3.5-turbo'"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 1. Input moderation\n",
+    "Input Moderation focuses on preventing harmful or inappropriate content from reaching the LLM, with common applications including:\n",
+    "- **Content Filtering:** Prevent the spread of harmful content such as hate speech, harassment, explicit material, and misinformation on social media, forums, and content creation platforms.\n",
+    "- **Community Standards Enforcement:** Ensure that user interactions, such as comments, forum posts, and chat messages, adhere to the community guidelines and standards of online platforms, including educational environments, gaming communities, or dating apps.\n",
+    "- **Spam and Fraud Prevention:** Filter out spam, fraudulent content, and misleading information in online forums, comment sections, e-commerce platforms, and customer reviews.\n",
+    "\n",
+    "These measures act as preventive controls, operating before or alongside the LLM to alter your application's behavior if specific criteria are met."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Embrace async\n",
+    "A common design to minimize latency is to send your moderations asynchronously along with your main LLM call. If your moderation gets triggered you send back a placeholder response, otherwise send back the LLM response. This pattern can also be found in our [Guardrails Cookbook](https://cookbook.openai.com/examples/how_to_use_guardrails). It's important to note that while the async mode is effective in minimizing latency, it can also lead to unnecessary costs. Specifically, you could avoid completion costs if the content is flagged before processing. Therefore, it's crucial to balance the benefits of reduced latency with the potential for increased expenses when using async mode.\n",
+    "\n",
+    "We'll use this approach, creating an execute_chat_with_moderation function that will run our LLM's get_chat_response and the check_expression moderation function in parallel, and return the LLM response only if the moderation returns False (not triggered).\n",
+    "\n",
+    "#### Workflow:\n",
+    "We will create a workflow that incorporates the Moderation API to check user input for any potentially unsafe content before it is sent to the Language Model (LLM). This ensures that only appropriate content is processed further in the application.\n",
+    "\n",
+    "1. Receive an input from the user\n",
+    "2. Use the Moderation API to analyze the input for any problematic content. \n",
+    "3. **Conditional Processing:** \n",
+    "- If the input is flagged by the moderation check, handle it accordingly (e.g., reject the input, ask the user to rephrase, etc.).\n",
+    "- If the input is not flagged, pass it to the LLM for further processing.\n",
+    "\n",
+    "We will demonstrate this workflow with two example prompts."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "system_prompt = \"You are a helpful assistant.\"\n",
+    "\n",
+    "bad_request = \"I want to hurt them. How can i do this?\"\n",
+    "good_request = \"I would kill for a cup of coffe. Where can I get one nearby?\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import asyncio\n",
+    "\n",
+    "async def check_moderation_flag(expression):\n",
+    "    moderation_response = client.moderations.create(input=expression)\n",
+    "    flagged = moderation_response.results[0].flagged\n",
+    "    return flagged\n",
+    "    \n",
+    "async def get_chat_response(user_request):\n",
+    "    print(\"Getting LLM response\")\n",
+    "    messages = [\n",
+    "        {\"role\": \"system\", \"content\": system_prompt},\n",
+    "        {\"role\": \"user\", \"content\": user_request},\n",
+    "    ]\n",
+    "    response = client.chat.completions.create(\n",
+    "        model=GPT_MODEL, messages=messages, temperature=0.5\n",
+    "    )\n",
+    "    print(\"Got LLM response\")\n",
+    "    return response.choices[0].message.content\n",
+    "\n",
+    "\n",
+    "async def execute_chat_with_input_moderation(user_request):\n",
+    "    # Create tasks for moderation and chat response\n",
+    "    moderation_task = asyncio.create_task(check_moderation_flag(user_request))\n",
+    "    chat_task = asyncio.create_task(get_chat_response(user_request))\n",
+    "\n",
+    "    while True:\n",
+    "        # Wait for either the moderation task or chat task to complete\n",
+    "        done, _ = await asyncio.wait(\n",
+    "            [moderation_task, chat_task], return_when=asyncio.FIRST_COMPLETED\n",
+    "        )\n",
+    "\n",
+    "        # If moderation task is not completed, wait and continue to the next iteration\n",
+    "        if moderation_task not in done:\n",
+    "            await asyncio.sleep(0.1)\n",
+    "            continue\n",
+    "\n",
+    "        # If moderation is triggered, cancel the chat task and return a message\n",
+    "        if moderation_task.result() == True:\n",
+    "            chat_task.cancel()\n",
+    "            print(\"Moderation triggered\")\n",
+    "            return \"We're sorry, but your input has been flagged as inappropriate. Please rephrase your input and try again.\"\n",
+    "\n",
+    "        # If chat task is completed, return the chat response\n",
+    "        if chat_task in done:\n",
+    "            return chat_task.result()\n",
+    "\n",
+    "        # If neither task is completed, sleep for a bit before checking again\n",
+    "        await asyncio.sleep(0.1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Getting LLM response\n",
+      "Got LLM response\n",
+      "I can help you with that! To find a nearby coffee shop, you can use a mapping app on your phone or search online for coffee shops in your current location. Alternatively, you can ask locals or check for any cafes or coffee shops in the vicinity. Enjoy your coffee!\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Call the main function with the good request - this should go through\n",
+    "good_response = await execute_chat_with_input_moderation(good_request)\n",
+    "print(good_response)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Getting LLM response\n",
+      "Got LLM response\n",
+      "Moderation triggered\n",
+      "We're sorry, but your input has been flagged as inappropriate. Please rephrase your input and try again.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Call the main function with the bad request - this should get blocked\n",
+    "bad_response = await execute_chat_with_input_moderation(bad_request)\n",
+    "print(bad_response)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Looks like our moderation worked - the first question was allowed through, but the second was blocked for inapropriate content. Now we'll extend this concept to moderate the response we get from the LLM as well."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 2. Output moderation\n",
+    "\n",
+    "Output moderation is crucial for controlling the content generated by the Language Model (LLM). While LLMs should not output illegal or harmful content, it can be helpful to put additional guardrails in place to further ensure that the content remains within acceptable and safe boundaries, enhancing the overall security and reliability of the application. Common types of output moderation include:\n",
+    "\n",
+    "- **Content Quality Assurance:** Ensure that generated content, such as articles, product descriptions, and educational materials, is accurate, informative, and free from inappropriate information.\n",
+    "- **Community Standards Compliance:** Maintain a respectful and safe environment in online forums, discussion boards, and gaming communities by filtering out hate speech, harassment, and other harmful content.\n",
+    "- **User Experience Enhancement:** Improve the user experience in chatbots and automated services by providing responses that are polite, relevant, and free from any unsuitable language or content.\n",
+    "\n",
+    "In all these scenarios, output moderation plays a crucial role in maintaining the quality and integrity of the content generated by language models, ensuring that it meets the standards and expectations of the platform and its users."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Setting moderation thresholds\n",
+    "OpenAI has selected thresholds for moderation categories that balance precision and recall for our use cases, but your use case or tolerance for moderation may be different. Setting this threshold is a common area for optimization - we recommend building an evaluation set and grading the results using a confusion matrix to set the right tolerance for your moderation. The trade-off here is generally:\n",
+    "\n",
+    "- More false positives leads to a fractured user experience, where customers get annoyed and the assistant seems less helpful.\n",
+    "- More false negatives can cause lasting harm to your business, as people get the assistant to answer inappropriate questions, or provide inappropriate responses.\n",
+    "\n",
+    "For example, on a platform dedicated to creative writing, the moderation threshold for certain sensitive topics might be set higher to allow for greater creative freedom while still providing a safety net to catch content that is clearly beyond the bounds of acceptable expression. The trade-off is that some content that might be considered inappropriate in other contexts is allowed, but this is deemed acceptable given the platform's purpose and audience expectations.\n",
+    "\n",
+    "#### Workflow:\n",
+    "We will create a workflow that incorporates the Moderation API to check the LLM response for any potentially unsafe content before it is sent to the Language Model (LLM). This ensures that only appropriate content is displayed to the user.\n",
+    "\n",
+    "1. Receive an input from the user\n",
+    "2. Send prompt to LLM and generate a response\n",
+    "3. Use the Moderation API to analyze the LLM's response for any problematic content. \n",
+    "3. **Conditional Processing:** \n",
+    "- If the response is flagged by the moderation check, handle it accordingly (e.g., reject the response, show a placeholder message, etc.).\n",
+    "- If the response is not flagged, display it to the user.\n",
+    "\n",
+    "We will demonstrate this workflow with the previous two example prompts."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "async def execute_all_moderations(user_request):\n",
+    "    # Create tasks for moderation and chat response\n",
+    "    input_moderation_task = asyncio.create_task(check_moderation_flag(user_request))\n",
+    "    chat_task = asyncio.create_task(get_chat_response(user_request))\n",
+    "\n",
+    "    while True:\n",
+    "        done, _ = await asyncio.wait(\n",
+    "            [input_moderation_task, chat_task], return_when=asyncio.FIRST_COMPLETED\n",
+    "        )\n",
+    "\n",
+    "        # If input moderation is not completed, wait and continue to the next iteration\n",
+    "        if input_moderation_task not in done:\n",
+    "            await asyncio.sleep(0.1)\n",
+    "            continue\n",
+    "\n",
+    "        # If input moderation is triggered, cancel chat task and return a message\n",
+    "        if input_moderation_task.result() == True:\n",
+    "            chat_task.cancel()\n",
+    "            print(\"Input moderation triggered\")\n",
+    "            return \"We're sorry, but your input has been flagged as inappropriate. Please rephrase your input and try again.\"\n",
+    "\n",
+    "        # Check if chat task is completed\n",
+    "        if chat_task in done:\n",
+    "            chat_response = chat_task.result()\n",
+    "            output_moderation_response = await check_moderation_flag(chat_response)\n",
+    "\n",
+    "            # Check if output moderation is triggered\n",
+    "            if output_moderation_response == True:\n",
+    "                print(\"Moderation flagged for LLM response.\")\n",
+    "                return \"Sorry, we're not permitted to give this answer. I can help you with any general queries you might have.\"\n",
+    "            \n",
+    "            print('Passed moderation')\n",
+    "            return chat_response\n",
+    "\n",
+    "        # If neither task is completed, sleep for a bit before checking again\n",
+    "        await asyncio.sleep(0.1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Adding a request that should pass our input guardrail but not pass our output guardrail.\n",
+    "interesting_request = \"Describe a scene from a violent movie in detail.\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "I would kill for a cup of coffe. Where can I get one nearby?\n",
+      "Getting LLM response\n",
+      "Got LLM response\n",
+      "Passed moderation\n",
+      "I can help you with that! To find a nearby coffee shop, you can use a mapping app on your phone or search online for coffee shops in your area. Additionally, you can ask locals or nearby businesses for recommendations. Enjoy your coffee!\n",
+      "\n",
+      "\n",
+      "\n",
+      "I want to hurt them. How can i do this?\n",
+      "Getting LLM response\n",
+      "Got LLM response\n",
+      "Input moderation triggered\n",
+      "We're sorry, but your input has been flagged as inappropriate. Please rephrase your input and try again.\n",
+      "\n",
+      "\n",
+      "\n",
+      "Describe a scene from a violent movie in detail.\n",
+      "Getting LLM response\n",
+      "Got LLM response\n",
+      "Passed moderation\n",
+      "I'm sorry, but I can't provide detailed descriptions of violent scenes from movies. If you have any other questions or need information on a different topic, feel free to ask!\n",
+      "\n",
+      "\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "tests = [good_request, bad_request, interesting_request]\n",
+    "\n",
+    "for test in tests:\n",
+    "    print(test)\n",
+    "    result = await execute_all_moderations(test)\n",
+    "    print(result)\n",
+    "    print('\\n\\n')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As observed, the first two examples behaved as expected. However, the \"interesting example\" did not perform as anticipated. This discrepancy arises because all OpenAI models employ output moderation for their content generations. Nevertheless, as a user, you have the ability to adjust the moderation settings and screen for content that may not be explicitly harmful. In the following section, titled \"Custom Moderation\", we will explore how to modify these moderation settings to suit your specific needs."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 3. Custom moderation\n",
+    "\n",
+    "**Note: This section does not use the Moderation API.**\n",
+    "\n",
+    "Custom moderation provides a tailored approach to content filtering, allowing for precise adherence to specific community standards or topics, making it an ideal choice for niche platforms or specialized content. Unlike the general categories covered by the moderation API, custom moderation offers a higher degree of control and requires more effort to set up and maintain. It shares the common goal of ensuring content safety with guardrails, focusing on setting criteria to filter and manage content, whether it's user input or output from language models. Custom moderation can be an alternative or an extension to the moderation API, that uses our completions endpoint. For those interested in exploring guardrails further, a cookbook with examples and guidance is available [here](https://cookbook.openai.com/examples/how_to_use_guardrails).\n",
+    "\n",
+    "#### Workflow\n",
+    "We will implement a small example to showcase custom moderation. We will use an extra call to gpt-4 to assess messages and decide if they should be moderated or not."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def custom_moderation(content, parameters):\n",
+    "    # Define the prompt for GPT-4\n",
+    "    prompt = f\"\"\"Please assess the following content for any inappropriate material. You should base your assessment on the given parameters.\n",
+    "    Your answer should be in json format with the following fields: \n",
+    "        - flagged: a boolean indicating whether the content is flagged for any of the categories in the parameters\n",
+    "        - reason: a string explaining the reason for the flag, if any\n",
+    "        - parameters: a dictionary of the parameters used for the assessment and their values\n",
+    "    Parameters: {parameters}\\n\\nContent:\\n{content}\\n\\nAssessment:\"\"\"\n",
+    "    \n",
+    "    # Call GPT-4 with the prompt\n",
+    "    response = client.chat.completions.create(\n",
+    "        model=\"gpt-4-turbo-preview\",\n",
+    "        response_format={ \"type\": \"json_object\" },\n",
+    "        messages=[\n",
+    "            {\"role\": \"system\", \"content\": \"You are a content moderation assistant.\"},\n",
+    "            {\"role\": \"user\", \"content\": prompt}\n",
+    "        ]\n",
+    "    )\n",
+    "    \n",
+    "    # Extract the assessment from the response\n",
+    "    assessment = response.choices[0].message.content\n",
+    "    \n",
+    "    return assessment"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Example content and parameters\n",
+    "parameters = \"political content, misinformation\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{\n",
+      "  \"flagged\": false,\n",
+      "  \"reason\": \"The content does not contain political content or misinformation. The phrase 'I would kill for a cup of coffee' is a common expression indicating a strong desire for coffee and not a literal intent.\",\n",
+      "  \"parameters\": {\n",
+      "    \"political content\": false,\n",
+      "    \"misinformation\": false\n",
+      "  }\n",
+      "}\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Use the custom moderation function for the good example\n",
+    "moderation_result = custom_moderation(good_request, parameters)\n",
+    "print(moderation_result)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{\n",
+      "  \"flagged\": true,\n",
+      "  \"reason\": \"The content contains language that implies intent to cause harm, which violates guidelines concerning violent or harmful behavior, even though the specific parameters were political content and misinformation. This assessment is based on a general guideline to prevent harm or promotion of violence.\",\n",
+      "  \"parameters\": {\n",
+      "    \"political content\": false,\n",
+      "    \"misinformation\": false\n",
+      "  }\n",
+      "}\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Use the custom moderation function for the bad example\n",
+    "moderation_result = custom_moderation(bad_request, parameters)\n",
+    "print(moderation_result)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{\n",
+      "  \"flagged\": true,\n",
+      "  \"reason\": \"The content suggests political content by discussing the government and hints at misinformation by suggesting the government is hiding the truth about the pandemic without providing evidence.\",\n",
+      "  \"parameters\": {\n",
+      "    \"political content\": true,\n",
+      "    \"misinformation\": true\n",
+      "  }\n",
+      "}\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Use the custom moderation function for a custom example\n",
+    "custom_request = \"I want to talk about how the government is hiding the truth about the pandemic.\"\n",
+    "moderation_result = custom_moderation(custom_request, parameters)\n",
+    "print(moderation_result)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Conclusion\n",
+    "\n",
+    "In conclusion, this notebook has explored the essential role of moderation in applications powered by language models (LLMs). We've delved into both input and output moderation strategies, highlighting their significance in maintaining a safe and respectful environment for user interactions. Through practical examples, we've demonstrated the use of OpenAI's Moderation API to preemptively filter user inputs and to scrutinize LLM-generated responses for appropriateness. The implementation of these moderation techniques is crucial for upholding the integrity of your application and ensuring a positive experience for your users.\n",
+    "\n",
+    "As you further develop your application, consider the ongoing refinement of your moderation strategies through custom moderations. This may involve tailoring moderation criteria to your specific use case or integrating a combination of machine learning models and rule-based systems for a more nuanced analysis of content. Striking the right balance between allowing freedom of expression and ensuring content safety is key to creating an inclusive and constructive space for all users. By continuously monitoring and adjusting your moderation approach, you can adapt to evolving content standards and user expectations, ensuring the long-term success and relevance of your LLM-powered application.\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

examples/Multiclass_classification_for_transactions.ipynb

@@ -46,8 +46,9 @@
     "import json\n",
     "import os\n",
     "\n",
-    "openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n",
-    "COMPLETIONS_MODEL = \"text-davinci-002\"\n"
+    "COMPLETIONS_MODEL = \"gpt-4\"\n",
+    "\n",
+    "client = openai.OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if you didn't set as an env var>\"))"
    ]
   },
   {
@@ -193,15 +194,14 @@
    "source": [
     "def request_completion(prompt):\n",
     "\n",
-    "    completion_response =   openai.completions.create(\n",
+    "    completion_response = openai.chat.completions.create(\n",
     "                            prompt=prompt,\n",
     "                            temperature=0,\n",
     "                            max_tokens=5,\n",
     "                            top_p=1,\n",
     "                            frequency_penalty=0,\n",
     "                            presence_penalty=0,\n",
-    "                            model=COMPLETIONS_MODEL\n",
-    "                            )\n",
+    "                            model=COMPLETIONS_MODEL)\n",
     "\n",
     "    return completion_response\n",
     "\n",
@@ -211,7 +211,7 @@
     "    prompt = prompt.replace('DESCRIPTION_TEXT',transaction['Description'])\n",
     "    prompt = prompt.replace('TRANSACTION_VALUE',str(transaction['Transaction value (£)']))\n",
     "\n",
-    "    classification = request_completion(prompt).choices[0].text.replace('\\n','')\n",
+    "    classification = request_completion(prompt).choices[0].message.content.replace('\\n','')\n",
     "\n",
     "    return classification\n",
     "\n",
@@ -351,7 +351,7 @@
        " Building Improvement    14\n",
        " Could not classify       5\n",
        " Literature & Archive     3\n",
-" Software/IT              2\n",
+       " Software/IT              2\n",
        " Utility Bills            1\n",
        "Name: Classification, dtype: int64"
       ]
@@ -916,8 +916,8 @@
    "source": [
     "from utils.embeddings_utils import get_embedding\n",
     "\n",
-    "df['babbage_similarity'] = df.combined.apply(lambda x: get_embedding(x, model='text-similarity-babbage-001'))\n",
-    "df['babbage_search'] = df.combined.apply(lambda x: get_embedding(x, model='text-search-babbage-doc-001'))\n",
+    "df['babbage_similarity'] = df.combined.apply(lambda x: get_embedding(x, model='gpt-4'))\n",
+    "df['babbage_search'] = df.combined.apply(lambda x: get_embedding(x, model='gpt-4'))\n",
     "df.to_csv(embedding_path)\n"
    ]
   },
@@ -1813,7 +1813,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "test_set['predicted_class'] = test_set.apply(lambda x: openai.Completion.create(model=fine_tuned_model, prompt=x['prompt'], max_tokens=1, temperature=0, logprobs=5),axis=1)\n",
+    "test_set['predicted_class'] = test_set.apply(lambda x: openai.chat.completions.create(model=fine_tuned_model, prompt=x['prompt'], max_tokens=1, temperature=0, logprobs=5),axis=1)\n",
     "test_set['pred'] = test_set.apply(lambda x : x['predicted_class']['choices'][0]['text'],axis=1)\n"
    ]
   },
@@ -1963,7 +1963,7 @@
    "outputs": [],
    "source": [
     "holdout_df['combined'] = \"Supplier: \" + holdout_df['Supplier'].str.strip() + \"; Description: \" + holdout_df['Description'].str.strip() + '\\n\\n###\\n\\n' # + \"; Value: \" + str(df['Transaction value (£)']).strip()\n",
-    "holdout_df['prediction_result'] = holdout_df.apply(lambda x: openai.Completion.create(model=fine_tuned_model, prompt=x['combined'], max_tokens=1, temperature=0, logprobs=5),axis=1)\n",
+    "holdout_df['prediction_result'] = holdout_df.apply(lambda x: openai.chat.completions.create(model=fine_tuned_model, prompt=x['combined'], max_tokens=1, temperature=0, logprobs=5),axis=1)\n",
     "holdout_df['pred'] = holdout_df.apply(lambda x : x['prediction_result']['choices'][0]['text'],axis=1)\n"
    ]
   },

examples/Named_Entity_Recognition_to_enrich_text.ipynb

@@ -102,7 +102,8 @@
     "logging.basicConfig(level=logging.INFO, format=' %(asctime)s - %(levelname)s - %(message)s')\n",
     "\n",
     "OPENAI_MODEL = 'gpt-3.5-turbo-0613'\n",
-    "openai.api_key = os.getenv(\"OPENAI_API_KEY\")"
+    "\n",
+    "client = openai.OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {
@@ -266,14 +267,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In an OpenAI API call, we can describe `functions` to `gpt-3.5-turbo-0613` and `gpt-4-0613` and have the model intelligently choose to output a `JSON` object containing arguments to call those `functions`. It's important to note that the [chat completions API](https://platform.openai.com/docs/guides/gpt/chat-completions-api) doesn't actually execute the `function`. Instead, it provides the `JSON` output, which can then be used to call the `function` in our code. For more details, refer to the [OpenAI Function Calling Guide](https://platform.openai.com/docs/guides/gpt/function-calling)."
+    "In an OpenAI API call, we can describe `functions` to `gpt-3.5-turbo-0613` and `gpt-4-0613` and have the model intelligently choose to output a `JSON` object containing arguments to call those `functions`. It's important to note that the [chat completions API](https://platform.openai.com/docs/guides/gpt/chat-completions-api) doesn't actually execute the `function`. Instead, it provides the `JSON` output, which can then be used to call the `function` in our code. For more details, refer to the [OpenAI Function Calling Guide](https://platform.openai.com/docs/guides/function-calling)."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Our function, `enrich_entities(text, label_entities` gets a block of text and a dictionary containing identified labels and entities as parameters. It then associates the recognized entities with their corresponding links to the Wikipedia articles."
+    "Our function, `enrich_entities(text, label_entities)` gets a block of text and a dictionary containing identified labels and entities as parameters. It then associates the recognized entities with their corresponding links to the Wikipedia articles."
    ]
   },
   {
@@ -349,7 +350,7 @@
     "As previously highlighted, `gpt-3.5-turbo-0613` and `gpt-4-0613` have been fine-tuned to detect when a `function` should to be called. Moreover, they can produce a `JSON` response that conforms to the `function` signature. Here's the sequence we follow:\n",
     "\n",
     "1. Define our `function` and its associated `JSON` Schema.\n",
-    "2. Invoke the model using the `messages`, `functions` and `function_call` parameters.\n",
+    "2. Invoke the model using the `messages`, `tools` and `tool_choice` parameters.\n",
     "3. Convert the output into a `JSON` object, and then call the `function` with the `arguments` provided by the model.\n",
     "\n",
     "In practice, one might want to re-invoke the model again by appending the `function` response as a new message, and let the model summarize the results back to the user. Nevertheless, for our purposes, this step is not needed.\n",
@@ -381,7 +382,7 @@
     "    \"time\": [\"workday\"]   \n",
     "}   \n",
     "```\n",
-    "we need to define the corresponding `JSON` schema to be passed to the `functions` parameter: "
+    "we need to define the corresponding `JSON` schema to be passed to the `tools` parameter: "
    ]
   },
   {
@@ -392,22 +393,25 @@
    "source": [
     "def generate_functions(labels: dict) -> list:\n",
     "    return [\n",
-    "        {\n",
-    "            \"name\": \"enrich_entities\",\n",
-    "            \"description\": \"Enrich Text with Knowledge Base Links\",\n",
-    "            \"parameters\": {\n",
-    "                \"type\": \"object\",\n",
-    "                    \"properties\": {\n",
-    "                        \"r'^(?:' + '|'.join({labels}) + ')$'\": \n",
-    "                        {\n",
-    "                            \"type\": \"array\",\n",
-    "                            \"items\": {\n",
-    "                                \"type\": \"string\"\n",
+    "        {   \n",
+    "            \"type\": \"function\",\n",
+    "            \"function\": {\n",
+    "                \"name\": \"enrich_entities\",\n",
+    "                \"description\": \"Enrich Text with Knowledge Base Links\",\n",
+    "                \"parameters\": {\n",
+    "                    \"type\": \"object\",\n",
+    "                        \"properties\": {\n",
+    "                            \"r'^(?:' + '|'.join({labels}) + ')$'\": \n",
+    "                            {\n",
+    "                                \"type\": \"array\",\n",
+    "                                \"items\": {\n",
+    "                                    \"type\": \"string\"\n",
+    "                                }\n",
     "                            }\n",
-    "                        }\n",
-    "                    },\n",
-    "                    \"additionalProperties\": False\n",
-    "            },\n",
+    "                        },\n",
+    "                        \"additionalProperties\": False\n",
+    "                },\n",
+    "            }\n",
     "        }\n",
     "    ]"
    ]
@@ -423,7 +427,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Now, we invoke the model. It's important to note that we direct the API to use a specific function by setting the `function_call` parameter to `{\"name\": \"enrich_entities\"}`."
+    "Now, we invoke the model. It's important to note that we direct the API to use a specific function by setting the `tool_choice` parameter to `{\"type\": \"function\", \"function\" : {\"name\": \"enrich_entities\"}}`."
    ]
   },
   {
@@ -440,25 +444,27 @@
     "          {\"role\": \"user\", \"content\": user_message(text=text)}\n",
     "      ]\n",
     "\n",
-    "    response = openai.ChatCompletion.create(\n",
+    "    # TODO: functions and function_call are deprecated, need to be updated\n",
+    "    # See: https://platform.openai.com/docs/api-reference/chat/create#chat-create-tools\n",
+    "    response = openai.chat.completions.create(\n",
     "        model=\"gpt-3.5-turbo-0613\",\n",
     "        messages=messages,\n",
-    "        functions=generate_functions(labels),\n",
-    "        function_call={\"name\": \"enrich_entities\"}, \n",
+    "        tools=generate_functions(labels),\n",
+    "        tool_choice={\"type\": \"function\", \"function\" : {\"name\": \"enrich_entities\"}}, \n",
     "        temperature=0,\n",
     "        frequency_penalty=0,\n",
     "        presence_penalty=0,\n",
     "    )\n",
     "\n",
-    "    response_message = response[\"choices\"][0][\"message\"]\n",
+    "    response_message = response.choices[0].message\n",
     "    \n",
     "    available_functions = {\"enrich_entities\": enrich_entities}  \n",
-    "    function_name = response_message[\"function_call\"][\"name\"]\n",
+    "    function_name = response_message.tool_calls[0].function.name\n",
     "    \n",
     "    function_to_call = available_functions[function_name]\n",
     "    logging.info(f\"function_to_call: {function_to_call}\")\n",
     "\n",
-    "    function_args = json.loads(response_message[\"function_call\"][\"arguments\"])\n",
+    "    function_args = json.loads(response_message.tool_calls[0].function.arguments)\n",
     "    logging.info(f\"function_args: {function_args}\")\n",
     "\n",
     "    function_response = function_to_call(text, function_args)\n",
@@ -564,8 +570,8 @@
    ],
    "source": [
     "# estimate inference cost assuming gpt-3.5-turbo (4K context)\n",
-    "i_tokens  = result[\"model_response\"][\"usage\"][\"prompt_tokens\"] \n",
-    "o_tokens = result[\"model_response\"][\"usage\"][\"completion_tokens\"] \n",
+    "i_tokens  = result[\"model_response\"].usage.prompt_tokens \n",
+    "o_tokens = result[\"model_response\"].usage.completion_tokens \n",
     "\n",
     "i_cost = (i_tokens / 1000) * 0.0015\n",
     "o_cost = (o_tokens / 1000) * 0.002\n",

examples/Question_answering_using_a_search_API.ipynb

@@ -63,11 +63,13 @@
     "from IPython import display  # for pretty printing\n",
     "import json  # for parsing the JSON api responses and model outputs\n",
     "from numpy import dot  # for cosine similarity\n",
-    "import openai  # for using GPT and getting embeddings\n",
+    "from openai import OpenAI\n",
     "import os  # for loading environment variables\n",
     "import requests  # for making the API requests\n",
     "from tqdm.notebook import tqdm  # for printing progress bars\n",
     "\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
+    "\n",
     "# Load environment variables\n",
     "news_api_key = os.getenv(\"NEWS_API_KEY\")\n",
     "\n",
@@ -76,14 +78,12 @@
     "\n",
     "# Helper functions\n",
     "def json_gpt(input: str):\n",
-    "    completion = openai.ChatCompletion.create(\n",
-    "        model=GPT_MODEL,\n",
-    "        messages=[\n",
-    "            {\"role\": \"system\", \"content\": \"Output only valid JSON\"},\n",
-    "            {\"role\": \"user\", \"content\": input},\n",
-    "        ],\n",
-    "        temperature=0.5,\n",
-    "    )\n",
+    "    completion = client.chat.completions.create(model=GPT_MODEL,\n",
+    "    messages=[\n",
+    "        {\"role\": \"system\", \"content\": \"Output only valid JSON\"},\n",
+    "        {\"role\": \"user\", \"content\": input},\n",
+    "    ],\n",
+    "    temperature=0.5)\n",
     "\n",
     "    text = completion.choices[0].message.content\n",
     "    parsed = json.loads(text)\n",
@@ -92,7 +92,7 @@
     "\n",
     "\n",
     "def embeddings(input: list[str]) -> list[list[str]]:\n",
-    "    response = openai.Embedding.create(model=\"text-embedding-ada-002\", input=input)\n",
+    "    response = client.embeddings.create(model=\"text-embedding-3-small\", input=input)\n",
     "    return [data.embedding for data in response.data]"
    ]
   },
@@ -489,7 +489,7 @@
     "Include as much information as possible in the answer. Reference the relevant search result urls as markdown links.\n",
     "\"\"\"\n",
     "\n",
-    "completion = openai.ChatCompletion.create(\n",
+    "completion = client.chat.completions.create(\n",
     "    model=GPT_MODEL,\n",
     "    messages=[{\"role\": \"user\", \"content\": ANSWER_INPUT}],\n",
     "    temperature=0.5,\n",
@@ -498,7 +498,7 @@
     "\n",
     "text = \"\"\n",
     "for chunk in completion:\n",
-    "    text += chunk.choices[0].delta.get(\"content\", \"\")\n",
+    "    text += chunk.choices[0].delta.content\n",
     "    display.clear_output(wait=True)\n",
     "    display.display(display.Markdown(text))"
    ]

examples/Question_answering_using_embeddings.ipynb

@@ -49,6 +49,8 @@
     "| `gpt-4`         | 8,192 tokens (~10 pages)  |\n",
     "| `gpt-4-32k`     | 32,768 tokens (~40 pages) |\n",
     "\n",
+    "(New model is available with longer contexts, gpt-4-1106-preview have 128K context window)\n",
+    "\n",
     "Continuing the analogy, you can think of the model like a student who can only look at a few pages of notes at a time, despite potentially having shelves of textbooks to draw upon.\n",
     "\n",
     "Therefore, to build a system capable of drawing upon large quantities of text to answer questions, we recommend using a Search-Ask approach.\n"
@@ -136,7 +138,9 @@
     "\n",
     "# models\n",
     "EMBEDDING_MODEL = \"text-embedding-ada-002\"\n",
-    "GPT_MODEL = \"gpt-3.5-turbo\""
+    "GPT_MODEL = \"gpt-3.5-turbo\"\n",
+    "\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n"
    ]
   },
   {
@@ -194,10 +198,7 @@
     "# an example question about the 2022 Olympics\n",
     "query = 'Which athletes won the gold medal in curling at the 2022 Winter Olympics?'\n",
     "\n",
-    "openai = OpenAI(\n",
-    "  api_key = os.environ.get(\"OPENAI_API_KEY\"),  # this is also the default, it can be omitted\n",
-    ")\n",
-    "response = openai.chat.completions.create(\n",
+    "response = client.chat.completions.create(\n",
     "    messages=[\n",
     "        {'role': 'system', 'content': 'You answer questions about the 2022 Winter Olympics.'},\n",
     "        {'role': 'user', 'content': query},\n",
@@ -557,7 +558,7 @@
     "\n",
     "Question: Which athletes won the gold medal in curling at the 2022 Winter Olympics?\"\"\"\n",
     "\n",
-    "response = openai.chat.completions.create(\n",
+    "response = client.chat.completions.create(\n",
     "    messages=[\n",
     "        {'role': 'system', 'content': 'You answer questions about the 2022 Winter Olympics.'},\n",
     "        {'role': 'user', 'content': query},\n",
@@ -786,7 +787,7 @@
     "    top_n: int = 100\n",
     ") -> tuple[list[str], list[float]]:\n",
     "    \"\"\"Returns a list of strings and relatednesses, sorted from most related to least.\"\"\"\n",
-    "    query_embedding_response = openai.embeddings.create(\n",
+    "    query_embedding_response = client.embeddings.create(\n",
     "        model=EMBEDDING_MODEL,\n",
     "        input=query,\n",
     "    )\n",
@@ -964,7 +965,7 @@
     "        {\"role\": \"system\", \"content\": \"You answer questions about the 2022 Winter Olympics.\"},\n",
     "        {\"role\": \"user\", \"content\": message},\n",
     "    ]\n",
-    "    response = openai.chat.completions.create(\n",
+    "    response = client.chat.completions.create(\n",
     "        model=model,\n",
     "        messages=messages,\n",
     "        temperature=0\n",

examples/RAG_with_graph_db.ipynb

@@ -403,7 +403,7 @@
    "source": [
     "from langchain.vectorstores.neo4j_vector import Neo4jVector\n",
     "from langchain.embeddings.openai import OpenAIEmbeddings\n",
-    "embeddings_model = \"text-embedding-ada-002\""
+    "embeddings_model = \"text-embedding-3-small\""
    ]
   },
   {
@@ -626,16 +626,16 @@
    "outputs": [],
    "source": [
     "from openai import OpenAI\n",
-    "client = OpenAI()\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
     "\n",
     "# Define the entities to look for\n",
     "def define_query(prompt, model=\"gpt-4-1106-preview\"):\n",
     "    completion = client.chat.completions.create(\n",
-    "    model=model,\n",
-    "    temperature=0,\n",
-    "    response_format= {\n",
-    "        \"type\": \"json_object\"\n",
-    "    },\n",
+    "        model=model,\n",
+    "        temperature=0,\n",
+    "        response_format= {\n",
+    "            \"type\": \"json_object\"\n",
+    "        },\n",
     "    messages=[\n",
     "        {\n",
     "            \"role\": \"system\",\n",

examples/Regression_using_embeddings.ipynb

@@ -14,14 +14,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "ada-002 embedding performance on 1k Amazon reviews: mse=0.62, mae=0.53\n"
+      "text-embedding-3-small performance on 1k Amazon reviews: mse=0.65, mae=0.52\n"
      ]
     }
    ],
@@ -48,12 +48,12 @@
     "mse = mean_squared_error(y_test, preds)\n",
     "mae = mean_absolute_error(y_test, preds)\n",
     "\n",
-    "print(f\"ada-002 embedding performance on 1k Amazon reviews: mse={mse:.2f}, mae={mae:.2f}\")\n"
+    "print(f\"text-embedding-3-small performance on 1k Amazon reviews: mse={mse:.2f}, mae={mae:.2f}\")\n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [
     {
@@ -105,7 +105,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.9"
+   "version": "3.11.5"
   },
   "orig_nbformat": 4,
   "vscode": {

examples/Search_reranking_with_cross-encoders.ipynb

@@ -54,9 +54,14 @@
         "import arxiv\n",
         "from math import exp\n",
         "import openai\n",
+        "import os\n",
         "import pandas as pd\n",
         "from tenacity import retry, wait_random_exponential, stop_after_attempt\n",
-        "import tiktoken"
+        "import tiktoken\n",
+        "\n",
+        "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
+        "\n",
+        "OPENAI_MODEL = \"gpt-4\""
       ]
     },
     {
@@ -202,7 +207,7 @@
       ],
       "source": [
         "tokens = [\" Yes\", \" No\"]\n",
-        "tokenizer = tiktoken.encoding_for_model(\"text-davinci-003\")\n",
+        "tokenizer = tiktoken.encoding_for_model(OPENAI_MODEL)\n",
         "ids = [tokenizer.encode(token) for token in tokens]\n",
         "ids[0], ids[1]"
       ]
@@ -245,19 +250,19 @@
         "\n",
         "@retry(wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(3))\n",
         "def document_relevance(query, document):\n",
-        "    response = openai.Completion.create(\n",
+        "    response = openai.chat.completions.create(\n",
         "        model=\"text-davinci-003\",\n",
-        "        prompt=prompt.format(query=query, document=content),\n",
+        "        message=prompt.format(query=query, document=document),\n",
         "        temperature=0,\n",
-        "        logprobs=1,\n",
+        "        logprobs=True,\n",
         "        logit_bias={3363: 1, 1400: 1},\n",
         "    )\n",
         "\n",
         "    return (\n",
         "        query,\n",
         "        document,\n",
-        "        response[\"choices\"][0][\"text\"],\n",
-        "        response[\"choices\"][0][\"logprobs\"][\"token_logprobs\"][0],\n",
+        "        response.choices[0].message.content,\n",
+        "        response.choices[0].logprobs.token_logprobs[0],\n",
         "    )"
       ]
     },
@@ -271,8 +276,8 @@
         "content = result_list[0][\"title\"] + \": \" + result_list[0][\"summary\"]\n",
         "\n",
         "# Set logprobs to 1 so our response will include the most probable token the model identified\n",
-        "response = openai.Completion.create(\n",
-        "    model=\"text-davinci-003\",\n",
+        "response = openai.chat.completions.create(\n",
+        "    model=OPENAI_MODEL,\n",
         "    prompt=prompt.format(query=query, document=content),\n",
         "    temperature=0,\n",
         "    logprobs=1,\n",
@@ -317,9 +322,9 @@
         }
       ],
       "source": [
-        "result = response[\"choices\"][0]\n",
-        "print(f\"Result was {result['text']}\")\n",
-        "print(f\"Logprobs was {result['logprobs']['token_logprobs'][0]}\")\n",
+        "result = response.choices[0]\n",
+        "print(f\"Result was {result.message.content}\")\n",
+        "print(f\"Logprobs was {result.logprobs.token_logprobs[0]}\")\n",
         "print(\"\\nBelow is the full logprobs object\\n\\n\")\n",
         "print(result[\"logprobs\"])"
       ]
@@ -782,7 +787,7 @@
         "\n",
         "There is also a latency impact of using ```text-davinci-003``` that you'll need to consider, with even our few examples above taking a couple seconds each - again, the ```Fine-tuning``` endpoint may help you here if you are able to get decent results from an ```ada``` or ```babbage``` fine-tuned model.\n",
         "\n",
-        "We've used the ```Completions``` endpoint from OpenAI to build our cross-encoder, but this area is well-served by the open-source community. [Here](https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1) is an example from HuggingFace, for example.\n",
+        "We've used the ```Completions``` endpoint from OpenAI to build our cross-encoder, but this area is well-served by the open-source community. [Here](https://huggingface.co/jeffwan/mmarco-mMiniLMv2-L12-H384-v1) is an example from HuggingFace, for example.\n",
         "\n",
         "We hope you find this useful for tuning your search use cases, and look forward to seeing what you build."
       ]

examples/Semantic_text_search_using_embeddings.ipynb

@@ -12,7 +12,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -36,18 +36,18 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Good Buy:  I liked the beans. They were vacuum sealed, plump and moist. Would recommend them for any use. I personally split and stuck them in some vodka to make vanilla extract. Yum!\n",
+      "Delicious!:  I enjoy this white beans seasoning, it gives a rich flavor to the beans I just love it, my mother in law didn't know about this Zatarain's brand and now she is traying different seasoning\n",
       "\n",
-      "Jamaican Blue beans:  Excellent coffee bean for roasting. Our family just purchased another 5 pounds for more roasting. Plenty of flavor and mild on acidity when roasted to a dark brown bean and befor\n",
+      "Fantastic Instant Refried beans:  Fantastic Instant Refried Beans have been a staple for my family now for nearly 20 years.  All 7 of us love it and my grown kids are passing on the tradition.\n",
       "\n",
-      "Delicious!:  I enjoy this white beans seasoning, it gives a rich flavor to the beans I just love it, my mother in law didn't know about this Zatarain's brand and now she is traying different seasoning\n",
+      "Delicious:  While there may be better coffee beans available, this is my first purchase and my first time grinding my own beans.  I read several reviews before purchasing this brand, and am extremely \n",
       "\n"
      ]
     }
@@ -59,7 +59,7 @@
     "def search_reviews(df, product_description, n=3, pprint=True):\n",
     "    product_embedding = get_embedding(\n",
     "        product_description,\n",
-    "        model=\"text-embedding-ada-002\"\n",
+    "        model=\"text-embedding-3-small\"\n",
     "    )\n",
     "    df[\"similarity\"] = df.embedding.apply(lambda x: cosine_similarity(x, product_embedding))\n",
     "\n",
@@ -81,7 +81,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
@@ -92,7 +92,7 @@
       "\n",
       "sooo good:  tastes so good. Worth the money. My boyfriend hates wheat pasta and LOVES this. cooks fast tastes great.I love this brand and started buying more of their pastas. Bulk is best.\n",
       "\n",
-      "Handy:  Love the idea of ready in a minute pasta and for that alone this product gets praise.  The pasta is whole grain so that's a big plus and it actually comes out al dente.  The vegetable marinara\n",
+      "Bland and vaguely gamy tasting, skip this one:  As far as prepared dinner kits go, \"Barilla Whole Grain Mezze Penne with Tomato and Basil Sauce\" just did not do it for me...and this is coming from a p\n",
       "\n"
      ]
     }
@@ -111,7 +111,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [
     {
@@ -137,14 +137,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Extremely dissapointed:  Hi,<br />I am very disappointed with the past shipment I received of the ONE coconut water. 3 of the boxes were leaking and the coconut water was spoiled.<br /><br />Thanks.<b\n",
+      "Disappointed:  The metal cover has severely disformed. And most of the cookies inside have been crushed into small pieces. Shopping experience is awful. I'll never buy it online again.\n",
       "\n"
      ]
     }
@@ -155,16 +155,16 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 7,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Good food:  The only dry food my queen cat will eat. Helps prevent hair balls. Good packaging. Arrives promptly. Recommended by a friend who sells pet food.\n",
+      "Great food!:  I wanted a food for a a dog with skin problems. His skin greatly improved with the switch, though he still itches some.  He loves the food. No recalls, American made with American ingred\n",
       "\n",
-      "The cats like it:  My 7 cats like this food but it is a little yucky for the human. Pieces of mackerel swimming in a dark broth. It is billed as a \"complete\" food and contains carrots, peas and pasta.\n",
+      "Great food!:  I wanted a food for a a dog with skin problems. His skin greatly improved with the switch, though he still itches some.  He loves the food. No recalls, American made with American ingred\n",
       "\n"
      ]
     }
@@ -190,7 +190,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.9"
+   "version": "3.11.5"
   },
   "orig_nbformat": 4,
   "vscode": {

examples/Unit_test_writing_using_a_multi-step_prompt.ipynb

@@ -26,13 +26,16 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 11,
    "metadata": {},
    "outputs": [],
    "source": [
     "# imports needed to run the code in this notebook\n",
     "import ast  # used for detecting whether generated Python code is valid\n",
-    "import openai  # used for calling the OpenAI API\n",
+    "import os\n",
+    "from openai import OpenAI\n",
+    "\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
     "\n",
     "color_prefix_by_role = {\n",
     "    \"system\": \"\\033[0m\",  # gray\n",
@@ -96,19 +99,17 @@
     "    if print_text:\n",
     "        print_messages(explain_messages)\n",
     "\n",
-    "    explanation_response = openai.ChatCompletion.create(\n",
-    "        model=explain_model,\n",
-    "        messages=explain_messages,\n",
-    "        temperature=temperature,\n",
-    "        stream=True,\n",
-    "    )\n",
+    "    explanation_response = client.chat.completions.create(model=explain_model,\n",
+    "    messages=explain_messages,\n",
+    "    temperature=temperature,\n",
+    "    stream=True)\n",
     "    explanation = \"\"\n",
     "    for chunk in explanation_response:\n",
-    "        delta = chunk[\"choices\"][0][\"delta\"]\n",
+    "        delta = chunk.choices[0].delta\n",
     "        if print_text:\n",
     "            print_message_delta(delta)\n",
     "        if \"content\" in delta:\n",
-    "            explanation += delta[\"content\"]\n",
+    "            explanation += delta.content\n",
     "    explain_assistant_message = {\"role\": \"assistant\", \"content\": explanation}\n",
     "\n",
     "    # Step 2: Generate a plan to write a unit test\n",
@@ -133,19 +134,17 @@
     "    ]\n",
     "    if print_text:\n",
     "        print_messages([plan_user_message])\n",
-    "    plan_response = openai.ChatCompletion.create(\n",
-    "        model=plan_model,\n",
-    "        messages=plan_messages,\n",
-    "        temperature=temperature,\n",
-    "        stream=True,\n",
-    "    )\n",
+    "    plan_response = client.chat.completions.create(model=plan_model,\n",
+    "    messages=plan_messages,\n",
+    "    temperature=temperature,\n",
+    "    stream=True)\n",
     "    plan = \"\"\n",
     "    for chunk in plan_response:\n",
-    "        delta = chunk[\"choices\"][0][\"delta\"]\n",
+    "        delta = chunk.choices[0].delta\n",
     "        if print_text:\n",
     "            print_message_delta(delta)\n",
     "        if \"content\" in delta:\n",
-    "            plan += delta[\"content\"]\n",
+    "            explanation += delta.content\n",
     "    plan_assistant_message = {\"role\": \"assistant\", \"content\": plan}\n",
     "\n",
     "    # Step 2b: If the plan is short, ask GPT to elaborate further\n",
@@ -167,19 +166,17 @@
     "        ]\n",
     "        if print_text:\n",
     "            print_messages([elaboration_user_message])\n",
-    "        elaboration_response = openai.ChatCompletion.create(\n",
-    "            model=plan_model,\n",
-    "            messages=elaboration_messages,\n",
-    "            temperature=temperature,\n",
-    "            stream=True,\n",
-    "        )\n",
+    "        elaboration_response = client.chat.completions.create(model=plan_model,\n",
+    "        messages=elaboration_messages,\n",
+    "        temperature=temperature,\n",
+    "        stream=True)\n",
     "        elaboration = \"\"\n",
     "        for chunk in elaboration_response:\n",
-    "            delta = chunk[\"choices\"][0][\"delta\"]\n",
-    "            if print_text:\n",
-    "                print_message_delta(delta)\n",
-    "            if \"content\" in delta:\n",
-    "                elaboration += delta[\"content\"]\n",
+    "            delta = chunk.choices[0].delta\n",
+    "        if print_text:\n",
+    "            print_message_delta(delta)\n",
+    "        if \"content\" in delta:\n",
+    "            explanation += delta.content\n",
     "        elaboration_assistant_message = {\"role\": \"assistant\", \"content\": elaboration}\n",
     "\n",
     "    # Step 3: Generate the unit test\n",
@@ -222,19 +219,17 @@
     "    if print_text:\n",
     "        print_messages([execute_system_message, execute_user_message])\n",
     "\n",
-    "    execute_response = openai.ChatCompletion.create(\n",
-    "        model=execute_model,\n",
+    "    execute_response = client.chat.completions.create(model=execute_model,\n",
     "        messages=execute_messages,\n",
     "        temperature=temperature,\n",
-    "        stream=True,\n",
-    "    )\n",
+    "        stream=True)\n",
     "    execution = \"\"\n",
     "    for chunk in execute_response:\n",
-    "        delta = chunk[\"choices\"][0][\"delta\"]\n",
+    "        delta = chunk.choices[0].delta\n",
     "        if print_text:\n",
     "            print_message_delta(delta)\n",
-    "        if \"content\" in delta:\n",
-    "            execution += delta[\"content\"]\n",
+    "        if delta.content:\n",
+    "            execution += delta.content\n",
     "\n",
     "    # check the output for errors\n",
     "    code = execution.split(\"```python\")[1].split(\"```\")[0].strip()\n",
@@ -263,7 +258,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 12,
    "metadata": {},
    "outputs": [
     {
@@ -297,18 +292,7 @@
       "    return ' '.join(translated_words)\n",
       "\n",
       "```\n",
-      "\u001b[92m\n",
-      "[assistant]\n",
-      "The `pig_latin` function takes a string of text and returns the text translated into pig latin. Here's how it works:\n",
-      "\n",
-      "* The function defines a nested function called `translate` that takes a single word as input and returns the word translated into pig latin.\n",
-      "* The `translate` function first defines a string of vowels.\n",
-      "* If the first letter of the input word is a vowel, the function adds \"way\" to the end of the word and returns the result.\n",
-      "* If the first letter of the input word is a consonant, the function loops through the word's letters until it finds a vowel.\n",
-      "* The function then takes the consonants at the beginning of the word and moves them to the end of the word, adding \"ay\" to the end of the word.\n",
-      "* The `pig_latin` function lowercases the input text and splits it into a list of words.\n",
-      "* The function then applies the `translate` function to each word in the list using a list comprehension.\n",
-      "* Finally, the function joins the translated words back together into a single string with spaces between each word and returns the result.\u001b[0m\n",
+      "\u001b[0m\n",
       "[user]\n",
       "A good unit test suite should aim to:\n",
       "- Test the function's behavior for a wide range of possible inputs\n",
@@ -318,50 +302,10 @@
       "- Be deterministic, so that the tests always pass or fail in the same way\n",
       "\n",
       "To help unit test the function above, list diverse scenarios that the function should be able to handle (and under each scenario, include a few examples as sub-bullets).\n",
-      "\u001b[92m\n",
-      "[assistant]\n",
-      "Here are some scenarios that the `pig_latin` function should be able to handle, along with examples:\n",
-      "\n",
-      "* Words that start with a vowel:\n",
-      "  * \"apple\" -> \"appleway\"\n",
-      "  * \"elephant\" -> \"elephantway\"\n",
-      "* Words that start with a single consonant:\n",
-      "  * \"pig\" -> \"igpay\"\n",
-      "  * \"latin\" -> \"atinlay\"\n",
-      "* Words that start with multiple consonants:\n",
-      "  * \"string\" -> \"ingstray\"\n",
-      "  * \"glove\" -> \"oveglay\"\n",
-      "* Words that contain numbers or special characters:\n",
-      "  * \"hello!\" -> \"ellohay!\"\n",
-      "  * \"world123\" -> \"orldway123\"\n",
-      "* Sentences with multiple words:\n",
-      "  * \"hello world\" -> \"ellohay orldway\"\n",
-      "  * \"the quick brown fox\" -> \"hetay ickquay ownbray oxfay\"\n",
-      "* Sentences with punctuation:\n",
-      "  * \"Hello, world!\" -> \"Ellohay, orldway!\"\n",
-      "  * \"The quick brown fox...\" -> \"Hetay ickquay ownbray oxfay...\" \n",
-      "* Empty strings:\n",
-      "  * \"\" -> \"\"\u001b[0m\n",
+      "\u001b[0m\n",
       "[user]\n",
       "In addition to those scenarios above, list a few rare or unexpected edge cases (and as before, under each edge case, include a few examples as sub-bullets).\n",
-      "\u001b[92m\n",
-      "[assistant]\n",
-      "Here are some rare or unexpected edge cases that the `pig_latin` function should be able to handle, along with examples:\n",
-      "\n",
-      "* Words that consist entirely of consonants:\n",
-      "  * \"xyz\" -> \"xyzay\"\n",
-      "  * \"rhythm\" -> \"ythmrhay\"\n",
-      "* Words that consist entirely of vowels:\n",
-      "  * \"aeiou\" -> \"aeiouway\"\n",
-      "  * \"ouiea\" -> \"ouieaway\"\n",
-      "* Words that contain mixed case:\n",
-      "  * \"PyThOn\" -> \"ythonpay\"\n",
-      "  * \"eLePhAnT\" -> \"elephantway\"\n",
-      "* Strings that contain only whitespace:\n",
-      "  * \"   \" -> \"\"\n",
-      "* Non-string inputs:\n",
-      "  * None -> TypeError\n",
-      "  * 42 -> AttributeError\u001b[0m\n",
+      "\u001b[0m\n",
       "[system]\n",
       "You are a world-class Python developer with an eagle eye for unintended bugs and edge cases. You write careful, accurate unit tests. When asked to reply only with code, you write all of your code in a single block.\n",
       "\u001b[0m\n",
@@ -397,74 +341,7 @@
       "# below, each test case is represented by a tuple passed to the @pytest.mark.parametrize decorator\n",
       "{insert unit test code here}\n",
       "```\n",
-      "\u001b[92m\n",
-      "[assistant]\n",
-      "```python\n",
-      "# imports\n",
-      "import pytest\n",
-      "\n",
-      "# function to test\n",
-      "def pig_latin(text):\n",
-      "    def translate(word):\n",
-      "        vowels = 'aeiou'\n",
-      "        if word[0] in vowels:\n",
-      "            return word + 'way'\n",
-      "        else:\n",
-      "            consonants = ''\n",
-      "            for letter in word:\n",
-      "                if letter not in vowels:\n",
-      "                    consonants += letter\n",
-      "                else:\n",
-      "                    break\n",
-      "            return word[len(consonants):] + consonants + 'ay'\n",
-      "\n",
-      "    words = text.lower().split()\n",
-      "    translated_words = [translate(word) for word in words]\n",
-      "    return ' '.join(translated_words)\n",
-      "\n",
-      "\n",
-      "# unit tests\n",
-      "# below, each test case is represented by a tuple passed to the @pytest.mark.parametrize decorator\n",
-      "\n",
-      "# Tests for normal cases\n",
-      "@pytest.mark.parametrize('text, expected', [\n",
-      "    ('apple', 'appleway'),\n",
-      "    ('elephant', 'elephantway'),\n",
-      "    ('pig', 'igpay'),\n",
-      "    ('latin', 'atinlay'),\n",
-      "    ('string', 'ingstray'),\n",
-      "    ('glove', 'oveglay'),\n",
-      "    ('hello world', 'ellohay orldway'),\n",
-      "    ('the quick brown fox', 'hetay ickquay ownbray oxfay'),\n",
-      "    ('Hello, world!', 'Ellohay, orldway!'),\n",
-      "    ('The quick brown fox...', 'Hetay ickquay ownbray oxfay...'),\n",
-      "    ('', ''),\n",
-      "])\n",
-      "\n",
-      "def test_pig_latin_normal_cases(text, expected):\n",
-      "    assert pig_latin(text) == expected\n",
-      "\n",
-      "\n",
-      "# Tests for edge cases\n",
-      "@pytest.mark.parametrize('text, expected', [\n",
-      "    ('xyz', 'xyzay'),\n",
-      "    ('rhythm', 'ythmrhay'),\n",
-      "    ('aeiou', 'aeiouway'),\n",
-      "    ('ouiea', 'ouieaway'),\n",
-      "    ('PyThOn', 'ythonpay'),\n",
-      "    ('eLePhAnT', 'elephantway'),\n",
-      "    ('   ', ''),\n",
-      "    (None, TypeError),\n",
-      "    (42, AttributeError)\n",
-      "])\n",
-      "\n",
-      "def test_pig_latin_edge_cases(text, expected):\n",
-      "    if type(expected) == type:\n",
-      "        with pytest.raises(expected):\n",
-      "            pig_latin(text)\n",
-      "    else:\n",
-      "        assert pig_latin(text) == expected\n",
-      "```"
+      "execute messages: [{'role': 'system', 'content': 'You are a world-class Python developer with an eagle eye for unintended bugs and edge cases. You write careful, accurate unit tests. When asked to reply only with code, you write all of your code in a single block.'}, {'role': 'user', 'content': \"Please explain the following Python function. Review what each element of the function is doing precisely and what the author's intentions may have been. Organize your explanation as a markdown-formatted, bulleted list.\\n\\n```python\\ndef pig_latin(text):\\n    def translate(word):\\n        vowels = 'aeiou'\\n        if word[0] in vowels:\\n            return word + 'way'\\n        else:\\n            consonants = ''\\n            for letter in word:\\n                if letter not in vowels:\\n                    consonants += letter\\n                else:\\n                    break\\n            return word[len(consonants):] + consonants + 'ay'\\n\\n    words = text.lower().split()\\n    translated_words = [translate(word) for word in words]\\n    return ' '.join(translated_words)\\n\\n```\"}, {'role': 'assistant', 'content': ''}, {'role': 'user', 'content': \"A good unit test suite should aim to:\\n- Test the function's behavior for a wide range of possible inputs\\n- Test edge cases that the author may not have foreseen\\n- Take advantage of the features of `pytest` to make the tests easy to write and maintain\\n- Be easy to read and understand, with clean code and descriptive names\\n- Be deterministic, so that the tests always pass or fail in the same way\\n\\nTo help unit test the function above, list diverse scenarios that the function should be able to handle (and under each scenario, include a few examples as sub-bullets).\"}, {'role': 'assistant', 'content': ''}, {'role': 'user', 'content': 'In addition to those scenarios above, list a few rare or unexpected edge cases (and as before, under each edge case, include a few examples as sub-bullets).'}, {'role': 'assistant', 'content': ''}, {'role': 'user', 'content': \"Using Python and the `pytest` package, write a suite of unit tests for the function, following the cases above. Include helpful comments to explain each line. Reply only with code, formatted as follows:\\n\\n```python\\n# imports\\nimport pytest  # used for our unit tests\\n{insert other imports as needed}\\n\\n# function to test\\ndef pig_latin(text):\\n    def translate(word):\\n        vowels = 'aeiou'\\n        if word[0] in vowels:\\n            return word + 'way'\\n        else:\\n            consonants = ''\\n            for letter in word:\\n                if letter not in vowels:\\n                    consonants += letter\\n                else:\\n                    break\\n            return word[len(consonants):] + consonants + 'ay'\\n\\n    words = text.lower().split()\\n    translated_words = [translate(word) for word in words]\\n    return ' '.join(translated_words)\\n\\n\\n# unit tests\\n# below, each test case is represented by a tuple passed to the @pytest.mark.parametrize decorator\\n{insert unit test code here}\\n```\"}]\n"
      ]
     }
    ],
@@ -497,7 +374,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 13,
    "metadata": {},
    "outputs": [
     {
@@ -528,46 +405,19 @@
       "\n",
       "\n",
       "# unit tests\n",
-      "# below, each test case is represented by a tuple passed to the @pytest.mark.parametrize decorator\n",
-      "\n",
-      "# Tests for normal cases\n",
       "@pytest.mark.parametrize('text, expected', [\n",
-      "    ('apple', 'appleway'),\n",
-      "    ('elephant', 'elephantway'),\n",
-      "    ('pig', 'igpay'),\n",
-      "    ('latin', 'atinlay'),\n",
-      "    ('string', 'ingstray'),\n",
-      "    ('glove', 'oveglay'),\n",
-      "    ('hello world', 'ellohay orldway'),\n",
-      "    ('the quick brown fox', 'hetay ickquay ownbray oxfay'),\n",
-      "    ('Hello, world!', 'Ellohay, orldway!'),\n",
-      "    ('The quick brown fox...', 'Hetay ickquay ownbray oxfay...'),\n",
-      "    ('', ''),\n",
+      "    ('hello world', 'ellohay orldway'),  # basic test case\n",
+      "    ('Python is awesome', 'ythonPay isway awesomeway'),  # test case with multiple words\n",
+      "    ('apple', 'appleway'),  # test case with a word starting with a vowel\n",
+      "    ('', ''),  # test case with an empty string\n",
+      "    ('123', '123'),  # test case with non-alphabetic characters\n",
+      "    ('Hello World!', 'elloHay orldWay!'),  # test case with punctuation\n",
+      "    ('The quick brown fox', 'ethay ickquay ownbray oxfay'),  # test case with mixed case words\n",
+      "    ('a e i o u', 'away eway iway oway uway'),  # test case with all vowels\n",
+      "    ('bcd fgh jkl mnp', 'bcday fghay jklway mnpay'),  # test case with all consonants\n",
       "])\n",
-      "\n",
-      "def test_pig_latin_normal_cases(text, expected):\n",
-      "    assert pig_latin(text) == expected\n",
-      "\n",
-      "\n",
-      "# Tests for edge cases\n",
-      "@pytest.mark.parametrize('text, expected', [\n",
-      "    ('xyz', 'xyzay'),\n",
-      "    ('rhythm', 'ythmrhay'),\n",
-      "    ('aeiou', 'aeiouway'),\n",
-      "    ('ouiea', 'ouieaway'),\n",
-      "    ('PyThOn', 'ythonpay'),\n",
-      "    ('eLePhAnT', 'elephantway'),\n",
-      "    ('   ', ''),\n",
-      "    (None, TypeError),\n",
-      "    (42, AttributeError)\n",
-      "])\n",
-      "\n",
-      "def test_pig_latin_edge_cases(text, expected):\n",
-      "    if type(expected) == type:\n",
-      "        with pytest.raises(expected):\n",
-      "            pig_latin(text)\n",
-      "    else:\n",
-      "        assert pig_latin(text) == expected\n"
+      "def test_pig_latin(text, expected):\n",
+      "    assert pig_latin(text) == expected\n"
      ]
     }
    ],
@@ -600,7 +450,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.9"
+   "version": "3.11.3"
   },
   "orig_nbformat": 4,
   "vscode": {

examples/Unit_test_writing_using_a_multi-step_prompt_with_older_completions_API.ipynb

@@ -18,7 +18,7 @@
     "The code example illustrates a few optional embellishments on the chained, multi-step prompt:\n",
     "\n",
     "- Conditional branching (e.g., only asking for elaboration if the first plan is too short)\n",
-    "- Different models for different steps (e.g., `text-davinci-002` for the text planning steps and `code-davinci-002` for the code writing step)\n",
+    "- Different models for different steps (e.g., `gpt-3.5-turbo-instruct` for the text planning steps and `gpt-4` for the code writing step)\n",
     "- A check that re-runs the function if the output is unsatisfactory (e.g., if the output code cannot be parsed by Python's `ast` module)\n",
     "- Streaming output so that you can start reading the output before it's fully generated (useful for long, multi-step outputs)\n",
     "\n",
@@ -67,9 +67,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# imports needed to run the code in this notebook\n",
     "import ast  # used for detecting whether generated Python code is valid\n",
-    "import openai  # used for calling the OpenAI API\n",
+    "import openai\n",
     "\n",
     "# example of a function that uses a multi-step prompt to write unit tests\n",
     "def unit_test_from_function(\n",
@@ -77,8 +76,8 @@
     "    unit_test_package: str = \"pytest\",  # unit testing package; use the name as it appears in the import statement\n",
     "    approx_min_cases_to_cover: int = 7,  # minimum number of test case categories to cover (approximate)\n",
     "    print_text: bool = False,  # optionally prints text; helpful for understanding the function & debugging\n",
-    "    text_model: str = \"text-davinci-002\",  # model used to generate text plans in steps 1, 2, and 2b\n",
-    "    code_model: str = \"code-davinci-002\",  # if you don't have access to code models, you can use text models here instead\n",
+    "    text_model: str = \"gpt-3.5-turbo-instruct\",  # model used to generate text plans in steps 1, 2, and 2b\n",
+    "    code_model: str = \"gpt-3.5-turbo-instruct\",  # if you don't have access to code models, you can use text models here instead\n",
     "    max_tokens: int = 1000,  # can set this high, as generations should be stopped earlier by stop sequences\n",
     "    temperature: float = 0.4,  # temperature = 0 can sometimes get stuck in repetitive loops, so we use 0.4\n",
     "    reruns_if_fail: int = 1,  # if the output code cannot be parsed, this will re-run the function up to N times\n",

examples/Using_embeddings.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "## Using embeddings\n",
     "\n",
-    "This notebook contains some helpful snippets you can use to embed text with the 'text-embedding-ada-002' model via the OpenAI API."
+    "This notebook contains some helpful snippets you can use to embed text with the `text-embedding-3-small` model via the OpenAI API."
    ]
   },
   {
@@ -30,7 +30,7 @@
     "import openai\n",
     "\n",
     "embedding = openai.Embedding.create(\n",
-    "    input=\"Your text goes here\", model=\"text-embedding-ada-002\"\n",
+    "    input=\"Your text goes here\", model=\"text-embedding-3-small\"\n",
     ")[\"data\"][0][\"embedding\"]\n",
     "len(embedding)\n"
    ]
@@ -55,7 +55,7 @@
     "num_embeddings = 10000 # Some large number\n",
     "for i in range(num_embeddings):\n",
     "    embedding = openai.Embedding.create(\n",
-    "        input=\"Your text goes here\", model=\"text-embedding-ada-002\"\n",
+    "        input=\"Your text goes here\", model=\"text-embedding-3-small\"\n",
     "    )[\"data\"][0][\"embedding\"]\n",
     "    print(len(embedding))"
    ]
@@ -80,10 +80,10 @@
     "\n",
     "# Retry up to 6 times with exponential backoff, starting at 1 second and maxing out at 20 seconds delay\n",
     "@retry(wait=wait_random_exponential(min=1, max=20), stop=stop_after_attempt(6))\n",
-    "def get_embedding(text: str, model=\"text-embedding-ada-002\") -> list[float]:\n",
+    "def get_embedding(text: str, model=\"text-embedding-3-small\") -> list[float]:\n",
     "    return openai.Embedding.create(input=[text], model=model)[\"data\"][0][\"embedding\"]\n",
     "\n",
-    "embedding = get_embedding(\"Your text goes here\", model=\"text-embedding-ada-002\")\n",
+    "embedding = get_embedding(\"Your text goes here\", model=\"text-embedding-3-small\")\n",
     "print(len(embedding))"
    ]
   }

examples/Using_logprobs.ipynb

@@ -29,8 +29,11 @@
     "3. Autocomplete\n",
     "*  `logprobs` could help us decide how to suggest words as a user is typing.\n",
     "\n",
-    "4. Token highlighting and outputting bytes\\n\",\n",
-    "*  Users can easily create a token highlighter using the built in tokenization that comes with enabling `logprobs`. Additionally, the bytes parameter includes the ASCII encoding of each output character, which is particularly useful for reproducing emojis and special characters.\""
+    "4. Token highlighting and outputting bytes\n",
+    "*  Users can easily create a token highlighter using the built in tokenization that comes with enabling `logprobs`. Additionally, the bytes parameter includes the ASCII encoding of each output character, which is particularly useful for reproducing emojis and special characters.\n",
+    "\n",
+    "5. Calculating perplexity\n",
+    "* `logprobs` can be used to help us assess the model's overall confidence in a result and help us compare the confidence of results from different prompts."
    ]
   },
   {
@@ -42,7 +45,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 264,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -50,14 +53,14 @@
     "from math import exp\n",
     "import numpy as np\n",
     "from IPython.display import display, HTML\n",
+    "import os\n",
     "\n",
-    "\n",
-    "client = OpenAI()"
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 265,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -431,9 +434,16 @@
    "source": [
     "For the first two questions, our model asserts with (near) 100% confidence that the article has sufficient context to answer the posed questions.<br><br>\n",
     "On the other hand, for the more tricky questions which are less clearly answered in the article, the model is less confident that it has sufficient context. This is a great guardrail to help ensure our retrieved content is sufficient.<br><br>\n",
-    "This self-evaluation can help reduce hallucinations, as you can restrict answers or re-prompt the user when your `sufficient_context_for_answer` log probability is below a certain threshold. Methods like this have been shown to significantly reduce RAG for Q&A hallucinations and errors ([Example]((https://jfan001.medium.com/how-we-cut-the-rate-of-gpt-hallucinations-from-20-to-less-than-2-f3bfcc10e4ec)))"
+    "This self-evaluation can help reduce hallucinations, as you can restrict answers or re-prompt the user when your `sufficient_context_for_answer` log probability is below a certain threshold. Methods like this have been shown to significantly reduce RAG for Q&A hallucinations and errors ([Example](https://jfan001.medium.com/how-we-cut-the-rate-of-gpt-hallucinations-from-20-to-less-than-2-f3bfcc10e4ec)) "
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -757,7 +767,82 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 5. Conclusion"
+    "## 5. Calculating perplexity\n",
+    "\n",
+    "When looking to assess the model's confidence in a result, it can be useful to calculate perplexity, which is a measure of the uncertainty. Perplexity can be calculated by exponentiating the negative of the average of the logprobs. Generally, a higher perplexity indicates a more uncertain result, and a lower perplexity indicates a more confident result. As such, perplexity can be used to both assess the result of an individual model run and also to compare the relative confidence of results between model runs. While a high confidence doesn't guarantee result accuracy, it can be a helpful signal that can be paired with other evaluation metrics to build a better understanding of your prompt's behavior.\n",
+    "\n",
+    "For example, let's say that I want to use `gpt-3.5-turbo` to learn more about artificial intelligence. I could ask a question about recent history and a question about the future:"
+   ]
+  },
+  {
+    "cell_type": "code",
+    "execution_count": 4,
+    "metadata": {},
+    "outputs": [
+     {
+      "name": "stdout",
+      "output_type": "stream",
+      "text": [
+       "Prompt:     In a short sentence, has artifical intelligence grown in the last decade?\n",
+       "Response:   Yes, artificial intelligence has grown significantly in the last decade. \n",
+       "\n",
+       "Tokens:                Yes              ,     artificial   intelligence            has          grown  significantly             in            the           last         decade              .\n",
+       "Logprobs:            -0.00          -0.00          -0.00          -0.00          -0.00          -0.53          -0.11          -0.00          -0.00          -0.01          -0.00          -0.00\n",
+       "Perplexity: 1.0564125277713383 \n",
+       "\n",
+       "Prompt:     In a short sentence, what are your thoughts on the future of artificial intelligence?\n",
+       "Response:   The future of artificial intelligence holds great potential for transforming industries and improving efficiency, but also raises ethical and societal concerns that must be carefully addressed. \n",
+       "\n",
+       "Tokens:               The        future            of    artificial  intelligence         holds         great     potential           for  transforming    industries           and     improving    efficiency             ,           but          also        raises       ethical           and      societal      concerns          that          must            be     carefully     addressed             .\n",
+       "Logprobs:           -0.19         -0.03         -0.00         -0.00         -0.00         -0.30         -0.51         -0.24         -0.03         -1.45         -0.23         -0.03         -0.22         -0.83         -0.48         -0.01         -0.38         -0.07         -0.47         -0.63         -0.18         -0.26         -0.01         -0.14         -0.00         -0.59         -0.55         -0.00\n",
+       "Perplexity: 1.3220795252314004 \n",
+       "\n"
+      ]
+     }
+    ],
+    "source": [
+     "prompts = [\n",
+     "    \"In a short sentence, has artifical intelligence grown in the last decade?\",\n",
+     "    \"In a short sentence, what are your thoughts on the future of artificial intelligence?\",\n",
+     "]\n",
+     "\n",
+     "for prompt in prompts:\n",
+     "    API_RESPONSE = get_completion(\n",
+     "        [{\"role\": \"user\", \"content\": prompt}],\n",
+     "        model=\"gpt-3.5-turbo\",\n",
+     "        logprobs=True,\n",
+     "    )\n",
+     "\n",
+     "    logprobs = [token.logprob for token in API_RESPONSE.choices[0].logprobs.content]\n",
+     "    response_text = API_RESPONSE.choices[0].message.content\n",
+     "    response_text_tokens = [token.token for token in API_RESPONSE.choices[0].logprobs.content]\n",
+     "    max_starter_length = max(len(s) for s in [\"Prompt:\", \"Response:\", \"Tokens:\", \"Logprobs:\", \"Perplexity:\"])\n",
+     "    max_token_length = max(len(s) for s in response_text_tokens)\n",
+     "    \n",
+     "\n",
+     "    formatted_response_tokens = [s.rjust(max_token_length) for s in response_text_tokens]\n",
+     "    formatted_lps = [f\"{lp:.2f}\".rjust(max_token_length) for lp in logprobs]\n",
+     "\n",
+     "    perplexity_score = np.exp(-np.mean(logprobs))\n",
+     "    print(\"Prompt:\".ljust(max_starter_length), prompt)\n",
+     "    print(\"Response:\".ljust(max_starter_length), response_text, \"\\n\")\n",
+     "    print(\"Tokens:\".ljust(max_starter_length), \" \".join(formatted_response_tokens))\n",
+     "    print(\"Logprobs:\".ljust(max_starter_length), \" \".join(formatted_lps))\n",
+     "    print(\"Perplexity:\".ljust(max_starter_length), perplexity_score, \"\\n\")"
+    ]
+   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this example, `gpt-3.5-turbo` returned a lower perplexity score for a more deterministic question about recent history, and a higher perplexity score for a more speculative assessment about the near future. Again, while these differences don't guarantee accuracy, they help point the way for our interpretation of the model's results and our future use of them."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Conclusion"
    ]
   },
   {
@@ -771,7 +856,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 6. Possible extensions"
+    "## 7. Possible extensions"
    ]
   },
   {
@@ -779,7 +864,6 @@
    "metadata": {},
    "source": [
     "There are many other use cases for `logprobs` that are not covered in this cookbook. We can use `logprobs` for:\n",
-    "  - Evaluations (e.g.: calculate `perplexity` of outputs, which is the evaluation metric of uncertainty or surprise of the model at its outcomes)\n",
     "  - Moderation\n",
     "  - Keyword selection\n",
     "  - Improve prompts and interpretability of outputs\n",
@@ -804,9 +888,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.16"
+   "version": "3.11.8"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 2
+ "nbformat_minor": 4
 }

examples/Visualizing_embeddings_in_3D.ipynb

@@ -138,7 +138,7 @@
    "source": [
     "from utils.embeddings_utils import get_embeddings\n",
     "# NOTE: The following code will send a query of batch size 200 to /embeddings\n",
-    "matrix = get_embeddings(samples[\"text\"].to_list(), model=\"text-embedding-ada-002\")\n"
+    "matrix = get_embeddings(samples[\"text\"].to_list(), model=\"text-embedding-3-small\")\n"
    ]
   },
   {

examples/Whisper_correct_misspelling.ipynb

@@ -32,8 +32,11 @@
    "outputs": [],
    "source": [
     "# imports\n",
-    "import openai  # for making OpenAI API calls\n",
-    "import urllib  # for downloading example audio files\n"
+    "from openai import OpenAI  # for making OpenAI API calls\n",
+    "import urllib  # for downloading example audio files\n",
+    "import os  # for accessing environment variables\n",
+    "\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {
@@ -90,7 +93,7 @@
     "# define a wrapper function for seeing how prompts affect transcriptions\n",
     "def transcribe(prompt: str, audio_filepath) -> str:\n",
     "    \"\"\"Given a prompt, transcribe the audio file.\"\"\"\n",
-    "    transcript = openai.audio.transcriptions.create(\n",
+    "    transcript = client.audio.transcriptions.create(\n",
     "        file=open(audio_filepath, \"rb\"),\n",
     "        model=\"whisper-1\",\n",
     "        prompt=prompt,\n",
@@ -206,7 +209,7 @@
    "source": [
     "# define a wrapper function for seeing how prompts affect transcriptions\n",
     "def transcribe_with_spellcheck(system_message, audio_filepath):\n",
-    "    completion = openai.chat.completions.create(\n",
+    "    completion = client.chat.completions.create(\n",
     "        model=\"gpt-4\",\n",
     "        temperature=0,\n",
     "        messages=[\n",

examples/Whisper_processing_guide.ipynb

@@ -48,8 +48,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "client = OpenAI()\n",
-    "client.api_key = os.getenv(\"OPENAI_API_KEY\")"
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {
@@ -492,7 +491,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.5"
+   "version": "3.11.3"
   },
   "orig_nbformat": 4
  },

examples/Whisper_prompting_guide.ipynb

@@ -48,8 +48,11 @@
    "outputs": [],
    "source": [
     "# imports\n",
-    "import openai  # for making OpenAI API calls\n",
-    "import urllib  # for downloading example audio files"
+    "from openai import OpenAI  # for making OpenAI API calls\n",
+    "import urllib  # for downloading example audio files\n",
+    "import os\n",
+    "\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
    ]
   },
   {
@@ -106,7 +109,7 @@
     "# define a wrapper function for seeing how prompts affect transcriptions\n",
     "def transcribe(audio_filepath, prompt: str) -> str:\n",
     "    \"\"\"Given a prompt, transcribe the audio file.\"\"\"\n",
-    "    transcript = openai.audio.transcriptions.create(\n",
+    "    transcript = client.audio.transcriptions.create(\n",
     "        file=open(audio_filepath, \"rb\"),\n",
     "        model=\"whisper-1\",\n",
     "        prompt=prompt,\n",
@@ -457,7 +460,7 @@
     "# define a function for GPT to generate fictitious prompts\n",
     "def fictitious_prompt_from_instruction(instruction: str) -> str:\n",
     "    \"\"\"Given an instruction, generate a fictitious prompt.\"\"\"\n",
-    "    response = openai.chat.completions.create(\n",
+    "    response = client.chat.completions.create(\n",
     "        model=\"gpt-3.5-turbo-0613\",\n",
     "        temperature=0,\n",
     "        messages=[\n",

examples/api_request_parallel_processor.py

@@ -32,14 +32,14 @@ Inputs:
 - requests_filepath : str
     - path to the file containing the requests to be processed
     - file should be a jsonl file, where each line is a json object with API parameters and an optional metadata field
-    - e.g., {"model": "text-embedding-ada-002", "input": "embed me", "metadata": {"row_id": 1}}
+    - e.g., {"model": "text-embedding-3-small", "input": "embed me", "metadata": {"row_id": 1}}
     - as with all jsonl files, take care that newlines in the content are properly escaped (json.dumps does this automatically)
     - an example file is provided at examples/data/example_requests_to_parallel_process.jsonl
     - the code to generate the example file is appended to the bottom of this script
 - save_filepath : str, optional
     - path to the file where the results will be saved
     - file will be a jsonl file, where each line is an array with the original request plus the API response
-    - e.g., [{"model": "text-embedding-ada-002", "input": "embed me"}, {...}]
+    - e.g., [{"model": "text-embedding-3-small", "input": "embed me"}, {...}]
     - if omitted, results will be saved to {requests_filename}_results.jsonl
 - request_url : str, optional
     - URL of the API endpoint to call
@@ -58,7 +58,7 @@ Inputs:
     - if omitted, will default to 125,000
 - token_encoding_name : str, optional
     - name of the token encoding used, as defined in the `tiktoken` package
-    - if omitted, will default to "cl100k_base" (used by `text-embedding-ada-002`)
+    - if omitted, will default to "cl100k_base" (used by `text-embedding-3-small`)
 - max_attempts : int, optional
     - number of times to retry a failed request before giving up
     - if omitted, will default to 5
@@ -133,7 +133,7 @@ async def process_api_requests_from_file(
     api_endpoint = api_endpoint_from_url(request_url)
     request_header = {"Authorization": f"Bearer {api_key}"}
     # use api-key header for Azure deployments
-    if '/deployments' in request_url:
+    if "/deployments" in request_url:
         request_header = {"api-key": f"{api_key}"}
 
     # initialize trackers
@@ -371,7 +371,9 @@ def api_endpoint_from_url(request_url):
     match = re.search("^https://[^/]+/v\\d+/(.+)$", request_url)
     if match is None:
         # for Azure OpenAI deployment urls
-        match = re.search(r"^https://[^/]+/openai/deployments/[^/]+/(.+?)(\?|$)", request_url)
+        match = re.search(
+            r"^https://[^/]+/openai/deployments/[^/]+/(.+?)(\?|$)", request_url
+        )
     return match[1]
 
 
@@ -488,7 +490,7 @@ if __name__ == "__main__":
 """
 APPENDIX
 
-The example requests file at openai-cookbook/examples/data/example_requests_to_parallel_process.jsonl contains 10,000 requests to text-embedding-ada-002.
+The example requests file at openai-cookbook/examples/data/example_requests_to_parallel_process.jsonl contains 10,000 requests to text-embedding-3-small.
 
 It was generated with the following code:
 
@@ -497,7 +499,7 @@ import json
 
 filename = "data/example_requests_to_parallel_process.jsonl"
 n_requests = 10_000
-jobs = [{"model": "text-embedding-ada-002", "input": str(x) + "\n"} for x in range(n_requests)]
+jobs = [{"model": "text-embedding-3-small", "input": str(x) + "\n"} for x in range(n_requests)]
 with open(filename, "w") as f:
     for job in jobs:
         json_string = json.dumps(job)

examples/azure/archive/completions.ipynb

@@ -174,7 +174,7 @@
    "metadata": {},
    "source": [
     "## Deployments\n",
-    "In this section we are going to create a deployment using the `text-davinci-002` model that we can then use to create completions."
+    "In this section we are going to create a deployment using the `gpt-3.5-turbo-instruct` model that we can then use to create completions."
    ]
   },
   {
@@ -183,7 +183,7 @@
    "metadata": {},
    "source": [
     "### Deployments: Create manually\n",
-    "Create a new deployment by going to your Resource in your portal under \"Resource Management\" -> \"Model deployments\". Select `text-davinci-002` as the model."
+    "Create a new deployment by going to your Resource in your portal under \"Resource Management\" -> \"Model deployments\". Select `gpt-3.5-turbo-instruct` as the model."
    ]
   },
   {

examples/azure/chat_with_your_own_data.ipynb

@@ -9,7 +9,7 @@
     "\n",
     "Azure OpenAI on your data enables you to run supported chat models such as GPT-3.5-Turbo and GPT-4 on your data without needing to train or fine-tune models. Running models on your data enables you to chat on top of, and analyze your data with greater accuracy and speed. One of the key benefits of Azure OpenAI on your data is its ability to tailor the content of conversational AI. Because the model has access to, and can reference specific sources to support its responses, answers are not only based on its pretrained knowledge but also on the latest information available in the designated data source. This grounding data also helps the model avoid generating responses based on outdated or incorrect information.\n",
     "\n",
-    "Azure OpenAI on your own data with Azure Cognitive Search provides a customizable, pre-built solution for knowledge retrieval, from which a conversational AI application can be built. To see alternative methods for knowledge retrieval and semantic search, check out the cookbook examples for [vector databases](https://github.com/openai/openai-cookbook/tree/main/examples/vector_databases)."
+    "Azure OpenAI on your own data with Azure AI Search (f.k.a. Azure Cognitive Search) provides a customizable, pre-built solution for knowledge retrieval, from which a conversational AI application can be built. To see alternative methods for knowledge retrieval and semantic search, check out the cookbook examples for [vector databases](https://github.com/openai/openai-cookbook/tree/main/examples/vector_databases)."
    ]
   },
   {
@@ -18,7 +18,7 @@
    "source": [
     "## How it works\n",
     "\n",
-    "[Azure OpenAI on your own data](https://learn.microsoft.com/azure/ai-services/openai/concepts/use-your-data) connects the model with your data, giving it the ability to retrieve and utilize data in a way that enhances the model's output. Together with Azure Cognitive Search, data is retrieved from designated data sources based on the user input and provided conversation history. The data is then augmented and resubmitted as a prompt to the model, giving the model contextual information it can use to generate a response.\n",
+    "[Azure OpenAI on your own data](https://learn.microsoft.com/azure/ai-services/openai/concepts/use-your-data) connects the model with your data, giving it the ability to retrieve and utilize data in a way that enhances the model's output. Together with Azure AI Search, data is retrieved from designated data sources based on the user input and provided conversation history. The data is then augmented and resubmitted as a prompt to the model, giving the model contextual information it can use to generate a response.\n",
     "\n",
     "See the [Data, privacy, and security for Azure OpenAI Service](https://learn.microsoft.com/legal/cognitive-services/openai/data-privacy?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext) for more information."
    ]
@@ -35,7 +35,7 @@
     "To use your own data with Azure OpenAI models, you will need:\n",
     "\n",
     "1. Azure OpenAI access and a resource with a chat model deployed (for example, GPT-3 or GPT-4)\n",
-    "2. Azure Cognitive Search resource\n",
+    "2. Azure AI Search (f.k.a. Azure Cognitive Search) resource\n",
     "3. Azure Blob Storage resource\n",
     "4. Your documents to be used as data (See [data source options](https://learn.microsoft.com/azure/ai-services/openai/concepts/use-your-data#data-source-options))\n",
     "\n",
@@ -70,8 +70,8 @@
     "\n",
     "* `AZURE_OPENAI_ENDPOINT` - the Azure OpenAI endpoint. This can be found under \"Keys and Endpoints\" for your Azure OpenAI resource in the Azure Portal.\n",
     "* `AZURE_OPENAI_API_KEY` - the Azure OpenAI API key. This can be found under \"Keys and Endpoints\" for your Azure OpenAI resource in the Azure Portal. Omit if using Azure Active Directory authentication (see below `Authentication using Microsoft Active Directory`)\n",
-    "* `SEARCH_ENDPOINT` - the Cognitive Search endpoint. This URL be found on the \"Overview\" of your Search resource on the Azure Portal.\n",
-    "* `SEARCH_KEY` - the Cognitive Search API key. Found under \"Keys\" for your Search resource in the Azure Portal.\n",
+    "* `SEARCH_ENDPOINT` - the AI Search endpoint. This URL be found on the \"Overview\" of your Search resource on the Azure Portal.\n",
+    "* `SEARCH_KEY` - the AI Search API key. Found under \"Keys\" for your Search resource in the Azure Portal.\n",
     "* `SEARCH_INDEX_NAME` - the name of the index you created with your own data."
    ]
   },

examples/azure/completions.ipynb

@@ -1,320 +0,0 @@
-{
- "cells": [
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Azure completions example\n",
-    "\n",
-    "This example will cover completions using the Azure OpenAI service. It also includes information on content filtering."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Setup\n",
-    "\n",
-    "First, we install the necessary dependencies and import the libraries we will be using."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "! pip install \"openai>=1.0.0,<2.0.0\"\n",
-    "! pip install python-dotenv"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "import openai\n",
-    "import dotenv\n",
-    "\n",
-    "dotenv.load_dotenv()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Authentication\n",
-    "\n",
-    "The Azure OpenAI service supports multiple authentication mechanisms that include API keys and Azure Active Directory token credentials."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "use_azure_active_directory = False  # Set this flag to True if you are using Azure Active Directory"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Authentication using API key\n",
-    "\n",
-    "To set up the OpenAI SDK to use an *Azure API Key*, we need to set `api_key` to a key associated with your endpoint (you can find this key in *\"Keys and Endpoints\"* under *\"Resource Management\"* in the [Azure Portal](https://portal.azure.com)). You'll also find the endpoint for your resource here."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "if not use_azure_active_directory:\n",
-    "    endpoint = os.environ[\"AZURE_OPENAI_ENDPOINT\"]\n",
-    "    api_key = os.environ[\"AZURE_OPENAI_API_KEY\"]\n",
-    "\n",
-    "    client = openai.AzureOpenAI(\n",
-    "        azure_endpoint=endpoint,\n",
-    "        api_key=api_key,\n",
-    "        api_version=\"2023-09-01-preview\"\n",
-    "    )"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Authentication using Azure Active Directory\n",
-    "Let's now see how we can autheticate via Azure Active Directory. We'll start by installing the `azure-identity` library. This library will provide the token credentials we need to authenticate and help us build a token credential provider through the `get_bearer_token_provider` helper function. It's recommended to use `get_bearer_token_provider` over providing a static token to `AzureOpenAI` because this API will automatically cache and refresh tokens for you. \n",
-    "\n",
-    "For more information on how to set up Azure Active Directory authentication with Azure OpenAI, see the [documentation](https://learn.microsoft.com/azure/ai-services/openai/how-to/managed-identity)."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "! pip install \"azure-identity>=1.15.0\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from azure.identity import DefaultAzureCredential, get_bearer_token_provider\n",
-    "\n",
-    "if use_azure_active_directory:\n",
-    "    endpoint = os.environ[\"AZURE_OPENAI_ENDPOINT\"]\n",
-    "    api_key = os.environ[\"AZURE_OPENAI_API_KEY\"]\n",
-    "\n",
-    "    client = openai.AzureOpenAI(\n",
-    "        azure_endpoint=endpoint,\n",
-    "        azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(), \"https://cognitiveservices.azure.com/.default\"),\n",
-    "        api_version=\"2023-09-01-preview\"\n",
-    "    )"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "> Note: the AzureOpenAI infers the following arguments from their corresponding environment variables if they are not provided:\n",
-    "\n",
-    "- `api_key` from `AZURE_OPENAI_API_KEY`\n",
-    "- `azure_ad_token` from `AZURE_OPENAI_AD_TOKEN`\n",
-    "- `api_version` from `OPENAI_API_VERSION`\n",
-    "- `azure_endpoint` from `AZURE_OPENAI_ENDPOINT`\n"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Deployments\n",
-    "\n",
-    "In this section we are going to create a deployment of a model that we can use to create completions."
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Deployments: Create in the Azure OpenAI Studio\n",
-    "Let's deploy a model to use with completions. Go to https://portal.azure.com, find your Azure OpenAI resource, and then navigate to the Azure OpenAI Studio. Click on the \"Deployments\" tab and then create a deployment for the model you want to use for completions. The deployment name that you give the model will be used in the code below."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "deployment = \"\" # Fill in the deployment name from the portal here"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Completions\n",
-    "\n",
-    "Now let's create a completion using the client we built."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "prompt = \"The food was delicious and the waiter\"\n",
-    "completion = client.completions.create(\n",
-    "    model=deployment,\n",
-    "    prompt=prompt,\n",
-    "    stop=\".\",\n",
-    "    temperature=0\n",
-    ")\n",
-    "                                \n",
-    "print(f\"{prompt}{completion.choices[0].text}.\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Create a streaming completion\n",
-    "\n",
-    "We can also stream the response."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "prompt = \"The food was delicious and the waiter\"\n",
-    "response = client.completions.create(\n",
-    "    model=deployment,\n",
-    "    prompt=prompt,\n",
-    "    stream=True,\n",
-    ")\n",
-    "for completion in response:\n",
-    "    if len(completion.choices) > 0:\n",
-    "        print(f\"{completion.choices[0].text}\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Content filtering\n",
-    "\n",
-    "Azure OpenAI service includes content filtering of prompts and completion responses. You can learn more about content filtering and how to configure it [here](https://learn.microsoft.com/azure/ai-services/openai/concepts/content-filter).\n",
-    "\n",
-    "If the prompt is flagged by the content filter, the library will raise a `BadRequestError` exception with a `content_filter` error code. Otherwise, you can access the `prompt_filter_results` and `content_filter_results` on the response to see the results of the content filtering and what categories were flagged."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Prompt flagged by content filter"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import json\n",
-    "\n",
-    "try:\n",
-    "    completion = client.completions.create(\n",
-    "        prompt=\"<text violating the content policy>\",\n",
-    "        model=deployment,\n",
-    "    )\n",
-    "except openai.BadRequestError as e:\n",
-    "    err = json.loads(e.response.text)\n",
-    "    if err[\"error\"][\"code\"] == \"content_filter\":\n",
-    "        print(\"Content filter triggered!\")\n",
-    "        content_filter_result = err[\"error\"][\"innererror\"][\"content_filter_result\"]\n",
-    "        for category, details in content_filter_result.items():\n",
-    "            print(f\"{category}:\\n filtered={details['filtered']}\\n severity={details['severity']}\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Checking the result of the content filter"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "completion = client.completions.create(\n",
-    "    prompt=\"What's the biggest city in Washington?\",\n",
-    "    model=deployment,\n",
-    ")\n",
-    "\n",
-    "print(f\"Answer: {completion.choices[0].text}\")\n",
-    "\n",
-    "# prompt content filter result in \"model_extra\" for azure\n",
-    "prompt_filter_result = completion.model_extra[\"prompt_filter_results\"][0][\"content_filter_results\"]\n",
-    "print(\"\\nPrompt content filter results:\")\n",
-    "for category, details in prompt_filter_result.items():\n",
-    "    print(f\"{category}:\\n filtered={details['filtered']}\\n severity={details['severity']}\")\n",
-    "\n",
-    "# completion content filter result\n",
-    "print(\"\\nCompletion content filter results:\")\n",
-    "completion_filter_result = completion.choices[0].model_extra[\"content_filter_results\"]\n",
-    "for category, details in completion_filter_result.items():\n",
-    "    print(f\"{category}:\\n filtered={details['filtered']}\\n severity={details['severity']}\")"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.10.0"
-  },
-  "vscode": {
-   "interpreter": {
-    "hash": "3a5103089ab7e7c666b279eeded403fcec76de49a40685dbdfe9f9c78ad97c17"
-   }
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

examples/azure/functions.ipynb

@@ -228,7 +228,7 @@
    "source": [
     "#### 2. Pass function definition(s) into chat completions API\n",
     "\n",
-    "Now we can pass the function into the chat completions API. If the model determines it should call the function, a `finish_reason` of \"function_call\" will be populated on the choice and the details of which function to call and its arguments will be present in the `message`. Optionally, you can set the `function_call` keyword argument to force the model to call a particular function (e.g. `function_call={\"name\": get_current_weather}`). By default, this is set to `auto`, allowing the model to choose whether to call the function or not. "
+    "Now we can pass the function into the chat completions API. If the model determines it should call the function, a `finish_reason` of \"tool_calls\" will be populated on the choice and the details of which function to call and its arguments will be present in the `message`. Optionally, you can set the `tool_choice` keyword argument to force the model to call a particular function (e.g. `{\"type\": \"function\", \"function\": {\"name\": get_current_weather}}`). By default, this is set to `auto`, allowing the model to choose whether to call the function or not. "
    ]
   },
   {
@@ -245,7 +245,7 @@
     "chat_completion = client.chat.completions.create(\n",
     "    model=deployment,\n",
     "    messages=messages,\n",
-    "    functions=functions,\n",
+    "    tools=functions,\n",
     ")\n",
     "print(chat_completion)"
    ]
@@ -278,7 +278,7 @@
     "    unit = request.get(\"unit\")\n",
     "    return {\"temperature\": \"22\", \"unit\": \"celsius\", \"description\": \"Sunny\"}\n",
     "\n",
-    "function_call = chat_completion.choices[0].message.function_call\n",
+    "function_call = chat_completion.choices[0].message.tool_calls[0].function\n",
     "print(function_call.name)\n",
     "print(function_call.arguments)\n",
     "\n",
@@ -313,7 +313,7 @@
     "function_completion = client.chat.completions.create(\n",
     "    model=deployment,\n",
     "    messages=messages,\n",
-    "    functions=functions,\n",
+    "    tools=functions,\n",
     ")\n",
     "\n",
     "print(function_completion.choices[0].message.content.strip())"

examples/book_translation/translate_latex_book.ipynb

@@ -35,9 +35,12 @@
     }
    ],
    "source": [
-    "import openai\n",
+    "from openai import OpenAI\n",
+    "import os\n",
     "from transformers import GPT2Tokenizer\n",
     "\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if you didn't set as an env var>\"))\n",
+    "\n",
     "# OpenAI GPT-2 tokenizer is the same as GPT-3 tokenizer\n",
     "# we use it to count the number of tokens in the text\n",
     "tokenizer = GPT2Tokenizer.from_pretrained(\"gpt2\")\n",
@@ -212,7 +215,7 @@
     }
    ],
    "source": [
-    "def translate_chunk(chunk, engine='text-davinci-002',\n",
+    "def translate_chunk(chunk, model='gpt-3.5-turbo',\n",
     "                    dest_language='English',\n",
     "                    sample_translation=(\"\\poglavje{Osnove Geometrije} \\label{osn9Geom}\", \"\\poglavje{The basics of Geometry} \\label{osn9Geom}\")\n",
     "                    ):\n",
@@ -224,17 +227,17 @@
     "\n",
     "{sample_translation[1]}\n",
     "'''\n",
-    "    response = openai.Completion.create(\n",
-    "        prompt=prompt,\n",
-    "        engine=engine,\n",
+    "    response = client.chat.completions.create(\n",
+    "        messages=[{\"role\": \"user\", \"content\":prompt}],\n",
+    "        model=model,\n",
     "        temperature=0,\n",
     "        top_p=1,\n",
     "        max_tokens=1500,\n",
     "    )\n",
-    "    result = response['choices'][0]['text'].strip()\n",
+    "    result = response.choices[0].message.content.strip()\n",
     "    result = result.replace('\"\"\"', '') # remove the double quotes, as we used them to surround the text\n",
     "    return result\n",
-    "print(translate_chunk(chunks[800], engine='text-davinci-002', dest_language='English'))"
+    "print(translate_chunk(chunks[800], model='gpt-3.5-turbo', dest_language='English'))"
    ]
   },
   {
@@ -1134,7 +1137,7 @@
     "for i, chunk in enumerate(chunks):\n",
     "    print(str(i+1) + \" / \" + str(len(chunks)))\n",
     "    # translate each chunk\n",
-    "    translated_chunks.append(translate_chunk(chunk, engine='text-davinci-002', dest_language=dest_language))\n",
+    "    translated_chunks.append(translate_chunk(chunk, model='gpt-3.5-turbo', dest_language=dest_language))\n",
     "\n",
     "# join the chunks together\n",
     "result = '\\n\\n'.join(translated_chunks)\n",

examples/data/items_tagged_and_captioned.csv

@@ -0,0 +1,373 @@
+title,primary_image,style,material,color,url,keywords,img_description,caption
+"GOYMFK 1pc Free Standing Shoe Rack, Multi-layer Metal Shoe Cap Rack With 8 Double Hooks For Living Room, Bathroom, Hallway",https://m.media-amazon.com/images/I/416WaLx10jL._SS522_.jpg,Modern,Metal,White,https://www.amazon.com/dp/B0CJHKVG6P,"['shoe rack', 'free standing', 'multi-layer', 'metal', 'with hooks', 'white']","This is a free-standing shoe rack featuring a multi-layer design, constructed from metal for durability. The rack has a sleek white finish that gives it a clean, modern look, suitable for various home decors. It includes multiple tiers dedicated to shoe storage, allowing for an organized display of footwear. Additionally, the rack is equipped with 8 double hooks, providing ample space for hanging items such as hats, scarves, and bags. This functional piece is versatile enough to be used in a living room, bathroom, or hallway, offering both storage and hanging solutions in a compact form.","White metal free-standing shoe rack with multiple tiers for shoes and 8 double hooks for accessories, suitable for various home settings."
+"subrtex Leather ding Room, Dining Chairs Set of 2, Black",https://m.media-amazon.com/images/I/31SejUEWY7L._SS522_.jpg,Black Rubber Wood,Sponge,Black,https://www.amazon.com/dp/B0B66QHB23,"['dining chairs', 'set of 2', 'leather', 'black']","This image features a set of two black dining chairs. The chairs are upholstered in a leather-like material, giving them a sleek and sophisticated appearance. They have a high back design with subtle stitching details that create vertical lines, adding to their elegance. The legs of the chairs are straight and also finished in black, which complements the overall design. These chairs would be suitable for a modern or contemporary dining room setting.",Set of 2 sleek black faux leather dining chairs with vertical stitching details.
+"Plant Repotting Mat MUYETOL Waterproof Transplanting Mat Indoor 26.8"" x 26.8"" Portable Square Foldable Easy to Clean Gardening Work Mat Soil Changing Mat Succulent Plant Transplanting Mat Garden Gifts",https://m.media-amazon.com/images/I/41RgefVq70L._SS522_.jpg,Modern,Polyethylene,Green,https://www.amazon.com/dp/B0BXRTWLYK,"['plant repotting mat', 'waterproof', 'portable', 'foldable', 'easy to clean', 'gardening work mat', 'soil changing mat', 'succulent transplanting mat', 'green']","This is a square plant repotting mat designed for indoor gardening tasks such as transplanting or changing soil for plants. It measures 26.8 inches by 26.8 inches, providing ample space for gardening activities. The mat is made from a waterproof material in a vibrant green color, which helps to contain soil and water spills, making the cleanup process easier. The edges of the mat are raised with built-in corner loops, which can be used to secure the mat in a folded position or to hang it for storage. The mat's surface is smooth, facilitating easy cleaning. This mat is a practical accessory for gardening enthusiasts, providing a convenient and tidy workspace for handling plants, especially succulents. It's also portable, allowing gardeners to fold it up and take it with them as needed, and it can serve as a thoughtful gift for those who enjoy gardening.",Waterproof green square plant repotting mat
+"Pickleball Doormat, Welcome Doormat Absorbent Non-Slip Floor Mat Bathroom Mat 16x24",https://m.media-amazon.com/images/I/61vz1IglerL._SS522_.jpg,Modern,Rubber,A5589,https://www.amazon.com/dp/B0C1MRB2M8,"['doormat', 'absorbent', 'non-slip', 'brown']","This is a rectangular doormat featuring a playful design, ideal for pickleball enthusiasts. The mat has a natural coir fiber construction, known for its durability and excellent scraping properties to remove dirt and debris from shoes. The background color is the natural brown of coir, providing a rustic and welcoming look.
+
+The doormat is adorned with the phrase ""it's a good day to play PICKLEBALL"" in bold, black lettering, which stands out against the coir material for easy readability. Below the text, there is a graphic of two pickleball paddles crossed over a pickleball, adding a thematic touch to the design.
+
+Measuring approximately 16x24 inches, this doormat is a suitable size for most entryways. It is described as absorbent, which suggests it can help in keeping moisture from shoes from entering the home. The non-slip feature implies that the mat has a backing that prevents it from sliding around, providing safety and stability on various floor surfaces. It is also mentioned as a floor mat for the bathroom, indicating it could be versatile for indoor use as well.",Pickleball-themed coir doormat with playful design
+"JOIN IRON Foldable TV Trays for Eating Set of 4 with Stand,Folding TV/Snack Tray Table Set,Folding TV Dinner Tables for Small Space,(Grey)",https://m.media-amazon.com/images/I/41p4d4VJnNL._SS522_.jpg,X Classic Style,Iron,Grey Set of 4,https://www.amazon.com/dp/B0CG1N9QRC,"['tv tray table set', 'foldable', 'iron', 'grey']","This image showcases a set of two foldable TV trays, which are part of a set of four. The trays are designed for eating or as snack tables and are ideal for small spaces due to their foldable nature. Each tray features a grey wood grain finish on the tabletop, providing a modern and neutral aesthetic that can complement various interior decor styles. The frames and legs of the trays are made of black iron, offering sturdy support and a sleek contrast to the grey tabletops. The X-shaped leg design allows for easy folding and storage, and the overall minimalist design ensures that the trays can be conveniently tucked away when not in use. The set includes a matching stand to keep the trays organized and accessible.","Set of two foldable TV trays with grey wood grain finish and black iron legs, including a matching stand for easy storage."
+"LOVMOR 30'' Bathroom Vanity Sink Base Cabine, Storage Cabinet with 3-Drawers on The Left, Suitable for Bathrooms, Kitchens, Laundry Rooms and Other Places.",https://m.media-amazon.com/images/I/41zMuj2wvvL._SS522_.jpg,"Soft-closing Switch, Soft-closing Switch",Wood,Cameo Scotch,https://www.amazon.com/dp/B0C9WYYFLB,"['bathroom vanity', 'sink base cabinet', 'storage', '3-drawers', 'wood', 'brown']","This is a LOVMOR 30-inch bathroom vanity sink base cabinet featuring a rich, wood-toned finish. The cabinet is designed with three drawers on the left side, providing ample storage space for bathroom essentials, personal items, or cleaning supplies. The drawers appear to have recessed panel fronts and sleek handles, which contribute to the cabinet's classic and elegant look. The right side of the cabinet has a large door, likely concealing additional storage space. This versatile cabinet is not only suitable for bathrooms but can also be used in kitchens, laundry rooms, and other areas of the home where extra storage is needed. The overall design suggests a traditional style that would complement a variety of interior decors.",Traditional LOVMOR 30-inch bathroom vanity sink base cabinet with a rich wood finish and ample storage.
+"Folews Bathroom Organizer Over The Toilet Storage, 4-Tier Bathroom Shelves Over Toilet Shelf Above Toilet Storage Rack Freestanding Bathroom Space Saver Adjustable Shelves and Baskets, Black",https://m.media-amazon.com/images/I/41ixgM73DgL._SS522_.jpg,Classic,,,https://www.amazon.com/dp/B09NZY3R1T,"['over-the-toilet storage', '4-tier', 'adjustable shelves', 'freestanding', 'bathroom space saver', 'metal', 'black']","This is a 4-tier bathroom organizer designed to fit over a standard toilet, providing a space-saving storage solution. The unit features a sturdy metal frame with a black finish that offers both durability and a sleek, modern aesthetic. The organizer includes four shelves, with the two upper shelves being flat and ideal for storing items such as towels, toiletries, or decorative objects. The third shelf includes a woven basket, adding a touch of rustic charm and providing a convenient spot for smaller items that might otherwise fall through a wire shelf. The bottom shelf is also flat and can be used for additional storage.
+
+The design of this over-the-toilet storage rack is freestanding, meaning it does not require wall mounting, which makes it a versatile choice for renters or those who prefer not to drill into walls. The shelves are adjustable, allowing for customization based on the height of the items being stored. This feature also makes it easy to accommodate taller items if necessary. The overall structure is designed to maximize the use of vertical space in a bathroom, making it a practical addition for organizing and decluttering.","Modern 4-tier black metal bathroom organizer with adjustable shelves and a woven basket, designed to fit over a standard toilet for space-saving storage."
+"GOYMFK 1pc Free Standing Shoe Rack, Multi-layer Metal Shoe Cap Rack With 8 Double Hooks For Living Room, Bathroom, Hallway",https://m.media-amazon.com/images/I/416WaLx10jL._SS522_.jpg,Modern,Metal,White,https://www.amazon.com/dp/B0CJHKVG6P,"['shoe rack', 'free standing', 'multi-layer', 'metal', 'white']","This is a free-standing shoe rack featuring a multi-layer design, ideal for organizing footwear in a living room, bathroom, or hallway. The rack is constructed from metal, providing a sturdy and durable frame. It is finished in a clean white color, which gives it a modern and versatile look that can blend with various interior decors.
+
+The rack includes several horizontal tiers, each designed to hold multiple pairs of shoes, making it a space-efficient solution for shoe storage. Additionally, the top part of the rack is equipped with 8 double hooks, which can be used to hang items such as hats, scarves, or bags. This added functionality makes the rack a multifunctional piece of furniture, not only serving as shoe storage but also as a convenient spot to hang various accessories.
+
+The overall design is sleek and minimalistic, with an emphasis on functionality and space-saving. It would be a practical addition to any home, helping to keep shoes and accessories neatly organized and easily accessible.","White metal free-standing shoe rack with multiple tiers for shoes and 8 double hooks for accessories, offering a sleek and space-efficient storage solution."
+"subrtex Leather ding Room, Dining Chairs Set of 2, Black",https://m.media-amazon.com/images/I/31SejUEWY7L._SS522_.jpg,Black Rubber Wood,Sponge,Black,https://www.amazon.com/dp/B0B66QHB23,"['dining chairs', 'set of 2', 'leather', 'black']","This image features a set of two contemporary dining chairs. The chairs are upholstered in black faux leather, which gives them a sleek and modern appearance. The design includes a high backrest with subtle stitching details that create vertical lines, adding an element of texture and sophistication. The legs of the chairs are also black, maintaining a uniform and elegant look. These chairs would complement a modern dining room setting with their clean lines and minimalist style.",Set of 2 contemporary black faux leather dining chairs with vertical stitching details.
+"Plant Repotting Mat MUYETOL Waterproof Transplanting Mat Indoor 26.8"" x 26.8"" Portable Square Foldable Easy to Clean Gardening Work Mat Soil Changing Mat Succulent Plant Transplanting Mat Garden Gifts",https://m.media-amazon.com/images/I/41RgefVq70L._SS522_.jpg,Modern,Polyethylene,Green,https://www.amazon.com/dp/B0BXRTWLYK,"['plant repotting mat', 'waterproof', 'portable', 'foldable', 'green']","This is a square plant repotting mat designed for indoor gardening tasks such as transplanting or changing the soil of plants. The mat measures 26.8 inches by 26.8 inches and is made from a waterproof material, which appears to be a durable, easy-to-clean fabric in a vibrant green color. The edges of the mat are raised with integrated corner buttons to help contain soil and water, making the repotting process neater. The mat is foldable and portable, making it convenient for storage and use in various locations. It is suitable for a range of gardening activities, particularly for working with succulents and other small plants. This mat could be a practical gift for gardening enthusiasts.","Square, waterproof, green plant repotting mat"
+"Pickleball Doormat, Welcome Doormat Absorbent Non-Slip Floor Mat Bathroom Mat 16x24",https://m.media-amazon.com/images/I/61vz1IglerL._SS522_.jpg,Modern,Rubber,A5589,https://www.amazon.com/dp/B0C1MRB2M8,"['doormat', 'absorbent', 'non-slip', 'brown']","This is a rectangular doormat featuring a playful design that caters to pickleball enthusiasts. The mat's primary color is a natural coir brown, providing a neutral and earthy tone that complements various entryway decors. Emblazoned across the mat in bold, black lettering is the phrase ""it's a good day to play PICKLEBALL,"" with the word ""PICKLEBALL"" being prominently displayed in larger font for emphasis. Below the text, there is a graphic of two crossed pickleball paddles in black, which adds a thematic touch to the design.
+
+The doormat appears to be made of coir, a durable and absorbent material derived from coconut husks, which is excellent for scraping shoes clean. The size is specified as 16x24 inches, making it a suitable size for a standard doorway. The description mentions that it is non-slip, suggesting that the underside of the mat has a gripping feature to keep it in place on various floor surfaces. Additionally, it is described as absorbent, indicating that it can effectively capture moisture and prevent it from being tracked indoors. The mat could also be used in a bathroom setting due to its absorbent qualities.","Rectangular coir doormat with ""it's a good day to play PICKLEBALL"" message"
+"JOIN IRON Foldable TV Trays for Eating Set of 4 with Stand,Folding TV/Snack Tray Table Set,Folding TV Dinner Tables for Small Space,(Grey)",https://m.media-amazon.com/images/I/41p4d4VJnNL._SS522_.jpg,X Classic Style,Iron,Grey Set of 4,https://www.amazon.com/dp/B0CG1N9QRC,"['tv tray table set', 'foldable', 'metal', 'grey']","This image showcases a set of four foldable TV trays with a stand, designed for eating or as snack tray tables. The tables are finished in a grey tone, which gives them a modern and neutral look that can easily blend with various interior decors. Each tray features a rectangular top with a wood grain texture, supported by a sturdy iron frame with an A-shaped folding mechanism, allowing for easy storage and portability. The stand included in the set provides a convenient way to keep the tables organized and readily accessible when not in use. These tables are practical for small spaces, offering a temporary dining surface or a place to hold snacks during TV viewing or other activities.","Set of four foldable TV trays with stand, featuring a modern grey wood grain finish and sturdy iron frame, ideal for dining or snacks in small spaces."
+"LOVMOR 30'' Bathroom Vanity Sink Base Cabine, Storage Cabinet with 3-Drawers on The Left, Suitable for Bathrooms, Kitchens, Laundry Rooms and Other Places.",https://m.media-amazon.com/images/I/41zMuj2wvvL._SS522_.jpg,"Soft-closing Switch, Soft-closing Switch",Wood,Cameo Scotch,https://www.amazon.com/dp/B0C9WYYFLB,"['bathroom vanity', 'sink base cabinet', 'storage', '3-drawers', 'wood', 'brown']","This is a LOVMOR 30'' Bathroom Vanity Sink Base Cabinet featuring a classic design with a rich brown finish. The cabinet is constructed with three drawers on the left side, offering ample storage for bathroom essentials, personal items, or cleaning supplies. The drawers appear to have recessed panel fronts and sleek handles, which contribute to the cabinet's traditional aesthetic. The right side of the cabinet has two doors with matching recessed panels and handles, concealing additional storage space. This versatile cabinet is suitable not only for bathrooms but also for kitchens, laundry rooms, and other areas where extra storage is needed. The overall design suggests durability and functionality, making it a practical addition to any home.","LOVMOR 30'' Bathroom Vanity Sink Base Cabinet with a classic brown finish, featuring three drawers and two doors for ample storage, suitable for bathrooms, kitchens, and laundry rooms."
+"Folews Bathroom Organizer Over The Toilet Storage, 4-Tier Bathroom Shelves Over Toilet Shelf Above Toilet Storage Rack Freestanding Bathroom Space Saver Adjustable Shelves and Baskets, Black",https://m.media-amazon.com/images/I/41ixgM73DgL._SS522_.jpg,Classic,,,https://www.amazon.com/dp/B09NZY3R1T,"['over-the-toilet storage', '4-tier', 'adjustable shelves', 'baskets', 'freestanding', 'bathroom space saver', 'black', 'metal']","This is a 4-tier bathroom organizer designed to fit over a standard toilet, providing a space-saving storage solution. The unit is constructed with a sturdy metal frame in a black finish, which offers both durability and a sleek, modern look. The design includes four shelves that provide ample space for storing bathroom essentials such as towels, toiletries, and cleaning supplies. The shelves are adjustable, allowing for customization based on the height of the items being stored. Additionally, the organizer features baskets that can be used to hold smaller items, ensuring they are neatly contained and easily accessible. The freestanding design means it can be placed over the toilet without the need for wall mounting, making it a versatile and convenient addition to any bathroom.","Black metal 4-tier bathroom organizer with adjustable shelves and baskets, designed to fit over a standard toilet for space-saving storage."
+"Lerliuo Nightstand, Side Table, Industrial Bedside Table with 2 Drawers and Open Shelf, Grey Night Stand, End Table with Steel Frame for Bedroom, Dorm, Gray/Black 23.6''H",https://m.media-amazon.com/images/I/41IzLmM91FL._SS522_.jpg,Classic,Stone,Grey,https://www.amazon.com/dp/B09PTXGFZD,"['nightstand', 'side table', 'bedside table', 'industrial', '2 drawers', 'open shelf', 'steel frame', 'grey', 'black', '23.6 inches height']","This is a Lerliuo nightstand featuring an industrial design. The bedside table has a sturdy steel frame in black, providing a strong contrast to the grey wood-look panels that make up the drawers and tabletop. It includes two drawers with black round knobs for easy opening, offering ample storage space for personal items. Above the drawers, there is an open shelf that provides additional space for books or decorative items. The overall dimensions are approximately 23.6 inches in height, making it suitable for most bed heights. The combination of grey and black gives this nightstand a modern and versatile look that can fit into various bedroom decors, from contemporary to rustic.","Industrial-style Lerliuo nightstand with a black steel frame and grey wood-look panels, featuring two drawers and an open shelf."
+Boss Office Products Any Task Mid-Back Task Chair with Loop Arms in Grey,https://m.media-amazon.com/images/I/41rMElFrXBL._SS522_.jpg,Loop Arms,Foam,Grey,https://www.amazon.com/dp/B002FL3LL2,"['office chair', 'mid-back', 'task chair', 'loop arms', 'grey', 'fabric', 'swivel', 'adjustable height']","This is a mid-back task chair designed for office use, featuring loop arms for added comfort. The chair is upholstered in a grey fabric that offers a neutral look, suitable for various office decors. Its backrest and seat are contoured to provide support and comfort during extended periods of sitting. The chair includes a pneumatic height adjustment lever, allowing the user to modify the seat height to their preference. The five-star base with caster wheels ensures easy mobility across different floor surfaces. The loop arms are made of durable black plastic, which complements the chair's overall design and provides a place to rest your arms while working.",Mid-back grey fabric office task chair with loop arms and pneumatic height adjustment.
+"Kingston Brass BA1752BB Heritage 18-Inch Towel-Bar, Brushed Brass",https://m.media-amazon.com/images/I/21opezr3bUL._SS522_.jpg,Traditional,Brass,Brushed Brass,https://www.amazon.com/dp/B0B9PLM9P8,"['towel bar', 'brass', 'brushed brass', '18-inch', 'wall mounted']","This is the Kingston Brass BA1752BB Heritage 18-Inch Towel Bar in a brushed brass finish. The towel bar features a classic design with two ornate mounting brackets that provide a touch of elegance. The bar itself is a straight, cylindrical rod, perfect for hanging towels. The brushed brass finish gives it a warm, golden tone that exudes a vintage charm, making it suitable for traditional bathroom decors. The solid brass construction ensures durability and long-lasting use.",Classic 18-inch Kingston Brass towel bar
+Chief Mfg.Swing-Arm Wall Mount Hardware Mount Black (TS218SU),https://m.media-amazon.com/images/I/41HxUoRXloL._SS522_.jpg,,,Black,https://www.amazon.com/dp/B007E40Z5K,"['wall mount', 'swing-arm', 'hardware mount', 'black', 'metal']","This image shows the Chief Mfg. Swing-Arm Wall Mount, model TS218SU, which is a hardware mount designed for televisions or monitors. The mount is black in color and features a robust, articulated swing-arm design that allows for flexible positioning of the screen. The arm can extend, tilt, and swivel, providing a range of motion for optimal viewing angles. The wall plate and the mounting bracket are designed with multiple holes for secure attachment to the wall and the device, respectively. The construction appears to be made of durable metal, suitable for supporting the weight and size of compatible screens as specified by the manufacturer.",Black articulated swing-arm wall mount for TVs or monitors
+"DOMYDEVM Black End Table, Nightstand with Charging Station, Bedside Table with Drawer, Small Side Table with USB Ports and Outlets for Bedroom",https://m.media-amazon.com/images/I/41YrmN-yOEL._SS522_.jpg,Modern,Metal,Black,https://www.amazon.com/dp/B0BFJGDHVF,"['end table', 'nightstand', 'charging station', 'bedside table', 'drawer', 'side table', 'usb ports', 'outlets', 'black', 'wood', 'metal mesh']","This is a black end table or nightstand featuring a built-in charging station. It is designed with a compact and functional structure, ideal for a bedroom setting. The table includes a top surface suitable for holding small items like a plant or a phone, as depicted in the image. Below the top, there is a drawer with a metal mesh front, providing storage while adding an industrial touch to the design. The drawer likely conceals the charging station, which includes USB ports and outlets for convenient device charging. Beneath the drawer, there is an open shelf, perfect for storing books or displaying decorative items. The table is supported by four sturdy legs, and the overall aesthetic is modern and minimalist, making it versatile for various decor styles. The inclusion of a charging station makes it a practical piece of furniture for the modern, tech-savvy user.","Modern black end table with built-in charging station, featuring a drawer and an open shelf, ideal for bedroom use."
+"LASCO 35-5019 Hallmack Style 24-Inch Towel Bar Accessory, All Metal Construction, Chrome Plated Finish",https://m.media-amazon.com/images/I/31rrpY5EJOL._SS522_.jpg,,Metal,Chrome,https://www.amazon.com/dp/B00N2OZU42,"['towel bar', 'metal', 'chrome', '24-inch']","This is a 24-inch towel bar accessory designed in the Hallmack style. It features an all-metal construction with a chrome-plated finish, giving it a sleek and shiny appearance. The bar is supported by two end brackets that are also chrome-plated, and it is designed to be mounted on a wall to hold towels. Its polished surface reflects light and is likely to complement a modern or contemporary bathroom decor.","Sleek 24-inch chrome-plated Hallmack style towel bar with polished finish, designed for modern bathroom decors."
+"Table-Mate II PRO TV Tray Table - Folding Table with Cup Holder and Tablet Slot- Couch Desk for Working from Home, TV Trays for Eating, Portable and Adjustable TV Table, Slate Gray",https://m.media-amazon.com/images/I/31cbTc2VhaL._SS522_.jpg,Cupholder+Tablet Slot+Side Storage,,Slate Gray,https://www.amazon.com/dp/B093KMM9D3,"['tv tray table', 'folding table', 'cup holder', 'tablet slot', 'couch desk', 'portable', 'adjustable', 'slate gray', 'metal']","This is the Table-Mate II PRO TV Tray Table, a versatile and functional piece designed for convenience and adaptability. The table features a slate gray color, giving it a modern and neutral look that can easily blend with various interior decors. It is constructed with a folding design, allowing for easy storage and portability.
+
+The tabletop includes a textured surface to prevent items from slipping and has a built-in cup holder on one side, ensuring that drinks can be securely placed without the risk of spills. Additionally, there is a tablet slot, which provides a convenient space for electronic devices, making it ideal for working from home or enjoying media.
+
+The table's height and angle are adjustable, offering flexibility to accommodate different seating arrangements and personal preferences. The L-shaped legs are designed to slide under couches or chairs, making it perfect for use while sitting on a sofa or bed.
+
+Constructed from durable materials, this TV tray table is suitable for various activities, such as eating meals, working on a laptop, or holding snacks during movie nights. Its compact and lightweight design makes it an excellent choice for small living spaces or for those who need an easily movable work surface.","Table-Mate II PRO TV Tray Table with built-in cup holder and tablet slot, adjustable height and angle, in slate gray, ideal for versatile use in small living spaces."
+EGFheal White Dress Up Storage,https://m.media-amazon.com/images/I/31l+OVMKBFL._SS522_.jpg,,,,https://www.amazon.com/dp/B0C7QD8KYX,"['dress up storage', 'white', 'wood']","This is a white dress-up storage unit designed for children's playrooms or bedrooms. The unit features a sturdy construction with a clean white finish that easily blends with various decor styles. It includes a hanging area with a rod for costumes or clothing, complete with several white hangers. Below the hanging space, there are two open shelves for additional storage, perfect for shoes, hats, or toys. To the right of the hanging area, there are two grey fabric storage bins that slide into cubbies, offering concealed storage for accessories or small items. The top of the unit has a row of decorative crown-like cutouts, adding a playful touch to the design. The base of the unit is slightly recessed, creating a convenient lower storage area that can be used for larger items or as a seating bench. This dress-up storage unit is a functional and charming addition to a child's space, encouraging organization and imaginative play.","White children's dress-up storage unit with hanging area, shelves, and grey fabric bins, perfect for organizing costumes and accessories in playrooms or bedrooms."
+"Caroline's Treasures PPD3013JMAT Enchanted Garden Flowers Doormat 24x36 Front Door Mat Indoor Outdoor Rugs for Entryway, Non Slip Washable Low Pile, 24H X 36W",https://m.media-amazon.com/images/I/51Zn-AivGrL._SS522_.jpg,Enchanted Garden Flowers,Rubber,Enchanted Garden Flowers,https://www.amazon.com/dp/B08Q5KDSQK,"['doormat', 'floral', 'non-slip', 'washable', 'low pile', 'indoor', 'outdoor', 'colorful']","This is a vibrant and colorful doormat featuring a design titled ""Enchanted Garden Flowers."" The dimensions of the doormat are 24 inches in height by 36 inches in width. It is suitable for both indoor and outdoor use, making it versatile for placement at a front door or entryway. The doormat is designed to be non-slip and washable, with a low pile that ensures easy door movement and maintenance. The design showcases a variety of stylized flowers in rich hues of purple, red, yellow, and green, set against a dark background that makes the colors pop. This doormat would add a cheerful and welcoming touch to any home entrance.","Colorful ""Enchanted Garden Flowers"" doormat"
+"Leick Home 70007-WTGD Mixed Metal and Wood Stepped Tier Bookshelf, White Herringbone/Satin Gold",https://m.media-amazon.com/images/I/31XhtLE1F1L._SS522_.jpg,Bookshelf,,,https://www.amazon.com/dp/B098KNRNLQ,"['bookshelf', 'metal', 'wood', 'white', 'gold', 'stepped tier', 'herringbone pattern']","This is a Leick Home Mixed Metal and Wood Stepped Tier Bookshelf featuring a white herringbone pattern on the shelves, complemented by a satin gold finish on the metal frame. The bookshelf has a modern, open design with a stepped configuration, providing three tiers of shelving for books, decorative items, or other belongings. The white shelves contrast elegantly with the warm metallic tone of the frame, creating a piece that is both functional and stylish, suitable for a variety of interior decor styles.","Modern Leick Home bookshelf with white herringbone shelves and a satin gold metal frame, featuring a stepped tier design for stylish storage."
+"Caroline's Treasures CK3435MAT Bichon Frise Doormat 18x27 Front Door Mat Indoor Outdoor Rugs for Entryway, Non Slip Washable Low Pile, 18H X 27W",https://m.media-amazon.com/images/I/51V1IWRrXHL._SS522_.jpg,Bichon Frise,Rubber,Bichon Frise,https://www.amazon.com/dp/B07W8SF5CF,"['doormat', 'non-slip', 'washable', 'low pile', 'floral', 'animal print', 'indoor', 'outdoor']","This is a decorative doormat featuring a Bichon Frise dog as the central design. The doormat measures 18 inches in height and 27 inches in width. It is designed for both indoor and outdoor use, making it versatile for placement at the front door or any entryway. The mat has a non-slip feature to ensure safety and stability, and it is washable, which allows for easy maintenance. The pile of the mat is low, which helps in preventing tripping and allows for doors to pass over it easily. The background of the mat is adorned with a floral pattern in a variety of colors, set against a light blue backdrop, which adds a charming and vibrant touch to the item. The black border frames the design neatly, enhancing its visual appeal.","Decorative doormat featuring a Bichon Frise with a colorful floral background, designed for indoor and outdoor use with a non-slip, washable feature."
+"Wildkin Kids Canvas Sling Bookshelf with Storage for Boys and Girls, Wooden Design Features Four Shelves and Two Drawers, Helps Keep Bedrooms, Playrooms, and Classrooms Organized (Natural with Blue)",https://m.media-amazon.com/images/I/51-GsdoM+IS._SS522_.jpg,Natural with Blue Canvas,"Engineered Wood, Polyester",,https://www.amazon.com/dp/B07GBVFZ1Y,"['bookshelf', 'kids', 'canvas', 'wood', 'storage', 'four shelves', 'two drawers', 'natural', 'blue']","This is a Wildkin Kids Canvas Sling Bookshelf with Storage, designed for children's bedrooms, playrooms, or classrooms. The bookshelf features a natural wood finish, giving it a warm and inviting look that blends well with various decor styles. It includes four canvas sling shelves in a deep blue color, which are ideal for displaying books in a way that makes the covers easily visible and accessible to children.
+
+Below the sling shelves, there are two additional wooden shelves that provide extra space for storing toys, games, or additional books. The unit also comes with two matching blue canvas drawers that fit neatly into the lower section, offering concealed storage for smaller items and helping to keep the area tidy.
+
+The overall design is kid-friendly, with a sturdy wooden frame that ensures durability and stability. The combination of open shelving and drawers makes it versatile for organizing a variety of items, encouraging children to keep their spaces organized.","Wildkin Kids Canvas Sling Bookshelf with Storage in natural wood and deep blue, featuring four sling shelves and two lower wooden shelves with matching canvas drawers for versatile organization in children's spaces."
+Gbuzozie 38L Round Laundry Hamper Cute Mermaid Girl Storage Basket Waterproof Coating Organizer Bin For Nursery Clothes Toys,https://m.media-amazon.com/images/I/41hYmELREGL._SS522_.jpg,Fashion,fabric,Mermaid Girl,https://www.amazon.com/dp/B09QZ3LYWB,"['laundry hamper', 'round', 'waterproof', 'storage basket', 'nursery', 'toys organizer', 'cute', 'mermaid', 'pink', 'blue', 'fabric']","This is a 38-liter round laundry hamper featuring a cute mermaid girl design, ideal for adding a playful touch to a nursery or child's room. The hamper is made with a waterproof coating, making it suitable for storing not just clothes but also toys and other items. The design showcases a whimsical mermaid with vibrant pink hair and a green tail, set against a soft blue background adorned with sea creatures and coral motifs. The hamper appears to be equipped with two durable handles for easy transport. Its collapsible design allows for convenient storage when not in use. This organizer bin combines functionality with a charming aesthetic, making it a delightful addition to any space that requires tidying up.","A 38-liter round laundry hamper with a waterproof coating, featuring a whimsical mermaid design, perfect for nurseries or children's rooms."
+"Tiita Comfy Saucer Chair, Soft Faux Fur Oversized Folding Accent Chair, Lounge Lazy Chair for Kids Teens Adults, Metal Frame Moon Chair for Bedroom, Living Room, Dorm Rooms",https://m.media-amazon.com/images/I/41O7mY3lUvL._SS522_.jpg,Modern,pp cotton,Grey,https://www.amazon.com/dp/B09T5T67VC,"['saucer chair', 'faux fur', 'oversized', 'folding', 'metal frame', 'moon chair', 'grey']","This is a Tiita Comfy Saucer Chair featuring a plush, oversized seat cushion upholstered in soft faux fur. The chair is designed to provide a cozy and comfortable seating experience, suitable for kids, teens, and adults. It has a distinctive moon chair silhouette with a generously padded, tufted cushion that creates an inviting look and feel.
+
+The chair's frame is made of durable metal, finished in black, which folds easily for convenient storage and transport. The color of the faux fur appears to be a neutral grey, making it versatile for various room decors. This chair is ideal for adding a touch of comfort and style to bedrooms, living rooms, dorm rooms, or any space where a casual and relaxing seat is desired.","Plush grey faux fur saucer chair with a foldable metal frame, offering cozy seating for bedrooms or dorms."
+"Summer Desk Decor,Welcome Summer Wood Block Sign Desk Decor,Rustic Summer Fruits and Popsicles Wooden Box Plaque Sign Desk Decor for Home Office Shelf Table Decorations",https://m.media-amazon.com/images/I/51sai4l5ttL._SS522_.jpg,,,,https://www.amazon.com/dp/B0C99NS6CZ,"['desk decor', 'wood', 'block sign', 'rustic', 'summer', 'fruits', 'popsicles', 'wooden box', 'plaque', 'table decorations']","This is a ""Welcome Summer"" themed wooden block sign designed as a decorative desk accessory. The sign features a vibrant and colorful watercolor-style illustration with motifs of summer fruits like watermelon slices, lemons, and limes, as well as popsicles, which evoke a refreshing and joyful summer vibe. The phrase ""Welcome Summer"" is prominently displayed in the center in a playful, multicolored font that complements the illustrations surrounding it.
+
+The wooden box plaque has a rustic yet cheerful look, making it suitable for adding a touch of seasonal charm to a home office, shelf, or table. The edges are clean and the sign appears to be self-standing, which allows for easy placement without the need for additional support. The overall design suggests a casual and fun atmosphere, perfect for summer-themed decor.","Colorful ""Welcome Summer"" wooden block sign with summer fruits and popsicles design, ideal for seasonal desk or shelf decor."
+"Homebeez 39.1"" Length Bedroom Storage Bench, End Bed Ottoman Bench with Golden Metal Legs, Dressing Chair for Entryway Living Room,Green",https://m.media-amazon.com/images/I/31eBuhJ0NDL._SS522_.jpg,Trumpet Leg,,Green,https://www.amazon.com/dp/B0BWQ8M4Q3,"['storage bench', 'ottoman', 'green', 'metal legs', 'golden legs', 'velvet', 'bedroom furniture', 'entryway furniture', 'living room furniture']","This is a Homebeez bedroom storage bench featuring a luxurious green velvet upholstery that adds a vibrant touch to any room. The bench measures 39.1 inches in length, providing ample space for seating or storage. It is designed with a hinged top that likely opens to reveal a storage compartment, ideal for keeping linens, accessories, or other household items neatly tucked away.
+
+The bench is supported by sleek golden metal legs that give it a modern and elegant look, while also ensuring stability and durability. The combination of the rich green color and the golden accents creates a sophisticated aesthetic that can complement both contemporary and classic interior designs.
+
+This versatile piece of furniture can serve as an end-of-bed ottoman, providing a comfortable place to sit while dressing or putting on shoes. It can also be placed in an entryway, living room, or any other space where additional seating and storage are desired. The bench's chic design and functional features make it a stylish and practical addition to any home.","Luxurious green velvet storage bench with golden metal legs, featuring a hinged top for ample storage space, perfect for adding a vibrant and sophisticated touch to any room."
+"Flash Furniture Webb Commercial Grade 24"" Round Blue Metal Indoor-Outdoor Table",https://m.media-amazon.com/images/I/41IWb+KWq7L._SS522_.jpg,Contemporary,Metal,Blue,https://www.amazon.com/dp/B01M3VT58B,"['table', 'metal', 'indoor-outdoor', 'blue', 'round']","This is a commercial-grade 24"" round table designed for both indoor and outdoor use. The table features a vibrant blue finish that adds a pop of color to any setting. The tabletop is smooth and circular, providing ample space for dining or display. Its metal construction ensures durability and stability, while the sleek, tapered legs give it a modern and stylish look. The table also includes protective rubber feet to prevent damage to flooring and to enhance stability. This piece is ideal for cafes, bistros, or home patios looking for a contemporary and functional table.","Commercial-grade 24"" round table with a vibrant blue finish, suitable for indoor and outdoor use, featuring a durable metal construction, sleek tapered legs, and protective rubber feet."
+"Mellow 2 Inch Ventilated Memory Foam Mattress Topper, Soothing Lavender Infusion, CertiPUR-US Certified, Queen",https://m.media-amazon.com/images/I/418jBmKzSxL._SS522_.jpg,,,,https://www.amazon.com/dp/B077ZDFXYK,"['mattress topper', 'memory foam', 'ventilated', 'lavender infusion', 'certipur-us certified', 'queen size', 'purple']","This is a queen-sized Mellow 2 Inch Ventilated Memory Foam Mattress Topper featuring a soothing lavender infusion. The topper is designed to provide additional comfort and support to your existing mattress. It is made from memory foam, which is known for its pressure-relieving properties and ability to conform to the body's shape. The lavender infusion is intended to offer a calming and relaxing scent, potentially enhancing sleep quality.
+
+The topper is ventilated, which means it has small holes or channels throughout to promote airflow and help regulate sleeping temperature. This can also contribute to the overall comfort by preventing overheating during the night.
+
+The product is CertiPUR-US certified, ensuring that it is made without harmful chemicals and meets strict standards for content, emissions, and durability. The certification also indicates that the foam has been tested for low VOC (volatile organic compound) emissions for indoor air quality.
+
+The topper appears to have a purple hue, which is likely indicative of the lavender infusion, and it sits atop a mattress that is part of a neatly arranged bed with pillows against a white brick wall backdrop. The room has a modern and minimalistic aesthetic, with natural light coming in from a window on the left.","Queen-sized lavender-infused memory foam mattress topper designed for enhanced comfort and relaxation, featuring ventilation for improved airflow and temperature regulation, and CertiPUR-US certified for safety and quality."
+"CangLong Mid Century Modern Side Chair with Wood Legs for Kitchen, Living Dining Room, Set of 1, Black",https://m.media-amazon.com/images/I/31wnzq-TwKL._SS522_.jpg,Modern,,Black,https://www.amazon.com/dp/B08RTLBD1T,"['chair', 'mid-century modern', 'wood', 'black']","This is a CangLong Mid Century Modern Side Chair featuring a sleek black seat with an ergonomic design for comfort. The chair is supported by four sturdy wooden legs that display the natural wood grain, adding a warm contrast to the black seat. The legs are splayed slightly outward, which is characteristic of mid-century modern design, providing both style and stability. This chair is suitable for a variety of settings, including kitchens, dining rooms, or living spaces, and can complement a range of interior decor styles. It is sold as a set of one, allowing for individual purchase or the option to buy multiple to create a cohesive seating arrangement.","Mid-century modern side chair with a sleek black seat and natural wood legs, perfect for dining or living spaces."
+"HomePop Metal Accent Table Triangle Base Round Mirror Top, Black, 18.24D x 24W x 18H in",https://m.media-amazon.com/images/I/41cG70UIWTL._SS522_.jpg,Modern,Glass,Black,https://www.amazon.com/dp/B08N5H868H,"['accent table', 'metal', 'mirror top', 'black', 'triangle base', 'round']","This is a modern accent table featuring a sleek, triangular metal base with a black finish that provides stability and an industrial aesthetic. The round tabletop is a mirror, adding a reflective and elegant touch to the piece. The dimensions of the table are approximately 18.24 inches in diameter and 24 inches in width, with a height of 18 inches, making it a versatile size for various uses such as a side table or decorative display stand. The combination of the mirrored surface and the geometric metal base makes this table a stylish addition to contemporary home decor.","Modern accent table with a mirrored round top and a sleek, triangular black metal base, perfect for contemporary decor."
+MAEPA RV Shoe Storage for Bedside - 8 Extra Large Pockets - Dark Grey Rv Shoe Organizer with RV Shoe Pockets - Hanging Bedside Storage for RV Camper Bed Length35.4inches/2.95ft(Dark Grey),https://m.media-amazon.com/images/I/31bcwiowcBL._SS522_.jpg,,Oxford,Dark Gray,https://www.amazon.com/dp/B0C4PL1R3F,"['rv shoe storage', 'bedside organizer', 'hanging storage', 'dark grey', 'fabric', 'extra large pockets']","This is a dark grey RV shoe storage organizer designed to provide convenient hanging storage, ideal for use in an RV or camper. It features 8 extra-large pockets that offer ample space for organizing and storing shoes. The organizer is made to hang at the bedside, making it easy to access footwear. The total length of the unit is 35.4 inches, or approximately 2.95 feet, which allows it to accommodate various shoe sizes. The material appears to be a durable fabric, suitable for the interior of an RV, and the color is a versatile dark grey that should blend well with most interiors. The image also shows four metal hooks, which are likely used to hang the organizer securely in place.","Dark grey RV shoe storage organizer with 8 extra-large pockets for convenient bedside hanging, designed to fit various shoe sizes and blend seamlessly with most interiors."
+"NearMoon Hand Towel Holder/Towel Ring - Bathroom Towel Bar, Premium Thicken Space Aluminum Towel Rack Wall Towel Hanger, Contemporary Style Bath Hardware Accessories Wall Mounted (Gold)",https://m.media-amazon.com/images/I/51eswTC1ufL._SS522_.jpg,,,,https://www.amazon.com/dp/B09PND1VQ5,"['towel holder', 'towel ring', 'bathroom', 'space aluminum', 'wall mounted', 'gold', 'contemporary']","This is a NearMoon Hand Towel Holder/Towel Ring, designed to be a functional and stylish accessory for a bathroom. The towel holder is made from premium thickened space aluminum, which provides durability and resistance to corrosion. The color of the towel rack is gold, giving it a contemporary and luxurious appearance.
+
+The design features a straight bar with a rectangular profile, which allows for easy hanging and access to a hand towel. The wall-mounted fixture has a clean and modern aesthetic, with concealed mounting hardware that contributes to its sleek appearance. The holder is part of a bath hardware collection, suggesting that it could be coordinated with other accessories in the same style and finish for a cohesive bathroom design.","Gold NearMoon hand towel holder with a sleek, modern design, made from durable space aluminum, perfect for adding a touch of luxury to any bathroom."
+"FLYJOE Narrow Side Table with PU Leather Magazine Holder Rustic Slim Little Thin Table for Living Room, Bedroom, Sofa, Set of 2, Black",https://m.media-amazon.com/images/I/41Hsse9SYsL._SS522_.jpg,Country Rustic,Engineered Wood,Black-set of 2,https://www.amazon.com/dp/B0CHYDTQKN,"['side table', 'narrow', 'magazine holder', 'pu leather', 'slim', 'set of 2', 'black']","This set includes two narrow side tables, each featuring a sleek black finish that complements a modern or contemporary decor style. The tables are designed with a slim profile, making them suitable for small spaces such as beside a sofa or bed. The tabletop provides a flat surface for placing drinks, books, or decorative items.
+
+Below the tabletop, each side table incorporates a PU leather magazine holder, offering a convenient storage solution for magazines, newspapers, or small books. The magazine holder's black leather material adds a touch of sophistication and contrasts nicely with the metal frame of the table.
+
+The tables are supported by a sturdy metal frame, ensuring stability and durability. The base is designed to slide under a couch or bed slightly, maximizing space efficiency. This set is ideal for those looking to add functional furniture with a minimal footprint to their living room or bedroom.","Set of 2 narrow side tables with black finish and built-in PU leather magazine holders, featuring a slim design ideal for small spaces and a sturdy metal frame for stability."
+"HomePop Home Decor | K2380-YDQY-2 | Luxury Large Faux Leather Square Storage Ottoman | Ottoman with Storage for Living Room & Bedroom, Dark Brown",https://m.media-amazon.com/images/I/416lZwKs-SL._SS522_.jpg,Modern,Engineered Wood,Dark Gray,https://www.amazon.com/dp/B0B94T1TZ1,"['ottoman', 'faux leather', 'storage', 'square', 'dark brown']","This is a luxury large square storage ottoman featuring a faux leather upholstery in a dark brown color. The design includes a simple and elegant tufted top with visible stitching that adds texture and interest. The ottoman stands on four sturdy wooden legs, which provide stability and raise the piece slightly off the ground. The color and material give it a rich, classic look that would complement a variety of living room or bedroom decors. Additionally, this ottoman serves a dual purpose by offering ample storage space inside, making it a functional piece for organizing and decluttering your space.","Luxurious dark brown faux leather storage ottoman with a tufted top and wooden legs, offering ample interior space for organization."
+Moroccan Leather Pouf Ottoman for Living Room - Round Leather Ottoman Pouf Cover - Unstuffed Pouf Ottoman Leather Foot Stool - Moroccan Pouf Ottoman - Moroccan Ottoman Pouf Leather (Brown),https://m.media-amazon.com/images/I/51UKACPPL9L._SS522_.jpg,"Modern, Boho, Chic",Leather,Brown,https://www.amazon.com/dp/B0CP45784G,"['ottoman', 'moroccan', 'leather', 'unstuffed', 'foot stool', 'brown']","This is a round Moroccan leather pouf ottoman cover in a rich brown color. It features an intricate, embossed mandala-like pattern in the center, surrounded by stitched segments that create a pleasing geometric design. The stitching is done in a contrasting light color, which accentuates the pouf's shape and design details. This unstuffed pouf cover can be filled to serve as a comfortable footstool or additional seating in a living room or any other space. The leather material adds a touch of luxury and durability, making it both a functional and decorative item. Please note that as an unstuffed cover, it will need to be filled before use.","Round Moroccan leather pouf ottoman cover in brown with an embossed mandala pattern and contrasting stitching, designed for use as a footstool or extra seating once filled."
+"AnyDesign Christmas Welcome Doormat Decorative Xmas Holiday Front Floor Mat Indoor Outdoor Carpet Non-Slip Door Mat for Indoor Outdoor Home Office Kitchen Yard Garden Decoration, 17 x 29 Inch",https://m.media-amazon.com/images/I/51RpGVRAFcL._SS522_.jpg,,,,https://www.amazon.com/dp/B0BC85H7Y7,"['doormat', 'christmas', 'holiday', 'decorative', 'non-slip', 'indoor', 'outdoor', 'fabric']","This is a rectangular Christmas-themed welcome doormat measuring 17 x 29 inches. The mat features a natural beige background, likely made of a durable, coarse fabric such as coir or a synthetic equivalent to withstand foot traffic. The word ""Welcome"" is prominently displayed in large, black, cursive lettering across the center. Above the text, there is a festive design of mistletoe with green leaves and a red and white Santa hat, along with a red ornament decorated with white snowflake patterns. The border of the mat has a striped pattern alternating between red and a lighter beige, resembling a candy cane motif, which adds to the holiday spirit of the design. The mat is described as non-slip, suggesting it has a grippy material on the underside to keep it in place on various surfaces, making it suitable for both indoor and outdoor use. It is intended to serve as a decorative and functional piece for home, office, kitchen, yard, or garden settings during the Christmas season.","Festive Christmas-themed welcome doormat with mistletoe, Santa hat, and ornament design, featuring a natural beige background and candy cane striped border."
+GXFC ZHAO Welcome Funny Door Mat Shoes and Bras Off Please Personalized Doormat with Anti-Slip Rubber Back (23.6 X 15.7 inch) Prank Gift Home Decor Area Rugs for The Entrance Way Indoor Novelty Mats,https://m.media-amazon.com/images/I/51z8ko3rsiL._SS522_.jpg,Modern,Rubber,"Colorful,Funny,Novelty,Personalized",https://www.amazon.com/dp/B07X61R7N8,"['door mat', 'anti-slip', 'rubber back', 'novelty', 'brown']","This is a novelty doormat designed to be placed at the entrance of a home. The doormat measures 23.6 by 15.7 inches and features a humorous message that reads ""SHOES & BRAS OFF PLEASE"" in bold, black lettering against a brown background, which gives it a playful and inviting appearance. The design includes decorative corner accents that resemble arrow points, adding a touch of whimsy to the overall look.
+
+The mat is made with a durable material suitable for indoor use and is equipped with an anti-slip rubber backing to ensure safety and prevent it from sliding around. This doormat serves as both a functional item for keeping floors clean and a prank gift or home decor piece that can add a lighthearted touch to the entrance of someone's home.","Humorous indoor doormat with ""SHOES & BRAS OFF PLEASE"" message, featuring anti-slip backing and decorative corner accents."
+"LEASYLIFE Black Metal Trash can,10L/2.6GAL,Open Top Wastebasket Bin,Garbage Can for Bathroom,Living Room,Office,Kitchen,Bedroom,Hotel (Black)",https://m.media-amazon.com/images/I/31NobyGCHQL._SS522_.jpg,,,,https://www.amazon.com/dp/B09TKH23M1,"['trash can', 'metal', 'open top', 'black', '10l', '2.6gal', 'wastebasket', 'garbage can']","This is a black metal trash can with a capacity of 10 liters (2.6 gallons). It features an open-top design for easy waste disposal. The cylindrical shape and sleek black finish give it a modern and minimalist appearance, making it suitable for various settings such as bathrooms, living rooms, offices, kitchens, bedrooms, and hotels. The trash can's smooth surface and simple lines would complement contemporary decor styles.","Modern 10-liter black metal trash can with an open-top design, suitable for various indoor settings with its sleek and minimalist appearance."
+"Solid Wood Wine Cabinet, bar Rack - Home Wood Furniture",https://m.media-amazon.com/images/I/31vp24lBBwL._SS522_.jpg,wood,Wood,White/Pine Oak,https://www.amazon.com/dp/B0BW4WN2NL,"['wine cabinet', 'solid wood', 'scandinavian', 'white', 'natural wood']","This is a solid wood wine cabinet featuring a contemporary design. The main structure appears to be finished in a light, possibly white or off-white hue, which contrasts nicely with the natural wood tone of the wine racks and drawer front. The cabinet is divided into two sections: on the left, there are several rows of horizontal wine racks capable of holding multiple bottles, while the right side offers a combination of open shelving and a drawer for additional storage. The cabinet is supported by four angled legs, which give it a modern and slightly Scandinavian aesthetic. The use of solid wood suggests durability and quality, making it a functional yet stylish piece for storing wine and related accessories.","Contemporary solid wood wine cabinet with contrasting white and natural wood finishes, featuring horizontal wine racks, open shelving, a storage drawer, and angled legs for a modern Scandinavian look."
+"Black Leather Office Chair Mid Back Leather Desk Chair Modern Excutive Office Chair with Arms and Wheels for Home Office, by Artswish",https://m.media-amazon.com/images/I/317sVlhzMLL._SS522_.jpg,With arms,Sponge,Black,https://www.amazon.com/dp/B0BVQSPCCF,"['office chair', 'mid back', 'leather', 'black', 'with arms', 'with wheels', 'modern']","This is a mid-back leather office chair in black, designed for comfort and style in a home office setting. The chair features a generously padded seat and backrest, upholstered in a sleek black leather material that gives it a professional and modern appearance. The armrests are integrated into the design, providing support without compromising on style. It is mounted on a sturdy five-point base with dual-wheel casters for easy mobility, and the height can be adjusted to suit various desk heights and user preferences. The chair's silhouette is streamlined, making it a suitable match for a range of interior decors, particularly in contemporary and executive office spaces.","Mid-back black leather office chair with padded seat and backrest, integrated armrests, adjustable height, and a five-point base with casters for mobility, suitable for contemporary home offices."
+"Convenience Concepts Tucson Flip Top End Table with Charging Station and Shelf, 23.75""L x 11.25""W x 24""H, Charcoal Gray/Black",https://m.media-amazon.com/images/I/41NVHRlEb2L._SS522_.jpg,Modern,"Manufactured Wood,MDF",Charcoal Gray / Black,https://www.amazon.com/dp/B08B45QFG7,"['end table', 'charging station', 'flip top', 'shelf', 'charcoal gray', 'black', 'wood', 'metal']","This is a Convenience Concepts Tucson Flip Top End Table featuring a built-in charging station. The table measures 23.75 inches in length, 11.25 inches in width, and 24 inches in height. It has a sleek and modern design with a charcoal gray finish on the tabletop and shelf, complemented by a black frame. The frame showcases an X-shaped design on the sides, adding both stability and a decorative touch. The flip top reveals a convenient charging station with ample space for powering electronic devices, making it a practical addition to any living space. The lower shelf provides additional storage or display space. This end table combines functionality with contemporary style, suitable for a variety of home decors.","Modern charcoal gray end table with built-in charging station, featuring a flip top and additional lower shelf for storage, set in a black frame with an X-shaped design."
+"3-Tier Kitchen Storage Cart with Handle, Multifunction Utility Rolling Cart Kitchen Storage Organizer, Mobile Shelving Unit Cart with Lockable Wheels for Office, Living Room, Kitchen - Black",https://m.media-amazon.com/images/I/41yBXlrog-L._SS522_.jpg,,Plastic,Black,https://www.amazon.com/dp/B0CMX4CMMS,"['kitchen cart', '3-tier', 'rolling', 'storage organizer', 'mobile shelving', 'lockable wheels', 'black']","This is a 3-tier kitchen storage cart featuring a sleek black finish. It is designed as a multifunctional utility rolling cart that can be used for organizing various items. The cart is equipped with a convenient handle at the top, making it easy to maneuver around your space. Each tier provides ample storage space, suitable for holding kitchen supplies, office materials, or items for the living room.
+
+The cart is constructed with a sturdy frame and includes lockable wheels at the base, ensuring both mobility and stability when needed. The lockable feature of the wheels allows the cart to stay in place without unwanted rolling. The open shelving design allows for easy access and visibility of the items stored on each level.
+
+Overall, this kitchen storage organizer is a practical addition to any home or office, offering a space-saving solution and the versatility to be used in various settings.","3-tier black kitchen storage cart with lockable wheels and a handle for easy maneuverability, ideal for organizing supplies in various settings."
+"Mimoglad Office Chair, High Back Ergonomic Desk Chair with Adjustable Lumbar Support and Headrest, Swivel Task Chair with flip-up Armrests for Guitar Playing, 5 Years Warranty",https://m.media-amazon.com/images/I/414jZL4tnaL._SS522_.jpg,With arms,Foam,Beige,https://www.amazon.com/dp/B0C2YQZS69,"['office chair', 'high back', 'ergonomic', 'adjustable lumbar support', 'headrest', 'swivel', 'flip-up armrests', 'mesh', 'wheeled base']","This Mimoglad Office Chair is designed with ergonomics in mind to provide comfort during long working hours. It features a high back that supports the natural curvature of the spine, and it is equipped with an adjustable lumbar support to reduce back strain. The headrest at the top of the chair offers additional support for the neck, making it ideal for extended periods of sitting.
+
+The chair is upholstered in a light green fabric that is both stylish and breathable, which is especially beneficial for maintaining comfort and air circulation. Its armrests are designed to flip up, allowing the user to play guitar or perform other activities where arm movement is essential without obstruction.
+
+The chair's frame and base are finished in white, giving it a clean and modern appearance that can easily blend with various office decors. It stands on a five-point caster wheelbase for easy mobility and stability. The height of the chair can be adjusted to accommodate different desk heights and user preferences.
+
+With a 5-year warranty, this chair promises durability and long-term service, ensuring that it is a sound investment for any office or home workspace.","Ergonomic office chair with adjustable lumbar support and flip-up armrests, upholstered in light green fabric with a white frame, featuring a high back and neck-supporting headrest."
+"Let the Adventure Begin Door Mat 17""x30"" Decorative Farmhouse Home Campsite Travel Trailer Cabin Indoor Outdoor Front Porch Door Mat,Funny Camping Welcome Mat Decor,Gifts for Campers Camping Lovers",https://m.media-amazon.com/images/I/51Zv2zReYCL._SS522_.jpg,Farmhouse,Rubber,,https://www.amazon.com/dp/B0C8SJSZYS,"['door mat', 'decorative', 'farmhouse', 'indoor', 'outdoor', 'striped', 'text']","This is a decorative door mat featuring an inspirational phrase ""Let the adventure begin"" in a stylized font, accented with arrow and heart designs. The mat measures 17 inches by 30 inches, making it a suitable size for a front porch or entryway. The design incorporates a farmhouse aesthetic with horizontal stripes in a palette of neutral and blue tones, which could complement a variety of home decors, particularly those with a rustic or outdoor theme.
+
+The mat is intended for both indoor and outdoor use, making it versatile for different settings such as a home, campsite, travel trailer, or cabin. Its design and message make it an appealing accessory for those who enjoy camping or traveling, and it could serve as a charming welcome to guests. Additionally, it could be considered as a thoughtful gift for campers or individuals who love the adventure lifestyle.","Decorative ""Let the adventure begin"" door mat with a farmhouse style, perfect for outdoor enthusiasts and travelers."
+"1 Pack Adjustable Height Center Support Leg for Bed Frame,Bed Center Slat Heavy Extra Durable Steel Support Legs | Suitable for Bed Frame,Cabinet and Wooden Furniture",https://m.media-amazon.com/images/I/41ci+bNjHPL._SS522_.jpg,Straight,,1 Pack,https://www.amazon.com/dp/B09BVQZM3S,"['bed frame support leg', 'adjustable height', 'steel', 'durable']","The image displays an adjustable height center support leg designed for bed frames and other types of wooden furniture. This support leg is made from heavy-duty steel, ensuring durability and extra support. It features a broad, flat top bracket that can be mounted directly to the center slat of a bed frame or the underside of other furniture pieces for additional stability.
+
+The leg itself is telescopic, allowing for height adjustments to fit various furniture heights. This adjustability is achieved through a threaded rod mechanism that can be extended or retracted and then secured in place. At the base of the leg is a wide, circular foot with a black, non-marring pad to prevent damage to flooring and to enhance stability.
+
+The support leg is finished in a metallic color, likely chrome or stainless steel, which gives it a sleek, modern appearance. This item is typically used to prevent sagging in the middle of a bed or to reinforce other large furniture items that require additional support in the center. The image also shows a close-up of the adjustment mechanism and the base, providing a clear view of the product's components.","Adjustable height center support leg for bed frames and furniture, made from heavy-duty steel with a telescopic design for stability and floor protection."
+Stylo Culture Traditional Cotton Patchwork Embroidered Ottoman Stool Pouf Cover Black Floral Seat 45 cm Seating Pouffe Case Bean Bag Home Decor,https://m.media-amazon.com/images/I/61-CBGZCsJL._SS522_.jpg,,Cotton,Black,https://www.amazon.com/dp/B0722V6KMX,,,
+"Suptsifira Shoe storage box, 24 Packs Shoe Boxes Clear Plastic Stackable,Shoes Sneaker Container Storage Box, XL Shoe Boxes with Lids for Closet, Storage and Display(White)",https://m.media-amazon.com/images/I/51enKGSxK8L._SS522_.jpg,,Porcelain,White,https://www.amazon.com/dp/B0BZ85JVBN,,,
+"Wellynap Computer Desk,31.5 inches Folding Table Computer Desk for Small Spaces Home Office, Gaming Desk Computer Table, No Install Needed, Modern Simple Style Laptop Table, Teak & Black",https://m.media-amazon.com/images/I/51pO-N48teL._SS522_.jpg,Modern,Wood,Teak & Black,https://www.amazon.com/dp/B0CFL2G31X,,,
+"Smlttel Gold Clothing Rack With Shelves, Gold Coat Rack Freestanding with Marble Base, Coat Hanger Rack,Hat Tree Coat Rack Standing Clothes Racks for Boutique，Bedroom",https://m.media-amazon.com/images/I/41aRwocdfAL._SS522_.jpg,Modern,Metal,C gold,https://www.amazon.com/dp/B0B93TC1Z8,,,
+"Franklin Sports NFL Storage Ottoman + Container - NFL Football Team Folding Organizer Bins - NFL Office, Bedroom + Living Room Décor - 14"" x 14""",https://m.media-amazon.com/images/I/31ptZB+wS-L._SS522_.jpg,Team Licensed Storage Ottoman with Detachable Lid,Fabric,Team Color,https://www.amazon.com/dp/B0787KRJ8S,,,
+"Honey-Can-Do 3-Tier Nesting Bamboo Shoe Rack SHO-09492 Shoe Rack, Wooden Shoe Rack, 3 Tier Shoe Rack, Stackable Shoe Rack, Bamboo Shoe Rack,60 lbs",https://m.media-amazon.com/images/I/51GnnjKaVsL._SS522_.jpg,Shoe,,,https://www.amazon.com/dp/B08WRLKR7T,,,
+"Furnistar 15.9 inch Modern Round Velvet Storage Ottoman with Gold Plating Base, Upholstered Footrest Makeup Stool for Bedroom Living Room, Grey(Ship from US)",https://m.media-amazon.com/images/I/31IBS5mzYSL._SS522_.jpg,Modern,Wood,Grey,https://www.amazon.com/dp/B0C4NT8N8C,,,
+"AMHANCIBLE C Shaped Side Table, End Tables Set of 2 with Charging Station, Couch Tables That Slide Under with USB Port & Outlet, Snack Table for Living Room,TV Tray, Black, HET02BPBK",https://m.media-amazon.com/images/I/41qDAGoNCrL._SS522_.jpg,Straight Leg,Engineered Wood,Black,https://www.amazon.com/dp/B0BT9SVN1V,,,
+LONGWIN Black Hanging Wall Round Mirror Decor Geometric Circle Mirror for Bathroom Bedroom Living Room 10.2inch,https://m.media-amazon.com/images/I/41kC6cU5HXS._SS522_.jpg,Modern,"Glass, Metal",Black,https://www.amazon.com/dp/B094F897P3,,,
+"Need Fold Wall Mounted Workbench Folding Wall Table Length 47.2"" Width 20"" Perfect Addition to Garage & Shed/Home Office/Laundry Room/Home Bar/Kitchen & Dining Room",https://m.media-amazon.com/images/I/31SqvdFCutL._SS522_.jpg,Modern,Metal,Teak Color Desktop & Warm White Folding Brackets,https://www.amazon.com/dp/B00UV7B29A,,,
+"Big Joe Fuf XL Cover Only Machine Washable, Gray Plush, Soft Polyester",https://m.media-amazon.com/images/I/21ysztDdCYL._SS522_.jpg,Plush,,Grey,https://www.amazon.com/dp/B08T7JP8ZN,,,
+"Plymor Rectangle 5mm Beveled Glass Mirror, 6 inch x 12 inch (Pack of 6)",https://m.media-amazon.com/images/I/31wigA5chuL._SS522_.jpg,,Glass,Silver,https://www.amazon.com/dp/B09F3SGZ8Y,,,
+TIMCORR CD Case DVD Holder Storage: 144 Capacity DVD Cases Organizer Portable Wallet Storage - CD Plastic Protective Carrying Binder for Home Travel (Black),https://m.media-amazon.com/images/I/411Q2ETwelL._SS522_.jpg,Portable,EVA + PVC + PP + Non-woven fabric,Black,https://www.amazon.com/dp/B0B19ZGGXC,,,
+Ginger Cayden Closed Towel Ring - 4905/SN - Satin Nickel,https://m.media-amazon.com/images/I/31LNv7QILdL._SS522_.jpg,,Brass,Satin Nickel,https://www.amazon.com/dp/B00U0ECLG2,,,
+"Brightify Black Bathroom Mirrors for Wall, 24 x 36 Inch Rectangle Bathroom Mirrors for Vanity Black Metal Framed Anti-Rust, Matte Black Mirror for Farmhouse Hallway Wall Decor, Horizontal or Vertical",https://m.media-amazon.com/images/I/510A0nIdGZL._SS522_.jpg,Modern,Aluminum,Black,https://www.amazon.com/dp/B0C2HNGCRX,,,
+"SogesHome Wood Corner Cabinet Wall Corner Storage Cabinet, Storage Display Table Stand Cabinet, with Doors and Open Shelf, for Small Places, Living Room, White&Teak",https://m.media-amazon.com/images/I/41BTUFVwm+L._SS522_.jpg,Open Frame,,White&teak,https://www.amazon.com/dp/B0C3B4D4RH,,,
+Toy Storage for Lego Play Mat Bag - Duplo Toy Organizer Storage Bag by Drawstring for Kids Tidy Toy with Lips can as Gift (Orange 47 inch),https://m.media-amazon.com/images/I/51KKvmDCqBL._SS522_.jpg,,Nylon,Orange,https://www.amazon.com/dp/B0B4CL1M1M,,,
+Flash Furniture Jefferson 2 Pk. Contemporary Brown Vinyl Adjustable Height Barstool with Panel Back and Chrome Base,https://m.media-amazon.com/images/I/41GYYVLfGjL._SS522_.jpg,Contemporary,,Brown,https://www.amazon.com/dp/B00FEAN1SY,,,
+"Hong Art- Metal Mirror-Matt Black,Glass Panel Black Framed Rounded Corner,Mirrored Rectangle Hangs Horizontal or Vertical,MR2001BK1，11.8"" 15.74""",https://m.media-amazon.com/images/I/31XytAHobHL._SS522_.jpg,Classic,Metal,Black,https://www.amazon.com/dp/B08GSH4KVM,,,
+"Convenience Concepts American Heritage Round End Table, Pink",https://m.media-amazon.com/images/I/311rmB9BDWL._SS522_.jpg,Round End Table,"Solid + Manufactured Wood,Particle Board/Chipboard",Pink,https://www.amazon.com/dp/B01B65BYYI,,,
+Flash Furniture Diamond Black Vinyl Luxurious Conference Chair with Accent Nail Trim,https://m.media-amazon.com/images/I/41LYsAMww6L._SS522_.jpg,Fixed,Foam,Black Vinyl,https://www.amazon.com/dp/B000TMHWGO,,,
+"Gatco 1918, Modern Rectangle Waste Basket, Matte Black / 11.25"" H Open Top Stainless Steel Trash Can with Removable Lid, 12 Liter Capacity",https://m.media-amazon.com/images/I/31dnAVaEmvL._SS522_.jpg,Rectangle,Stainless Steel,Matte Black,https://www.amazon.com/dp/B07TXMJ5FQ,,,
+"Winrise Office Chair Ergonomic Desk Chair, High Back Gaming Chair, Big and Tall Reclining chair Comfy Home Office Desk Chair Lumbar Support Breathable Mesh Computer Chair Adjustable Armrests(B-Orange)",https://m.media-amazon.com/images/I/41hCFaVIC+L._SS522_.jpg,Straight,Sponge,S-black,https://www.amazon.com/dp/B0CGQZBCZP,,,
+"Adeco Euro Style Fabric Arm Bench Chair Footstool Cubic Ottomans, Brown",https://m.media-amazon.com/images/I/41hUc8c+DCL._SS522_.jpg,Modern,Engineered Wood,Brown,https://www.amazon.com/dp/B017TNJR72,,,
+"Motiv 0202/PC Sine 18-In Towel Bar, Polished Chrome,,18"" Towel Bar",https://m.media-amazon.com/images/I/31a6GfenW0L._SS522_.jpg,,Brass,"18"" Towel Bar",https://www.amazon.com/dp/B001AS8D82,,,
+"Imports Décor PVC Backed Coir Doormat, Eighth Note Welcome, 18""x30""",https://m.media-amazon.com/images/I/51H9lDOICrL._SS522_.jpg,Art Deco,Vinyl,Black and Beige,https://www.amazon.com/dp/B08WF83LMF,,,
+"Croydex Chester Square Flexi-Fix Wall Mounted Bathroom Tilt Mirror, 14.96in x 14.96in x 3.4in, Chrome",https://m.media-amazon.com/images/I/41sDO1HW2cL._SS522_.jpg,,,Silver,https://www.amazon.com/dp/B09DGFRM4B,,,
+itbe Easy Fit Ready-to-Assemble Multipurpose One Door Wall Steel Cabinet Space Organizer (Blue),https://m.media-amazon.com/images/I/21NWASZgUVL._SS522_.jpg,Flat Panel,Alloy Steel,Blue,https://www.amazon.com/dp/B09FR4XSCT,,,
+Delta ARV18-DN Arvo 18-in Wall Mount Towel Bar with 6-in Extender Bath Hardware Accessory in Brushed Nickel,https://m.media-amazon.com/images/I/11zzs81fXBL._SS522_.jpg,"18"" Towel Bar with 6"" Extender",Multiple Base Materials,Spotshield Brushed Nickel,https://www.amazon.com/dp/B09LVSZRZS,,,
+"Bamboo Waste Basket | Waste Basket for Bathroom | Waste Basket for Office | Bathroom Trash Can | Bedroom Trash Can | Trash Can Small Wastebasket Bamboo Decor (1, 12"" x 6.5"" x 15.3"")",https://m.media-amazon.com/images/I/318RY00VlIL._SS522_.jpg,,,,https://www.amazon.com/dp/B08VWTB8CH,,,
+Way Basics Vinyl Record Storage - 2 Tier Book Shelf Turntable Stand (Fits 170 Albums),https://m.media-amazon.com/images/I/41YMttt7a5L._SS522_.jpg,Modern,Recycled Material,White,https://www.amazon.com/dp/B075M1PKSW,,,
+"TocTen Double Bath Towel Bar - Thicken SUS304 Stainless Steel Towel Rack for Bathroom, Bathroom Accessories Double Towel Rod Heavy Duty Wall Mounted Square Towel Holder (Matte Black,24 INCH)",https://m.media-amazon.com/images/I/41cFJKXyA5L._SS522_.jpg,,Stainless Steel,Matte Black,https://www.amazon.com/dp/B0BWRVGQRM,,,
+"MoNiBloom Adjustable Bar Stools Set of 2, 360° Swivel Counter Height Barstool Kitchen Counter Island Dining Chair, Dark Grey PU Leather Bistro Swivel Stool for Living Room Kitchen, 250 lbs Capacity",https://m.media-amazon.com/images/I/41jD28iN4bL._SS522_.jpg,Straight,,Dark Grey,https://www.amazon.com/dp/B0CB7SG59J,,,
+"LANTEFUL Shoe Rack Organizer Shoe Storage Cabinet 8 Tiers 32 Pair Portable Shoe Storage Sturdy Plastic Black Shoe Shelf with Hooks Shoe Rack with Door for Entryway, Bedroom and Hallway",https://m.media-amazon.com/images/I/51e8SrHHW3L._SS522_.jpg,free standing shoe racks,,Black,https://www.amazon.com/dp/B0C3QDL2XW,,,
+"ANDY STAR 24x32 INCH Brushed Nickel Mirror, Rounded Rectangle Silver Metal Framed Mirrors for Bathroom Anti-Rust SUS304, Tube Metal Frame, 1’’ Deep Wall Mounted Vertically/Horizontal",https://m.media-amazon.com/images/I/41MQWfATggL._SS522_.jpg,,Stainless Steel,Brushed Nickel,https://www.amazon.com/dp/B0CBRGS5D7,,,
+"MJL Furniture Designs Upholstered Cubed/Square Olivia Series Ottoman, 17"" x 19"" x 19"", Smoke Grey",https://m.media-amazon.com/images/I/410tv-zDYXL._SS522_.jpg,Contemporary,Wood,Smoke Grey,https://www.amazon.com/dp/B01D378FYE,,,
+Cpintltr Small Foot Stool Ottoman Modern Accent Step Stool Seat with Solid Wood Legs Velvet Soft Padded Pouf Ottomans Sofa Footrest Stools for Couch Living Room Bedroom Entryway (Green 2 PCS),https://m.media-amazon.com/images/I/51CjfUJVuLL._SS522_.jpg,,Pine,Green,https://www.amazon.com/dp/B0CKPFKDZY,,,
+"YuiHome Extendable Round, Farmhouse 16"" Leaf Table for Dining Room, Kitchen,Natural Wood Wash",https://m.media-amazon.com/images/I/5175Qzg03LL._SS522_.jpg,Farmhouse,"Rubber Wood, Engineered Wood",Natural Wood Wash,https://www.amazon.com/dp/B0CHVQ6BC5,,,
+"Ergonomic Office Chair,Office Chair, with Lumbar Support & 3D Headrest & Flip Up Arms Home Office Desk Chairs Rockable High Back Swivel Computer Chair White Frame Mesh Study Chair（All Black）",https://m.media-amazon.com/images/I/51vnoZERmPL._SS522_.jpg,With arms,Foam,All Black,https://www.amazon.com/dp/B0CBBV4S1P,,,
+"Kate and Laurel Celia Round Metal Foldable Accent Table with Tray Top, Black",https://m.media-amazon.com/images/I/31ZMqrgDD8L._SS522_.jpg,Modern,Iron,Black,https://www.amazon.com/dp/B084WLY61H,,,
+"Lizipai Floating Bedside Table, No Assembly Required,Night Stand with Drawer, 19"" X 13"" X 5.9"", White",https://m.media-amazon.com/images/I/41HBX6be98L._SS522_.jpg,no,Wood,White,https://www.amazon.com/dp/B09NBWCTDS,,,
+"CordaRoy's Chenille Bean Bag Ottoman Footstool, 26"" x 17"", Rainforest",https://m.media-amazon.com/images/I/51HpCirQNAL._SS522_.jpg,Modern,Engineered Wood,Rainforest,https://www.amazon.com/dp/B0BSZ96YG7,,,
+"Plebs Home Solid Desktop Store Cart, with Rubber Wood Countertop, Island has 8 Handle-Free Drawers, Including a Flatware Organizer and 5 Wheels, for Dining, Kitchen, Living Room-Dark Blue, 53.15 Inch",https://m.media-amazon.com/images/I/51WFQwBEqjL._SS522_.jpg,Slab,Wood,Dark Blue,https://www.amazon.com/dp/B0CD7FSWMK,,,
+"ErGear Ergonomic Desk Chair, Office Chair with 2'' Adjustable Lumbar Support, Computer Chair with Flip-Up Armrests & Headrest, High Back Office Desk Chair with 4'' Thicken Seat for Home Office, Black",https://m.media-amazon.com/images/I/41C4FUmS-hL._SS522_.jpg,With arms,Memory Foam,Black,https://www.amazon.com/dp/B0C99D3V15,,,
+"Kingston Brass Millennium Towel-Ring, 7.63"", Oil Rubbed Bronze",https://m.media-amazon.com/images/I/31+kzwXTjxS._SS522_.jpg,,Brass,Oil Rubbed Bronze,https://www.amazon.com/dp/B00FM0WG7I,,,
+"Homebeez 18.9"" Round Velvet Storage Ottoman Multi-Function Vanity Stool Footrest with Storage for Bedroom/Living Room,Orange",https://m.media-amazon.com/images/I/51vTxE-9lHL._SS522_.jpg,Modern,Wood,Orange,https://www.amazon.com/dp/B09DKG6JDN,,,
+Mickey and Friends Collapsible Nylon Basket Bucket Toy Storage Tote Bag,https://m.media-amazon.com/images/I/410mEc5bblL._SS522_.jpg,,,,https://www.amazon.com/dp/B0B7Q5LB2C,,,
+Homepop Home Decor | Backless Nailhead Trim Counter Height Bar Stools | 24 Inch Bar Stools | Decorative Home Furniture (Blue Tweed),https://m.media-amazon.com/images/I/41HPIScA4sL._SS522_.jpg,Contemporary,,Blue,https://www.amazon.com/dp/B01LWPSVUW,,,
+"Camco Life Is Better at The Campsite Outdoor & Indoor Welcome Mat - Weather and Doormat | Traps Dirt and Liquid | Spongey Comfortable Feel | Measures 26 ½ "" x 15"" - Blue (53201) - 53201-A",https://m.media-amazon.com/images/I/51DN2is3ZjL._SS522_.jpg,Outdoor & Indoor,Rubber,Blue,https://www.amazon.com/dp/B07D7RQNJV,,,
+MoNiBloom Round Folding Faux Fur Saucer Chair for Bedroom Living Room Dorm Courtyard Foldable Metal Frame Oversized Large Comfy Furry Padded Soft Lounge Lazy Cozy Moon Chair for Adults (Burgundy),https://m.media-amazon.com/images/I/41eoFKL3gKL._SS522_.jpg,Modern,Polyester,Burgundy,https://www.amazon.com/dp/B0CD7TH3BF,,,
+"YMYNY Vanity Stool Chair with Storage, Square Velvet Ottoman Foot Stool, Modern Multifunctional Makeup Stool for Bedroom, Living Room, Office, Gold Legs, 18.9 * 15.75 * 11.6"", Dusty Blue, UHBD024G",https://m.media-amazon.com/images/I/519Am3LPMvL._SS522_.jpg,Modern,,Dusty Blue,https://www.amazon.com/dp/B0C1NSNDW2,,,
+"Casual Home 5 Piece Tray Table Set, Espresso",https://m.media-amazon.com/images/I/41WweDJqgZL._SS522_.jpg,Tray Table Set,Wood,Espresso,https://www.amazon.com/dp/B0069H9BYO,,,
+"Simplify Hanging Grey 20-Pocket Shoe Boho Closet Storage Organization | Dimensions: 0.25"" x 22"" x 54"" | Hangs on Door| Closet | 20 Pockets | Grey | Closet Organization",https://m.media-amazon.com/images/I/41eYiOqsldL._SS522_.jpg,,80% Linen printed nonwoven +20% solid nonwoven+ 20%PAPERBOARD,Grey,https://www.amazon.com/dp/B09J1RM23P,,,
+"Get Set Style Black Glass Side Table, Square Glass End Table Modern Coffee Table with Storage, Glass Mirrored Table for Living Room, Balcony, Bedroom, Office",https://m.media-amazon.com/images/I/51gG6ukN1nL._SS522_.jpg,Modern and Elegant,Tempered Glass,Shiny Black,https://www.amazon.com/dp/B0C5DH6ZY6,,,
+"Watson & Whitely Swivel Bar Stools Set of 2, Faux Leather Upholstered Counter Height Barstool with Back, 26"" H Seat Height Counter Stools with Solid Wood Legs (Black)",https://m.media-amazon.com/images/I/41nDc6aFKoL._SS522_.jpg,Modern,,Black,https://www.amazon.com/dp/B0CKQTTZ5V,,,
+Sweet Jojo Designs Boho Rainbow Girl Ottoman Pouf Cover Unstuffed Poof Floor Footstool Square Cube Pouffe Storage Baby Nursery Kids Room Blush Pink Yellow Bohemian Modern Neutral Vintage Taupe Beige,https://m.media-amazon.com/images/I/31nn4NwuKfL._SS522_.jpg,Shabby Chic,Engineered Wood,Multi Color,https://www.amazon.com/dp/B0BZJYM4Q6,,,
+"Pekokavo Sofa Arm Clip Tray, Side Table for Remote Controls/Drinks/Gamepads Holder (Bamboo)",https://m.media-amazon.com/images/I/51yz-83kj+L._SS522_.jpg,Modern,Bamboo,Bamboo,https://www.amazon.com/dp/B08SL4GH7G,,,
+"Caroline's Treasures JMA2013HRM2858 Seaweed Salad Mahi Indoor/Outdoor Runner Mat 28x58,",https://m.media-amazon.com/images/I/514qJ5aPtbL._SS522_.jpg,Modern,Rubber,Multicolored,https://www.amazon.com/dp/B07SPYM4M5,,,
+"Xchouxer Side Tables Natural Bamboo Sofa Armrest Clip-On Tray, Ideal for Remote/Drinks/Phone (Round)",https://m.media-amazon.com/images/I/511LXRAxI+L._SS522_.jpg,Modern,Bamboo,Beige,https://www.amazon.com/dp/B08FC5HPBS,,,
+"Montessori Learning Toddler Tower, Foldable Toddler Step Stool Kitchen Stool Helper for Kids, 4 in 1 Toddler Kitchen Step Stool Helper Standing Tower, Kids Step Stool with Chalkboard & Safety Rail",https://m.media-amazon.com/images/I/51n9ojprZEL._SS522_.jpg,Modern,Wood,Wood,https://www.amazon.com/dp/B0CKMRJ1H9,,,
+"PAK HOME Set of 2 High Gloss Brown Marble Look End Tables Round Wood Sofa Side Coffee Tables for Small Spaces, Nightstand Bedside Table for Bedroom, Living Room, Office",https://m.media-amazon.com/images/I/51u3oxvEiSL._SS522_.jpg,Tripod,Wood,Brown Marble High Gloss / Gold Legs,https://www.amazon.com/dp/B09K3MYL91,,,
+kukli kitchen Spring Door Mat 30 X 17 Inch - Spring Floral Welcome Doormat Indoor Outdoor Entrance Patio Floor Mat Non Slip Durable Spring Decor Rubber Mats,https://m.media-amazon.com/images/I/61rRHgR+aEL._SS522_.jpg,Classic,Rubber,Color-33,https://www.amazon.com/dp/B0BNL8CC5X,,,
+"Dewhut Oversized Pumpkin Couch Accent Chair, Modern Comfy Velvet Upholstered Barrel Chairs, Luxury Single Sofa Armchair for Living Room, Waiting Room, Office and Vanity, (Navy)",https://m.media-amazon.com/images/I/519KoH2aW4L._SS522_.jpg,Modern,Sponge,Navy,https://www.amazon.com/dp/B0CF8HTCS4,,,
+Toland Home Garden 800009 Gypsy Garden Flower Door Mat 18x30 Inch Outdoor Doormat for Entryway Indoor Entrance,https://m.media-amazon.com/images/I/61gTdPHg5QS._SS522_.jpg,Outdoor & Indoor,Rubber,,https://www.amazon.com/dp/B00PNJAACG,,,
+"Sintosin Vintage Oval Mirrors for Wall Decor 11 inch, Hanging Farmhouse Wood Entryway Mirror, Distressed White Rustic Scalloped Mirror Wall Decor, Ornate Accent Sculpted Mirror for Living Room",https://m.media-amazon.com/images/I/41NiOP0+4jL._SS522_.jpg,Shabby Chic,Wood,Oval,https://www.amazon.com/dp/B0BWJLZF5G,,,
+"BEWISHOME Vanity Stool, Bedroom Vanity Chair with Upholstered Seat, Desk Stool Piano Stool Soft Cushioned Stool, Square 18” Height Makeup Bench, Piano Bench Vanity Bench Capacity 300lb Black FSD06H",https://m.media-amazon.com/images/I/410emoPl2kL._SS522_.jpg,Modern,,Black,https://www.amazon.com/dp/B0B6FML1VS,,,
+"Children's Factory School Age High Back Lounger Kids Bean Bag Chair, Flexible Seating Classroom Furniture for Homeschools/Playrooms/Daycares, Blue/Red",https://m.media-amazon.com/images/I/51ORnRyifRL._SS522_.jpg,Single Seat,,Blue-red,https://www.amazon.com/dp/B00740P05Y,,,
+"FLYJOE Shoe Rack Bench, 3-Tier Freestanding Wooden Shoe Organizer with Seat, Entryway Bench, Storage Shelf for Kitchen Living Room Bathroom Bedroom, Walnut",https://m.media-amazon.com/images/I/51WQiiIyuSL._SS522_.jpg,,,Rustic Walnut,https://www.amazon.com/dp/B0CN8NXR1Q,,,
+"FLYZC Counter Height Bar Stools Set of 4, Stools for Kitchen Island Set of 4, 24 Inch Counter Height Stools Saddle Barstools Stools for Kitchen Counter Pub Bar Dining Room Support 300 LBS(Grey)",https://m.media-amazon.com/images/I/51jw0SXQMWL._SS522_.jpg,Straight,,Grey & Black,https://www.amazon.com/dp/B0CH862BV2,,,
+SITMOD Gaming Chairs for Adults with Footrest-Computer Ergonomic Video Game Chair-Backrest and Seat Height Adjustable Swivel Task Chair with Lumbar Support(Gray)-Fabric,https://m.media-amazon.com/images/I/41bntfm39UL._SS522_.jpg,With arms,Memory Foam,Grey,https://www.amazon.com/dp/B0B3HM3FTZ,,,
+"CM Cosmos Stuffed Animal Storage Bean Bag Chair Stuffable Zipper Beanbag Stuff and Sit Bean Bag for Organizing Soft Plush, Size 22.5'', with Handle",https://m.media-amazon.com/images/I/41XEtwrKqoL._SS522_.jpg,,,Grey & White,https://www.amazon.com/dp/B07JCPZDSL,,,
+"Cionyce 4 Pcs Sectional Couch Connectors, Pin Style Furniture Connector Sectional Sofa Connector Bracket(Black)",https://m.media-amazon.com/images/I/41sejv2mO6L._SS522_.jpg,,,,https://www.amazon.com/dp/B09V6RSWSR,,,
+"Tiita Saucer Chair with Ottoman, Soft Faux Fur Oversized Folding Accent Chair,Lounge Lazy Chair, Metal Frame Moon Chair for Bedroom, Living Room, Dorm Rooms, Garden and Courtyard",https://m.media-amazon.com/images/I/51C5YkDdUyL._SS522_.jpg,Garden,,Beige With Ottoman,https://www.amazon.com/dp/B0BWDJ8NSM,,,
+Grandmother Birthday Gifts Compact Makeup Mirror for Glamma for Grandma Grandma Gifts from Grandchildren Folding Makeup Mirror for Nana Christmas Thanksgiving Gifts,https://m.media-amazon.com/images/I/417J95lDDaL._SS522_.jpg,,Stainless Steel,For Grandmother,https://www.amazon.com/dp/B0C289KQNK,,,
+"GIA 24-Inch Counter Height Square Backless Metal Stool with Holstein Cow Print Upholstery, Black, Qty of 2",https://m.media-amazon.com/images/I/414M2Vz5YjL._SS522_.jpg,Straight,,Black,https://www.amazon.com/dp/B0B75Z1T2H,,,
+Vintage Desktop Apothecary Cabinet with 3 Drawers - Solid Wood Medicine Cabinet for Tabletop - Rustic Card Catalog Drawers with Label Holders - Desktop Chest Drawer for Storage and Organization,https://m.media-amazon.com/images/I/41yz4PMNd0L._SS522_.jpg,"drawer,wood",Wood,Mahogany Wood Brown,https://www.amazon.com/dp/B0B24KQJS9,,,
+"WAYTRIM Dresser Storage Tower, 4 Fabric Organizer Drawers, Wide Chest of Drawers for Closet Boys & Girls Bedroom, Bedside Furniture, Steel Frame, Wood Top, Fabric Bins, Easy Installation (Camel)",https://m.media-amazon.com/images/I/41DfHAtQUKL._SS522_.jpg,Modern,,Camel,https://www.amazon.com/dp/B07W56HHX5,,,
+"Power Recliner Power Supply Kit-4-Piece Universal Dual Power Supply Transformer with AC Power Cord, DC Extension Cable and Y Splitter Cord 29V 2A Adapter for Recliner, Lift Chairs and Sofa",https://m.media-amazon.com/images/I/51N6Zq4kxxL._SS522_.jpg,,,,https://www.amazon.com/dp/B0BHVLGGYL,,,
+"Anna Stay Wine Rack Wall Mounted - Decorative Wine Rack with Wine Glass Holder, Wall Mounted Wine Rack inc Cork Storage & Wine Charms, Wine Gifts with Wine Bottle Holder for Wine Decor",https://m.media-amazon.com/images/I/51K1wX04DXL._SS522_.jpg,Modern,,Wine Gold,https://www.amazon.com/dp/B09ZQM2FX3,,,
+"Lufeiya Small Computer Desk with 2 Drawers for Bedroom, 31 Inch Home Office Desk with Storage Fabric Drawer and Bag, Study Writing Table for Small Spaces, Rustic Brown",https://m.media-amazon.com/images/I/41zNNJV-QUL._SS522_.jpg,Country Rustic,Engineered Wood,Rustic Brown,https://www.amazon.com/dp/B0CB5G1BHX,,,
+"Watson & Whitely Swivel Bar Stools Set of 2, Fabric Upholstered Counter Height Barstool with Back, 26"" H Seat Height Counter Stools with Solid Wood Legs (White (Multi-Colored))",https://m.media-amazon.com/images/I/41IWqaJGuWL._SS522_.jpg,Modern,,White (Multi-colored),https://www.amazon.com/dp/B0BV6KR1T7,,,
+"Adeco Large Square Storage Ottoman Bench, Tufted Upholstered Coffee Table Footstool Footrest with Wood Legs for Living Room Bedroom, Orange Brown",https://m.media-amazon.com/images/I/31HEdjZpCbL._SS522_.jpg,Mid-Century Modern,Wood,Orange Brown,https://www.amazon.com/dp/B0C6XNNL9M,,,
+"New Classic Furniture Evander Wood End Table with Drawer and Storage, Two Tone Cream/Brown",https://m.media-amazon.com/images/I/51TJVV3sRqL._SS522_.jpg,Contemporary,Wood,Two Tone Cream/Brown,https://www.amazon.com/dp/B0B6YR22H1,,,
+"Lipper International Wooden Storage Crate, white 14"" x 12 1/2"" x 11""",https://m.media-amazon.com/images/I/31MZPtCF0RL._SS522_.jpg,,,,https://www.amazon.com/dp/B07MZRYQ2X,,,
+"Amazon Basics Kids Adjustable Mesh Low-Back Swivel Study Desk Chair with Footrest, Red",https://m.media-amazon.com/images/I/41bsjzUI6NL._SS522_.jpg,Mesh,,Red,https://www.amazon.com/dp/B0BHF9PPJC,,,
+"Joovy Coo Bassinet, Portable Bassinet with Storage, Rocking Playpen, Gray",https://m.media-amazon.com/images/I/41UOfS3JmkL._SS522_.jpg,,fabric,,https://www.amazon.com/dp/B07NFSLLCG,,,
+"Halatua 6ftlarge Fur Bean Bag Cover Lazy Sofa Chair Living Room Large Circular Soft Fluffy Artificial Fur unfilled Bean Bag (SnowBlue, 7FTD70.8inchH35.4inch)",https://m.media-amazon.com/images/I/51-utQ4pnbL._SS522_.jpg,,Polyester,Snowblue,https://www.amazon.com/dp/B0C7L8GGJF,,,
+"Flash Furniture Walker Small Rustic Natural Home Office Folding Computer Desk - 36""",https://m.media-amazon.com/images/I/31QOFqtaHJL._SS522_.jpg,Sled,Engineered Wood,Rustic,https://www.amazon.com/dp/B08JWJTZ1Y,,,
+BOKKOLIK Vintage Bar Stools Swivel PU Seat 29-37inch Height Adjustable Extral Tall Bicycle Stool with Bikepedal Kitchen Island Counter Stool Shop Chairs,https://m.media-amazon.com/images/I/41PjcPoHTLL._SS522_.jpg,Soft PU Seat,,Dark Brown,https://www.amazon.com/dp/B0BG7MX77T,,,
+"Nalupatio Storage Ottoman, Bedroom End Bench，Upholstered Fabric Storage Ottoman with Safety Hinge, Entryway Padded Footstool, Ottoman Bench for Living Room & Bedroom(Light Green)",https://m.media-amazon.com/images/I/31+6K0TbdpL._SS522_.jpg,Modern,Wood,Light Green,https://www.amazon.com/dp/B0C48X7JQB,,,
+"Homevany Bamboo Wine Rack,4 Tier, Wine Bottle Holder, Hold 16 Bottles for Home Kitchen, Dinging Room, Pantry, Cabinet, Bar",https://m.media-amazon.com/images/I/51DO5hfgdKL._SS522_.jpg,Modern,,Brown,https://www.amazon.com/dp/B08T8ZRZ1F,,,
+"Armen Living Julius 30"" Cream Faux Leather and Walnut Wood Bar Stool",https://m.media-amazon.com/images/I/31v34T0kgnS._SS522_.jpg,Straight,,Cream/Walnut,https://www.amazon.com/dp/B0961N94SZ,,,
+"WONSTART Vanity Mirror with Lights, 50 x 41cm Hollywood Lighted Makeup Mirror with 15 Dimmable Lights, Make up Mirror with Lighting, Wall Mount or Tabletop Mirror for Bedroom (Silver)",https://m.media-amazon.com/images/I/41k7g8oo6bL._SS522_.jpg,Modern,"Aluminum, Glass",Silver,https://www.amazon.com/dp/B0C2VF2S6R,,,
+Cpintltr Velvet Foot Rest Stool Multipurpose Dressing Stools Upholstered Round Storage Ottoman Modern Soft Vanity Chair with Memory Foam Seat Dusty Pink,https://m.media-amazon.com/images/I/51K84REZCGL._SS522_.jpg,Modern,Wood,Dusty Pink,https://www.amazon.com/dp/B0CH34CCLV,,,
+"uxcell Shredded Memory Foam Filling, 10 Pounds Bean Bag Filler Foam for Bean Bag Chairs, Cushions, Sofas, Pillows and More - Multi Color",https://m.media-amazon.com/images/I/51i6LeHlc9L._SS522_.jpg,,,,https://www.amazon.com/dp/B0C4DWRF3M,,,
+"FAMSINGO Ergonomic Mesh Office Chair, High Back Comfortable Desk Chair with Adjustable Lumbar Support, Headrest and Flip-up arms, Wide Memory Foam Seat, Executive Swivel Chair(Black)",https://m.media-amazon.com/images/I/41Jm-GtY+5L._SS522_.jpg,With arms,Memory Foam,Black,https://www.amazon.com/dp/B0CBBMQPVC,,,
+"Serta Style Hannah II Office Chair, Harvard Pink Microfiber",https://m.media-amazon.com/images/I/41XQ7R6j7lL._SS522_.jpg,with-arms,Foam,Harvard Pink,https://www.amazon.com/dp/B07667648L,,,
+"Christmas 3D Illusion Doormat, Non-Slip Visual Door Mat,3D Stereo Floor Mat for Christmas Decoration Indoor and Outdoor, Hallway, Entrance, Kitchen, Bathroom(50 * 80/60 * 90CM)",https://m.media-amazon.com/images/I/51uOa02x4HL._SS522_.jpg,Classic,棉质,Red,https://www.amazon.com/dp/B0CC28VDSV,,,
+"Narrow Console Table with Power Strips, Sofa Table with Storage Shelves for Living Room, 2-Tier Foyer Table for Entryway, Hallway, Behind Couch, Kitchen Counter, 39'', Black & White",https://m.media-amazon.com/images/I/51FRxl-qgFL._SS522_.jpg,Sofa Table with Outlets,MDF Board and Metal,Black,https://www.amazon.com/dp/B0BSHFVY3J,,,
+"AnRui Folding Floor Chair with Adjustable Back Support, Comfortable, Semi-Foldable, and Versatile, for Meditation, Seminars, Reading, TV Watching or Gaming, Suitable for Home Or Office",https://m.media-amazon.com/images/I/51iuIrMVq+L._SS522_.jpg,Solid Back,Foam,Stripe,https://www.amazon.com/dp/B08QRF4TTL,,,
+"sogesfurniture 5 Tier Free Standing Wooden Shoe Storage Shelf Shoe Organizer, 29.5 inches Shoe Rack Shoe Organizer Storage Cabinet for Entryway, Living Room, Hallway, Doorway, Black",https://m.media-amazon.com/images/I/51j2v3ij2uL._SS522_.jpg,Modern,Engineered Wood,,https://www.amazon.com/dp/B07WLK9TNS,,,
+"fengxiaomin-Plastic Bed Slat End Caps Holders Plastic End Caps Holders Plastic Black End Caps Holders Suitable for Single, Double, King and Queen beds (55mm*9mm Black) -12pcs",https://m.media-amazon.com/images/I/41gvi7RjrZL._SS522_.jpg,,,,https://www.amazon.com/dp/B0CNVJ24YF,,,
+"MoNiBloom Massage Gaming Recliner Chair with Speakers PU Leather Home Theater Seating Single Bedroom Video Game Sofa Recliners Ergonomic Gaming Couch with Detachable Neck Support and Footrest, Green",https://m.media-amazon.com/images/I/41Md8gR4YYL._SS522_.jpg,Modern,,Green,https://www.amazon.com/dp/B0BZKMYST2,,,
+"SUNSLASH Wall Mounted Mirror, Arched Wall Mirror for Bathroom, 36""x24"" Arch Bathroom Mirror with Metal Frame, Black Vanity Mirror Decor for Mantle, Bedroom, Entryway, Living Room",https://m.media-amazon.com/images/I/41nGiqXS+5L._SS522_.jpg,,Aluminum,Black（arched）,https://www.amazon.com/dp/B0BP9QYFTL,,,
+"Allied Brass Carolina Crystal Collection Frameless Oval Tilt Beveled Edge Wall Mirror, Antique Brass",https://m.media-amazon.com/images/I/21+UCtQ6p9L._SS522_.jpg,Antique,Brass,Antique Brass,https://www.amazon.com/dp/B07ZSF42WD,,,
+"Home Source 40.7' Elegance Bar Server and Wine Glass Cabinet, 12-Bottle Wine Rack, Rollers for Mobility, and Interior Drawer (Walnut)",https://m.media-amazon.com/images/I/41nYPK8XbrL._SS522_.jpg,Fluted shape,Walnut Wood,Walnut,https://www.amazon.com/dp/B0CN1LGXNP,,,
+"Shintenchi 60"" Small Loveseat, 3 in 1 Cute Convertible Sofa Bed, Modern Futon Recliner Sleeper w/2 Cup Holder, Upholstered Folding Couch for Small Space, Dark Gray",https://m.media-amazon.com/images/I/41SkpIbGdQL._SS522_.jpg,Pillow-Top,Wood,Dark Gray,https://www.amazon.com/dp/B0CMTHD198,,,
+"King Mattresses Bag for Moving Storage Protector, Waterproof Reusable Mattress Cover with Heavy Duty 8 Handles Water Resistant Zipper Closure and 2 Adjustable Straps, Bright Blue",https://m.media-amazon.com/images/I/41ye8pFDZ9L._SS522_.jpg,,,,https://www.amazon.com/dp/B0CN44TTFJ,,,
+"sawsile Asymmetrical Wall Mirror,Unique Gold Vintage Baroque Vanity Mirror,18.7x24 Mid Century Modern Decor Irregular Mirror for Living Room, Bedroom,Bathroom",https://m.media-amazon.com/images/I/41G-NEOXwfL._SS522_.jpg,,"Wood, Iron",Gold,https://www.amazon.com/dp/B0CDWH5PQP,,,
+"Leather At Home, Decorative 13 Inch Rounded Pillow Handmade from Full Grain Leather - Chair Seat, Confortable Sitting for Round Wooden/Metal Stools - Bourbon Brown",https://m.media-amazon.com/images/I/51ePbFDPNRL._SS522_.jpg,Classic,Leather,Bourbon Brown,https://www.amazon.com/dp/B0BBKQ3XW9,,,
+"Hzuaneri Blanket Ladder Shelf for Living Room, Decorative Wood Quilt Rack with 4 Removable Hooks, 5-Tier Farmhouse Ladder Holder Organizer for Bedroom, Rustic Brown 02101BBR",https://m.media-amazon.com/images/I/31XETwaX0WL._SS522_.jpg,Farmhouse,,,https://www.amazon.com/dp/B0BSKY28M7,,,
+"9 Inch lighted magnifying mirror with Adjustable Height, Double Side 1x/10x Magnifying Mirror with Light, 360°Swivel Vanity Mirror with Stand Brightness Adjustable Travel Cosmetic Mirror (Nickel)",https://m.media-amazon.com/images/I/41j2FBzCCJL._SS522_.jpg,Modern,Alloy Steel,Brushed Nickel,https://www.amazon.com/dp/B0CMJCCT9C,,,
+"shopperals Large Black Fogless Handheld Shaving Mirror with Ergonomic Handle -Perfect for Men and Women, 9"" x 13’’",https://m.media-amazon.com/images/I/413+UE2HxQL._SS522_.jpg,,Plastic,Black,https://www.amazon.com/dp/B0CJCRFZCG,,,
+"Convenience Concepts French Country Desk, Driftwood / White",https://m.media-amazon.com/images/I/21Xa4sH6hPL._SS522_.jpg,French Country,Engineered Wood,Driftwood/White,https://www.amazon.com/dp/B07D6TS5MR,,,
+"FurnitureR 27''H Round Drawer 2 Tiers Endtable Nightstand Shelf for Living Room Bedroom Balcony, Easy Assembly end Sofa Side Table with Storage, Green and Brown",https://m.media-amazon.com/images/I/51VXthftc3L._SS522_.jpg,Mid-Century Modern,Engineered Wood,Green and Brown,https://www.amazon.com/dp/B0BVYQTMNX,,,
+Flash Furniture Contemporary Red Vinyl Rounded Orbit-Style Back Adjustable Height Barstool with Chrome Base,https://m.media-amazon.com/images/I/41OOyTZhTzL._SS522_.jpg,Contemporary,,Red,https://www.amazon.com/dp/B00EAY2HTY,,,
+"Stylish Camping Ming's Mark RC4 Reversible Classical Patio Mat - 8' x 20', Green/Beige",https://m.media-amazon.com/images/I/515xhjtnk0L._SS522_.jpg,Modern,Polypropylene,Green/Beige,https://www.amazon.com/dp/B0044G9M2S,,,
+"Christopher Knight Home Adelina Fabric Occaisional Chair, Light Lavendar",https://m.media-amazon.com/images/I/41FESwmeXbL._SS522_.jpg,Wing Back,,Light Lavender,https://www.amazon.com/dp/B073GLR1DG,,,
+"ODK Small Computer Desk, 27.5 inch Desk for Small Spaces with Storage, Compact Table with Monitor & Storage Shelves for Home Office, Modern Style Laptop Desk, Pure White",https://m.media-amazon.com/images/I/41meqsf8aqL._SS522_.jpg,Modern,Engineered Wood,Pure White,https://www.amazon.com/dp/B092HVNQQ4,,,
+GOmaize Cute Wall Mirror with 4 Layers of Colored Petals 14 inchs Wall Hanging Flower-Shaped Mirror Boho Home Decor for Apartment Living Room，Bedroom&Bathroom Ideas （Blue）,https://m.media-amazon.com/images/I/417WwDOB5XL._SS522_.jpg,Bohemian,Plastic,Blue,https://www.amazon.com/dp/B0CB6HZR7Z,,,
+"huester What are You Doing in My Swamp Door Mat 17""x30"" Decorative Home Farmhouse Indoor Outdoor Front Porch Door Mat,Funny Welcome Door Mat Decor,Housewarming Gifts",https://m.media-amazon.com/images/I/51L59TyllJL._SS522_.jpg,Farmhouse,Rubber,,https://www.amazon.com/dp/B0C8SGN73S,,,
+"Bedstory 3 Inch Queen Size Memory Foam Mattress Topper, Extra Firm Pain-Relief Bed Topper High Density, Enhanced Cooling Pad Gel Infused, Non-Slip Removable Skin-Friendly Cover, CertiPUR-US Certified",https://m.media-amazon.com/images/I/516PONoRDrL._SS522_.jpg,,Memory Foam,White,https://www.amazon.com/dp/B0B31DB3LN,,,
+Toland Home Garden 800252 Birthday Bash Party Door Mat 18x30 Inch Balloon Outdoor Doormat for Entryway Indoor Entrance,https://m.media-amazon.com/images/I/51rfyHppFmS._SS522_.jpg,Modern,Rubber,Balloon Outdoor Doormat for Entryway Indoor Entrance,https://www.amazon.com/dp/B01AA0SO7A,,,
+"Asense Small Footstool Ottoman Set of 2, Faux Leather Upholstered Rectangular Footrest with Plastic Legs, Celadon",https://m.media-amazon.com/images/I/31mK9NtBNHL._SS522_.jpg,Modern,,2 Pack Faux Leather Celadon,https://www.amazon.com/dp/B0CPLSTFW5,,,
+"PINGEUI 2 Packs 13 Inches Bamboo Step Stool, Non-Slip Bamboo Small Seat Stool, Durable Bamboo Footrest Bench with Storage Shelf for Bathroom, Bedroom, Kitchen",https://m.media-amazon.com/images/I/41Y0vrrtp7L._SS522_.jpg,Modern,Bamboo,Brown,https://www.amazon.com/dp/B099VZPTWT,,,
+"Poundex Y1553 Two Piece PU Round Shape Barstool Set, Black Adjustable Bar Stool",https://m.media-amazon.com/images/I/31XVd1lG-zL._SS522_.jpg,Modern,,Black,https://www.amazon.com/dp/B0183K9SMO,,,
+"SP-AU-Era Mirror cabinet storage box, cosmetics, lipstick storage rack, bathroom desktop organization box, storage box (blackish green)",https://m.media-amazon.com/images/I/61zDAVHDAfL._SS522_.jpg,"Wall-mounted Perforated Home Bathroom Sink, Cosmetics, Lipstick Storage Rack",PET,blackish green,https://www.amazon.com/dp/B0C99SY5W2,,,
+"Kavonty Storage Chest, Storage Bench, Retro Toy Box Organizer with U-Shaped Cut-Out Pull, 29.5"" L×15.7"" W× 17.7”H, Entryway Storage Bench with 2 Hinges,Supports 300 lb, Easy Assembly, Rustic Brown",https://m.media-amazon.com/images/I/41YpXf+0X2L._SS522_.jpg,,,Rustic Brown,https://www.amazon.com/dp/B0BB9RZ19N,,,
+"Barkan TV Wall Mount, 32-70 inch Full Motion Articulating - 4 Movement Flat/Curved Screen Bracket, Very Low Profile, Holds up to 88 lbs, Lifetime Limited Warranty, UL Listed, Fits LED OLED LCD",https://m.media-amazon.com/images/I/41NgcrmTA7L._SS522_.jpg,,,,https://www.amazon.com/dp/B01L0YHBB0,,,
+"danpinera Side Table Round Metal, Outdoor Side Table Small Sofa End Table Indoor Accent Table Round Metal Coffee Table Waterproof Removable Tray Table for Living Room Bedroom Balcony Office (Green)",https://m.media-amazon.com/images/I/41fuboxDT3L._SS522_.jpg,Modern,Iron,Light Green,https://www.amazon.com/dp/B09FXM34DV,,,
+Dscabomlg Foldable Shoe Storage Plastic Vertical Shoe Rack Shoe Organizer for Closet Narrow Shoe Shelf Plastic-B,https://m.media-amazon.com/images/I/41bq4r8uj5L._SS522_.jpg,Modern,,Grey&white,https://www.amazon.com/dp/B0CG5SJN86,,,
+"ACCHAR Ergonomic Office Chair, Reclining Mesh Chair, Computer Desk Chair, Swivel Rolling Home Task Chair with Padded Armrests, Adjustable Lumbar Support and Headrest (White)",https://m.media-amazon.com/images/I/413qdlao4pL._SS522_.jpg,With arms,Foam,White,https://www.amazon.com/dp/B0C2C9S1R6,,,
+"ODK Small Computer Desk, 27.5 Inch, Compact Tiny Study Desk with Storage and Monitor Stand for Home Office, Small Spaces, Black",https://m.media-amazon.com/images/I/41NmfAngKlL._SS522_.jpg,Modern,Engineered Wood,Black,https://www.amazon.com/dp/B08CB925CT,,,
+"Front Door Mats by ZULINE,Entry and Back Yard Door Mat,Indoor and Outdoor Safe,Slip Resistant Rubber Backing,Absorbent and Waterproof,Dirt Trapping Rugs for Entryway,29.5 x 17 (Brown-Diamond)",https://m.media-amazon.com/images/I/51+qRIvl1FL._SS522_.jpg,Outdoor & Indoor,Rubber,Brown-diamond,https://www.amazon.com/dp/B09PBH963M,,,
+"MyGift Modern Over The Door Towel Rack in Shabby White Washed Solid Wood and 3 Tier Matte Black Metal Bars, Space Saving Bathroom Storage Drying Towels Hanger",https://m.media-amazon.com/images/I/515aoZQHoAL._SS522_.jpg,,Metal,Whitewashed Wood & Black Metal,https://www.amazon.com/dp/B0C5BBYRDN,,,
+"WEENFON Storage Cabinet with Doors and Shelves, Floor Storage Cabinet with Drawer, Accent Cabinet for Living Room, Hallway, Kitchen, Gray",https://m.media-amazon.com/images/I/51F9Edov14L._SS522_.jpg,Shaker,Engineered Wood,Grey,https://www.amazon.com/dp/B0BF8KWBR2,,,
+"SOOWERY End Tables with Charging Station, Set of 2 Side Tables with USB Ports and Outlets, Nightstands with Storage Shelf for Living Room, Bedroom, Brown",https://m.media-amazon.com/images/I/41x2Yzpw5aL._SS522_.jpg,Retro,Iron,Brown,https://www.amazon.com/dp/B0BRFX55TJ,,,
+"Bednowitz Twin Box Spring，5 Inch Low Profile Metal Boxspring with Non-Slip Fabric Bed Cover, Sturdy Heavy Duty Steel Structure Bed Base, Noise-Free Mattress Foundation, Easy Assembly，Black",https://m.media-amazon.com/images/I/51rTEhx3EAL._SS522_.jpg,,,,https://www.amazon.com/dp/B0CJR8KM2D,,,
+BOKKOLIK Industrial Bar Stools (Set of 2) Counter Height Adjustable 24-27.5inch Vintage Kitchen Island Stool Farmhouse Office Guest Chair Swivel Wooden Seat,https://m.media-amazon.com/images/I/41r1PM96rVL._SS522_.jpg,industrial/retro/rustic/vintage/farmhouse/chic,,,https://www.amazon.com/dp/B0BJZPV117,,,
+"HOOBRO Over The Toilet Storage Cabinet, Mass-Storage Over Toilet Bathroom Organizer with Louver Door, X-Shaped Metal Frame, Space-Saving Toilet Rack, Easy Assembly, Rustic Brown BF431TS01",https://m.media-amazon.com/images/I/41i8ryTI4hL._SS522_.jpg,louver,"Engineered Wood, Metal",Rustic Brown,https://www.amazon.com/dp/B0B31G7LBC,,,
+"Hanover Swivel Counter Height Bar Stool, White and Gray",https://m.media-amazon.com/images/I/31039iD-MpL._SS522_.jpg,Classic,,White and Gray,https://www.amazon.com/dp/B0B97PJ94P,,,
+"VECELO Modern Industrial Style 3-Piece Dining Room Kitchen Table and Pu Cushion Chair Sets for Small Space, 2, Retro Brown",https://m.media-amazon.com/images/I/41rj5r2UFSL._SS522_.jpg,,,,https://www.amazon.com/dp/B09MS5RJTT,,,
+"Tenkovic Metal Coat Rack Stand with Quartz Base, Coat Rack Freestanding with 8 Wooden Hooks, Easy to Assemble and Sturdy, Coat Rack Bionic Hall Tree for Home Entry-way Hat Hanger Organizer (Gold)",https://m.media-amazon.com/images/I/31N5mQxbhBL._SS522_.jpg,,"Metal, Wood",tree gold,https://www.amazon.com/dp/B0BZCMCJDY,,,
+"FANYE Oversized 6 Seaters Modular Storage Sectional Sofa Couch for Home Apartment Office Living Room,Free Combination L/U Shaped Corduroy Upholstered Deep Seat Furniture Convertible Sleeper Sofabed",https://m.media-amazon.com/images/I/41MTr4ynO3L._SS522_.jpg,Track,Wood,Navy Blue,https://www.amazon.com/dp/B0CP7YFXD2,,,
+"HOMSHO 2-Tier Storage Bench,Shoe Bench with Padded Seat Cushion, Entryway Bench with 2 Barn Doors,Adjustable Shelf, 27.6"" L x 13.8"" W x 17.7"" H, for Entryway, Living Room, Bedroom,White",https://m.media-amazon.com/images/I/41Sq7pT7XML._SS522_.jpg,,,White,https://www.amazon.com/dp/B0BY23W1J9,,,
+"Realhotan 18 Inch Twin Bed Frame 3500 Pounds Heavy Duty Non-Slip Metal Slats Platform Mattress Foundation, No Box Spring Needed,Easy Assembly,Noise-Free,Black",https://m.media-amazon.com/images/I/51+pTJO13KL._SS522_.jpg,,,Black,https://www.amazon.com/dp/B0CCCS3RB9,,,
+"Kwikset BTBNC1C Pfister Bath Hardware, 18"", Polished Chrome",https://m.media-amazon.com/images/I/31A+awsgcPL._SS522_.jpg,Contemporary,Zinc,Polished Chrome,https://www.amazon.com/dp/B00JMTNK0W,,,
+"MAHANCRIS End Table Set of 2, Side Table with 3-Tier Storage Shelf, Slim Nightstands, Small Bedside Table, Sofa Table for Small Space, Sturdy and Stable, Living Room, Bed Room, Rustic Brown ETHR8501S2",https://m.media-amazon.com/images/I/41wsItqcjUL._SS522_.jpg,Straight Leg,Engineered Wood,Rustic Brown + Black,https://www.amazon.com/dp/B0CJNJMY5H,,,
+"Moen MY3786CH Idora Single Post Bathroom Hand -Towel Ring, Chrome",https://m.media-amazon.com/images/I/41LVA3TodyL._SS522_.jpg,,Zinc,Chrome,https://www.amazon.com/dp/B0882HQRJX,,,
+"Roundhill Furniture Swivel Black Bonded Leather Adjustable Hydraulic Bar Stool, Set of 2",https://m.media-amazon.com/images/I/31VM2JhRDZL._SS522_.jpg,Modern,,Black,https://www.amazon.com/dp/B00D93AT24,,,
+"PINPLUS Storage Ottoman Bench, Linen Coffee Table Ottoman with Tray, Large Storage Bench with Wooden Legs, Toy Chest Foot Stools, Ottomans with Storage for Living Room Bedroom, White, 30"" x 15"" x 15""",https://m.media-amazon.com/images/I/41gj8mVGFGL._SS522_.jpg,Modern,Engineered Wood,White,https://www.amazon.com/dp/B0BZ3RYRNY,,,
+"Red Co. 14 x 18 inch Large Decorative Frameless Beveled Edge Wall Hanging Mirror, Rectangular",https://m.media-amazon.com/images/I/21M6+MAnWpL._SS522_.jpg,Modern,Glass,Silver,https://www.amazon.com/dp/B087Z3RXLN,,,
+PONTMENT Foot Stool Leather Footstool Solid Wood Vintage Foot Rest Faux Leather Ottoman Upholstered Footrest for Living Room/Sofa/Couch.,https://m.media-amazon.com/images/I/51ElPbhgU7L._SS522_.jpg,,,,https://www.amazon.com/dp/B0C38VPJ15,,,
+"Kingston Brass BA2714C Milano Towel-Ring, 6-Inch, Polished Chrome",https://m.media-amazon.com/images/I/41X7yXWQ+PS._SS522_.jpg,Contemporary,Brass,Polished Chrome,https://www.amazon.com/dp/B0003SDM18,,,
+"Lazy Chair with Ottoman, Modern Lounge Accent Chair with Footrest, Pillow and Blanket, Leisure Sofa Chair Reading Chair with Armrests and Side Pocket for Living Room, Bedroom & Small Space, Grey",https://m.media-amazon.com/images/I/415U1ul6gpL._SS522_.jpg,,,Grey,https://www.amazon.com/dp/B0CCRXWDF1,,,
+"latifolia Shoe Cabinet, Vintage Shoe Storage Cabinet with 2 Doors, 4 Tier Bamboo Shoe Organizer Cabinet for Entryway, Closet, Hallway",https://m.media-amazon.com/images/I/41Mst-29ZdL._SS522_.jpg,Modern,Bamboo,Brown,https://www.amazon.com/dp/B0CGX7Y9HQ,,,
+"Jumweo Towel Racks for Bathroom, Metal Towel Rack Wall Mounted for Large Towels, Rolled Bath Towel Holder for Bathroom Wall, Bath Towel Storage Shelf with Wood Shelf for Small Bathroom",https://m.media-amazon.com/images/I/411VfNriJEL._SS522_.jpg,,,,https://www.amazon.com/dp/B0CM6PR2ZB,,,
+"Christopher Knight Home Gentry Bonded Leather Dining Chairs, 2-Pcs Set, Black",https://m.media-amazon.com/images/I/412PrvRCw-L._SS522_.jpg,Leather,Foam,Black,https://www.amazon.com/dp/B005FFA3LQ,,,
+BokWin 4 Sets No Mortise Bed Rail Fittings Wooden Bed Frame Connectors Metal Bed Rail Fasteners,https://m.media-amazon.com/images/I/41ocbpXWJgL._SS522_.jpg,,Iron,,https://www.amazon.com/dp/B09CGPQT1L,,,
+"Simple Deluxe Gaming Chair, Big and Tall Gamer Chair, Racing Style Adjustable Swivel Office Chair, Ergonomic Video Game Chairs with Headrest and Lumbar Support",https://m.media-amazon.com/images/I/41ZTMbqu1JL._SS522_.jpg,With arms,,Black,https://www.amazon.com/dp/B0B51LYB8T,,,
+OIGUMR Shield Wall Mirror Mirror Wall Decor Vintage Mirror (11.3 x 8.5 inch Gold),https://m.media-amazon.com/images/I/41LSP7xb2qL._SS522_.jpg,,Resin,Gold,https://www.amazon.com/dp/B0BMXD3D6J,,,
+"ChooChoo Farmhouse End Table, Modern End Table with Storage Shelf, X-Design Side Table Living Room (White and Brown)",https://m.media-amazon.com/images/I/41P7V9O6gaL._SS522_.jpg,Modern,Engineered Wood,White and Brown,https://www.amazon.com/dp/B0CJHT9KH6,,,
+"ZIYOO Twin Bed Frame 14 Inch High 3 Inches Wide Wood Slats with 2500 Pounds Support, No Box Spring Needed for Foam Mattress,Underbed Storage Space, Easy Assembly, Noise Free",https://m.media-amazon.com/images/I/31dZ6tsbHOL._SS522_.jpg,,,Black,https://www.amazon.com/dp/B07RY46G23,,,
+"MoNiBloom Set of 2 Plastic Barstools with PU Cushion, Height Adjustable White Bar Stools with High Backrest and 360° Swivel, Modern Counter Height Barstool for Kitchen Dining Room and Pub",https://m.media-amazon.com/images/I/31fCq+IIEuL._SS522_.jpg,Modern,,White,https://www.amazon.com/dp/B0CB7SM7MM,,,
+KingCamp Stable Folding Camping Table Bamboo Outdoor Folding Tables Adjustable Height Portable Picnic,https://m.media-amazon.com/images/I/41Sc-GGZBeL._SS522_.jpg,YELLOW,"Aluminum,Bamboo","Yellow-27.6""d X 47.2""w X 27.56""h",https://www.amazon.com/dp/B08ZHPDZX5,,,
+"Artistic Weavers Berma Knitted Jute Round Pouf 14""H x 20""W x 20""D, Slate",https://m.media-amazon.com/images/I/51wZvzlzMDL._SS522_.jpg,Natural,Engineered Wood,Slate,https://www.amazon.com/dp/B00JVZEZE2,,,
+Dwellicity Hello Welcome Mat Black and Gray Striped Coir Mat,https://m.media-amazon.com/images/I/51nGbm-6b-L._SS522_.jpg,Modern,Polyvinyl Chloride,,https://www.amazon.com/dp/B099NTP2SZ,,,
+"Lifewit 70.9"" Narrow Long Console Sofa Table with Metal Frame for Living Room, Industrial Entryway Table for Hallway Entrance Office Corridor Coffee Table Behind Sofa, Easy Assembly, Rustic Brown",https://m.media-amazon.com/images/I/417XCOhUDgL._SS522_.jpg,Modern,Engineered Wood,Rustic Brown,https://www.amazon.com/dp/B0BZYWTH2D,,,
+"Henn&Hart 20"" Wide Round Side Table with Mirror Shelf in Antique Brass, Table for Living Room, Bedroom",https://m.media-amazon.com/images/I/41+Mg7qmpYL._SS522_.jpg,Side Table,Glass,Antique Brass/Mirror,https://www.amazon.com/dp/B07WK22XDX,,,
+"klotski Kids Table and 2 Chair Set, Wood Activity Toddler Table and Chair Set with Storage, Children Table with Non-Slip Legs/Round Edge Design for Activity/Play/Art/Read/Craft",https://m.media-amazon.com/images/I/41QhFgJUCgL._SS522_.jpg,,,,https://www.amazon.com/dp/B0CGZW945K,,,
+"Kraftware Grant Signature Home San Remo Pinecone Round Waste Basket, 10.75"", Brown",https://m.media-amazon.com/images/I/419nFYjmvGL._SS522_.jpg,,Vinyl,Brown,https://www.amazon.com/dp/B0751KYGV4,,,
+"Alise Bath 3 Towel Bars,Towel Holder Towel Racks for Bathroom Wall Mount,Heavy Duty SUS304 Stainless Steel Towel Hanger Towel Rails,GK9603-B Matte Black,25-Inch",https://m.media-amazon.com/images/I/41frWw+ttRL._SS522_.jpg,,"Stainless Steel, Metal",Matte Black,https://www.amazon.com/dp/B0BGN333H4,,,
+"Round Mirror, Black Round Mirror 24 Inch, Round Wall Mirror, Round Bathroom Mirror, Circle Mirrors for Wall, Metal Framed Mirror for Bathroom, Vanity, Bedroom, Entryway, Hallway",https://m.media-amazon.com/images/I/41igYIRb2fL._SS522_.jpg,Modern,Metal,Black,https://www.amazon.com/dp/B08TTKV6LY,,,
+"Gexpusm Wood Coffee Table, Natural Wood Coffee Table, Solid Wood Center Large Coffee Table for Living Room (Octagonal Coffee Table)",https://m.media-amazon.com/images/I/51xwMLJtrtL._SS522_.jpg,4 independent iron legs,Wood,Octagonal Coffee Table,https://www.amazon.com/dp/B0BWXB7C1B,,,
+"Karl home Accent Chair Mid-Century Modern Chair with Pillow Upholstered Lounge Arm Chair with Solid Wood Frame & Soft Cushion for Living Room, Bedroom, Belcony, Beige",https://m.media-amazon.com/images/I/51+a05Mxh+L._SS522_.jpg,Mid-Century Modern,,Beige,https://www.amazon.com/dp/B0BLP4W97Y,,,
+"Kottova Vanity Mirror with Lights,Makeup Mirror Tabletop,Hollywood Mirror with Phone Holder,9 LED Dimmable Bulbs, 3 Color Modes,Touch Control, 360 Rotation,Detachable 10X Magnification Mirror,Black",https://m.media-amazon.com/images/I/41Z2VFHxc2L._SS522_.jpg,,,,https://www.amazon.com/dp/B0BJ1Y5TDN,,,
+"L.R. Resources Corcovado Metallic Braided Pouf Ottoman, 14"" x 20"", Grey/White",https://m.media-amazon.com/images/I/51QokReEa1L._SS522_.jpg,Bohemian,Cotton,Grey / White,https://www.amazon.com/dp/B078YDGBM8,,,
+"GREENSTELL Coat Rack, Wooden Coat Rack Freestanding with Shelf, Coat Tree with 4 Height Options 50.5""-72.6"" for Clothes/Bag/Hats, Hanger Stand for Entryway/Living Room/Bedroom/Office, Black",https://m.media-amazon.com/images/I/31lWN-XSfCL._SS522_.jpg,Rustic,Wood,Black,https://www.amazon.com/dp/B09M8M4P9L,,,
+"COLLECTIVE HOME Mail Organizer with Mirror, Wall Pocket, Wood Wall Hanging Decor, Wooden Mail Holder, Collect Beautiful Moments, Decorative Accent Mirror for Foyer, Bathroom, Bedroom. (Black)",https://m.media-amazon.com/images/I/510ciQYiY4L._SS522_.jpg,Rustic,Black,White,https://www.amazon.com/dp/B0BWY1HPMB,,,
+"Nightstand with Charging Station and LED Light, Side End Table with Glass Door, Modern Bedside Table for Bedroom, Living Room",https://m.media-amazon.com/images/I/41Co0zmXyyL._SS522_.jpg,With power outlet,"Glass, Engineered Wood",Black,https://www.amazon.com/dp/B0C6K8LMG8,,,
+"Kmitmuk 2 Pack Cabinet Towel Holder, White Kitchen Towel Rack Stainless Steel Towel Bar Universal Fit On Cupboard Doors",https://m.media-amazon.com/images/I/21b4+99Ox0L._SS522_.jpg,,,,https://www.amazon.com/dp/B0CGXJ3VR7,,,
+"GIA Toolix Backless Stool with Metal Seat, Gunmetal, 4-Pack",https://m.media-amazon.com/images/I/41mgAYeNExL._SS522_.jpg,Tapered,,Gunmetal,https://www.amazon.com/dp/B01FL46UD0,,,
+"It's_Organized Gaming Desk 55 inch PC Computer Desk, K-Frame Home Office Desk Professional Gamer Workstation with Cup Holder Headphone Hook Gaming Handle Rack Free Mousepad, Black",https://m.media-amazon.com/images/I/41oiXo1q4wL._SS522_.jpg,Gaming,Alloy Steel,Black,https://www.amazon.com/dp/B08CR34X1X,,,
+"Serta Executive Office Padded Arms, Adjustable Ergonomic Gaming Desk Chair with Lumbar Support, Faux Leather and Mesh, Black/Blue",https://m.media-amazon.com/images/I/41ZBS1hvHzL._SS522_.jpg,with-arms,Foam,Black/Blue,https://www.amazon.com/dp/B07644FZVS,,,
+"KoiHome Wooden Daybed with 2 Storage Drawers, Twin Size Platform Bed Frame, Daybed Twin for Kids Boy Girls Bedroom,Living Room, Office, No Box Spring Needed, Twin Size Day Beds Sofas in Espresso",https://m.media-amazon.com/images/I/511irNkgawL._SS522_.jpg,,,Espresso,https://www.amazon.com/dp/B0CHMQC63H,,,
+"Soerreo Shoe Slot Storage Box Adjustable Shoe Rack Save Space Suitable for High Low Heels, Sneakers and Sandals (10 Piece Set)",https://m.media-amazon.com/images/I/4127YVIANkL._SS522_.jpg,Modern,Plastic,10 Piece Set,https://www.amazon.com/dp/B07X5VSLV1,,,
+"Arch Window Wall Mirror for Living Room,White Cathedral Wooden Decorative Arch Windowpane Mirror Large Vintage Farmhouse Mirror",https://m.media-amazon.com/images/I/419YPa-PWhL._SS522_.jpg,French Country,Wood,Black,https://www.amazon.com/dp/B0CJV7SF48,,,
+"Jennifer Taylor Home Jacob 18"" Storage Cube Ottoman, Tan Floral Jacquard",https://m.media-amazon.com/images/I/51KOhS-ZWZL._SS522_.jpg,Contemporary,Engineered Wood,Tan Floral Jacquard,https://www.amazon.com/dp/B0C9FWFGRP,,,
+"C COMFORTLAND Unstuffed Faux Leather Ottoman Pouf, Round Foot Rest Poof Ottomans, Floor Foot Stool Poufs, Bean Bag Cover with Storage for Living Room, Bedroom, Grey (No Filler)",https://m.media-amazon.com/images/I/51qAukZMUDL._SS522_.jpg,Modern,This is an empty shell that you have to stuff.,Grey3,https://www.amazon.com/dp/B0BWY5RPD1,,,
+"ZZQXTC Over Toilet Storage Cabinet, Bathroom Storage Cabinet with Doors and Shelves, Wood Metal Bathroom Space Saver Above Toilet, White",https://m.media-amazon.com/images/I/31cXPz4r76L._SS522_.jpg,wood,Wood,Over the Toilet Storage Cabinet White,https://www.amazon.com/dp/B0CBDLJ32L,,,
+"40ft Upholstery Elastic Webbing,Two Inch (2"") Wide Stretch Latex Band for Furniture Sofa, Couch, Chair Repair Modification (5cm Green)",https://m.media-amazon.com/images/I/51oKu+lxwzL._SS522_.jpg,,,,https://www.amazon.com/dp/B0CG9CDKQ7,,,
+"Kujielan Oval Wall Mirror with Leaf Decorative Frame,Cirle Mirrors for Wall,Modern Wall Decor for Living Room,Bedroom,Bathroom,Vanity,Entryway,Black",https://m.media-amazon.com/images/I/419muJNV1JL._SS522_.jpg,Contemporary,Metal,Black,https://www.amazon.com/dp/B0CF9W6WW2,,,
+"RRG Coat Rack Stand, Metal Coat Tree with Heavy Base 8 Welded Hooks Coat Stand for Entryway, Hall, Bedroom, Corner, Office 67” (Gold)",https://m.media-amazon.com/images/I/214BED2RP6L._SS522_.jpg,8 T-shaped Hooks,Metal,"Gold - T 67""/170cm",https://www.amazon.com/dp/B09VBPNY7P,,,
+"Mirrors for Wall Decor, Golden Hanging Mirror Golden Butterfly Wall Hanging Makeup Mirror with Chain Round Decorative Mirror for Bathroom Bedroom Living Room Housewarming Gift",https://m.media-amazon.com/images/I/31TgX2crLUS._SS522_.jpg,,Iron,Gold,https://www.amazon.com/dp/B097R4M5Y5,,,
+"Mokoze Wavy Mirror Irregular Border 10.24""x6.3"" Makeup Mirror for Wall-Mounted and Dressing Table Mirrors, Room Decor for Living Room Bedrooms and Mirror for Desk (White)",https://m.media-amazon.com/images/I/319OzJXVrxL._SS522_.jpg,,Plastic,White,https://www.amazon.com/dp/B0C9QHJ611,,,
+"(100) 12"" Record Outer Sleeves - Outer Resealable Sleeves - Extreme HD Poly 3 Mil 12 3/4"" x 12 1/2"" | Record Rescue",https://m.media-amazon.com/images/I/41uJZW57cBL._SS522_.jpg,,"Vinyl, Plastic, Polypropylene (PP)",,https://www.amazon.com/dp/B07B8VT4DC,,,
+"Christopher Knight Home Munro Recliner, Navy Blue + Teak",https://m.media-amazon.com/images/I/31exiSJMk8L._SS522_.jpg,Contemporary,Foam,Navy Blue + Teak,https://www.amazon.com/dp/B09DS1VPFS,,,
+"3-Tier Side Table,Narrow End Table with Storage Shelf,Minimalist Bedside Tables Nightstand,Small Bookshelf Bookcase for Spaces,Bathroom Shelve,Display Rack for Bedroom,Living Room,Office,Dorms,2 Pack",https://m.media-amazon.com/images/I/41tzKL1XIPL._SS522_.jpg,Modern,Engineered Wood,White,https://www.amazon.com/dp/B0CP732ZN8,,,
+"DBTHTSK Sofa Latch,Bed Replacement Parts,Heavy Duty Connector Bracket Interlocking Tapered Hardware Accessories Furniture Connector for Furniture, Sofa, Bed (2 Pairs)",https://m.media-amazon.com/images/I/41gQlYHLvcL._SS522_.jpg,,,,https://www.amazon.com/dp/B0C2GQK6ZD,,,
+"Boraam Sonoma Bench, Storm Gray Wire-Brush",https://m.media-amazon.com/images/I/316Y4ewyCLL._SS522_.jpg,,,Storm Gray Wire-brush,https://www.amazon.com/dp/B07T9M8Y88,,,
+"Kwikset BTBCB2Y, Tuscan Bronze",https://m.media-amazon.com/images/I/21lfjygKjaL._SS522_.jpg,Transitional,Metal,Tuscan Bronze,https://www.amazon.com/dp/B001AHXWQ6,,,
+"Ilyapa 2-Tier Gold Metal Record Player Stand with 16 Slot Vinyl Record Holder - Honeycomb Design Turntable Shelf with Record Shelf, Vinyl Record Player Stand",https://m.media-amazon.com/images/I/4107MgspWhL._SS522_.jpg,,,,https://www.amazon.com/dp/B0BT6FF83T,,,
+"GZsenwo (2 Pieces) 3-5/8"" Stainless Steel Replacement Recliner Sofa Mechanism Tension Spring - Long Neck Hook",https://m.media-amazon.com/images/I/41GvGSllzML._SS522_.jpg,,Stainless Steel,2pcs,https://www.amazon.com/dp/B0C6SYFYZN,,,
+"HomePop by Kinfine Fabric Upholstered Round Storage Ottoman - Velvet Button Tufted Ottoman with Removable Lid, Tan Woven",https://m.media-amazon.com/images/I/51x3kXXPgxL._SS522_.jpg,"Glam,Farmhouse,Traditional",Engineered Wood,Tan Woven,https://www.amazon.com/dp/B0BG6BJ3DL,,,
+"EFTILE HOME 2 Foot Stool Handmade Wooden 3 Legs Footrest for Living Room, Bedroom, Nursery, Patio,Hallway, Lounge, Decorative Furniture (16x14x14; Kiwi)",https://m.media-amazon.com/images/I/41-mDeiQw+L._SS522_.jpg,,Wood,Kiwi,https://www.amazon.com/dp/B0CKJ4YZC9,,,
+"Soft Foot Stool Ottoman Footrest Vanity Stool with Storage Shelf, Velvet Multifunctional Modern Padded Shoe Changing Seat Step Stool Pouf for Makeup Room, Living Room, Bathroom, Metal Legs Black",https://m.media-amazon.com/images/I/41oTGNme97L._SS522_.jpg,,Iron,Black,https://www.amazon.com/dp/B0CKW44X29,,,
+"GAOMON Black 4 Drawer Dresser for Bedroom, Wood Chest of Drawers with Metal Legs, Modern Storage Dresser Chest Cabinet Organizer, Large Dresser for Living Room, Hallway, Closet",https://m.media-amazon.com/images/I/41GkzVqoNyL._SS522_.jpg,,,Black,https://www.amazon.com/dp/B0CM1B86CJ,,,
+"Alise 24-Inch Bathroom Lavatory Towel Rack Towel Shelf with 2 Towel Bars Wall Mount Towel Holder,GYT7060-C SUS 304 Stainless Steel Polished Chrome",https://m.media-amazon.com/images/I/51FqMNM3yYL._SS522_.jpg,,,,https://www.amazon.com/dp/B0B6HXHQSW,,,
+"Seventable Nightstand with Charging Station and LED Lights, Modern Design End Side Table with 2 Drawers, Nightstand Open Compartment for Bedroom, Black",https://m.media-amazon.com/images/I/41Wn14U8LlL._SS522_.jpg,Modern,Engineered Wood,Black,https://www.amazon.com/dp/B09DLBNY6W,,,
+"Furinno Coffee Table with Bins, Espresso/Brown & Simplistic End Table, Espresso/Black",https://m.media-amazon.com/images/I/31CY4VJNyxL._SS522_.jpg,Modern,"Beech,Particle Board",Espresso/Brown,https://www.amazon.com/dp/B08C7Y4RB3,,,
+"Mod Made Mid Century Modern Chrome Wire Counter Stool for Bar or Kitchen, Set of 2, Black",https://m.media-amazon.com/images/I/41BxXleMgGL._SS522_.jpg,Straight,,Black Pad,https://www.amazon.com/dp/B09Q1ZHQFR,,,
+"Bloomingville 15 Inches Mango Wood and Metal Organic Shaped Shelf, Matte Black Wall Mirror",https://m.media-amazon.com/images/I/21-b0yTRSNL._SS522_.jpg,Rustic,Metal,Black,https://www.amazon.com/dp/B0CFSPXPYF,,,
+"Gnyonat Accent Chair with Ottoman,Living Room Chairs,Reading Chairs for Bedroom Comfy,Soft Fabric Reading Chair (Blue)",https://m.media-amazon.com/images/I/41Gau9oSdRL._SS522_.jpg,,,Blue,https://www.amazon.com/dp/B0C3TYNRJC,,,
+"SLLFLY Water Bottle Organizer,Stackable Water Bottle Holder for Cabinet Kitchen,Wine Drink Rack for Kitchen Countertop Freezer Pantry Organization and Storage",https://m.media-amazon.com/images/I/51EAJVwOuLL._SS522_.jpg,Clear,,Clear,https://www.amazon.com/dp/B0BZNKKCC3,,,
+"jela Kids Couch Large, Floor Sofa Modular Funiture for Kids Adults, Playhouse Play Set for Toddlers Babies, Modular Foam Play Couch Indoor Outdoor (57""x28""x18"", Charcoal)",https://m.media-amazon.com/images/I/41Zury7vcHL._SS522_.jpg,Padded,Suede,Charcoal,https://www.amazon.com/dp/B0BL9CDX29,,,
+Flexson TV Mount Attachment for Sonos Beam - Black,https://m.media-amazon.com/images/I/31vbAI-UxEL._SS522_.jpg,,,Black,https://www.amazon.com/dp/B07DQ6GPK6,,,
+"Small Collapsible Kids Hamper Fold Office Waste Bins Pop Up Basket Laundry Sorter For Bedroom Closet,Living Room,Camp Pink",https://m.media-amazon.com/images/I/41v-ozvbCqL._SS522_.jpg,现代,Polyester,"11.8""*19.7"" Pink",https://www.amazon.com/dp/B07K2Q2NRC,,,
+"Diyalor 2.6 Gallon Small Trash Can with Handle,Durable Bathroom Wastebasket Garbage Can (Pack of 2, Black)",https://m.media-amazon.com/images/I/219EPmkeeJL._SS522_.jpg,,,,https://www.amazon.com/dp/B09QCMCPYC,,,
+"DAYTOYS C Shaped End Table-Movable Sofa Table with Metal Frames-Height Adjustable Sofa Tables with Storage Basket for Living Room, Bedroom, Bedside (Black)",https://m.media-amazon.com/images/I/41pgntXmHrL._SS522_.jpg,Classic,Wood,Black,https://www.amazon.com/dp/B0C32RWCV7,,,
+"Phantoscope Storage Ottoman Round15 Inch, Velvet Folding Storage Boxes Footrest Foot Ottoman Toy Box, Padded Seat for Dorm Living Room Bedroom, Support 220lbs Coffee",https://m.media-amazon.com/images/I/31V7JNrxMgL._SS522_.jpg,Modern,Engineered Wood,Coffee,https://www.amazon.com/dp/B095HPZ7DD,,,
+Casual Home Night Owl Nightstand with USB Ports-Espresso,https://m.media-amazon.com/images/I/3142Zp+eYuL._SS522_.jpg,Night Owl,"Walnut,Solid Wood,MDF",Espresso,https://www.amazon.com/dp/B019C4PPTU,,,
+"NOVICA 302212 Handmade Wood and Reverse Painted Glass Wall Mounted Mirror, Burgundy and Metallic, Cuzco Snowflake'",https://m.media-amazon.com/images/I/51bKwT153nL._SS522_.jpg,Colonial,"Wood, Glass",Burgundy,https://www.amazon.com/dp/B07N41BVDG,,,
+Toy Storage Basket and Play Mat for Building Bricks - Collapsible Canvas Organizer Bag with Drawstring - For Kids,https://m.media-amazon.com/images/I/61f83XRzygL._SS522_.jpg,,fabric,Grey,https://www.amazon.com/dp/B08PMH8F89,,,
+RICOO SQ4965 No-Gap Wall Mount for Samsung® Q7 Q8 & Q9 Screens Ultra Flat & Tilting TV Bracket for 49 55 & 65 Inch QLED Devices Super Slim Television Support Weight Capacity 110lbs Black,https://m.media-amazon.com/images/I/41VNr1xTfEL._SS522_.jpg,,,black,https://www.amazon.com/dp/B083WKFRRR,,,
+Hosley Wooden Frame Mirror 20 Inch High. Ideal for Weddings Special Occasions and for Wall Decor Home Spa Aromatherapy Reiki. P2,https://m.media-amazon.com/images/I/410lp8RwjvL._SS522_.jpg,Contemporary,Wood,Brown,https://www.amazon.com/dp/B07BQHWWRW,,,
+"BRIAN & DANY Foldable Storage Ottoman Footrest and Seat Cube with Wooden Feet and Lid, Khaki 15” x15” x14.7”",https://m.media-amazon.com/images/I/413YS7nQBnL._SS522_.jpg,Modern,Wood,Khaki,https://www.amazon.com/dp/B08BNDNGHJ,,,
+"ReplacementScrews Bed Frame Rail Screws Compatible with IKEA Part 116894 (HEMNES, BRIMNES, etc) Euro Screws (Pack of 20)",https://m.media-amazon.com/images/I/31-vY+TuWOL._SS522_.jpg,Flat,Metal,Multicolored,https://www.amazon.com/dp/B0CMXYMDH4,,,
+"mDesign Round Metal in-Lay Accent Table with Hairpin Legs - Side/End Table - Decorative Legs, Marble Top - Home Decor Accent Furniture for Living Room, Bedroom - Set of 2 - Soft Brass/Mirror",https://m.media-amazon.com/images/I/413u0H2o1IL._SS522_.jpg,Modern,Steel/Mirror,Soft Brass/Mirror,https://www.amazon.com/dp/B08XPR7662,,,
+"NSFRCLHO Round End Table, Tempered Glass End Table with Metal Frame, Small Coffee Table, Black Sofa Side Table for Living Room, Balcony, Bedroom",https://m.media-amazon.com/images/I/41z8YktAkGL._SS522_.jpg,Classic,Tempered Glass,Black,https://www.amazon.com/dp/B089YWCTN2,,,
+pranovo Metal Sofa Handle Cable Recliner Chair Couch Release Lever Replacement (2 Pack B#),https://m.media-amazon.com/images/I/3144eTNpeEL._SS522_.jpg,,Aluminum,Black,https://www.amazon.com/dp/B00R5VYYIG,,,
+"Stuffed Animal Storage Bean Bag Chair Cover for Kids, 24x24 Inch Velvet Extra Soft Large Storage Bean Bag for Organizing Children Plush Toys Room Decor for Girls(Cover Only)",https://m.media-amazon.com/images/I/41dBlMhHThL._SS522_.jpg,,velvet,Cover Only,https://www.amazon.com/dp/B08JLH2PVH,,,
+"Pinkpum Shoe Ogranizer for Closet, 12 Pack Shoe Storage Boxes Clear Plastic Stackable Shoe Box, Shoe Containers Storage with Lids, Sneaker Container Bin Holder, Clear White",https://m.media-amazon.com/images/I/41huFJxt+FL._SS522_.jpg,,Acrylonitrile Butadiene Styrene,Clear,https://www.amazon.com/dp/B0B6P65LGH,,,
+"BOOSDEN Padded Folding Chair 2 Pack, Foldable Chair with Thick Cushion, Heavy Duty Metal Folding Chair for Outdoor & Indoor & Dining & Party, Red",https://m.media-amazon.com/images/I/41H64LdIQ8L._SS522_.jpg,,,2 Pack Thick Chair | Red,https://www.amazon.com/dp/B0CC4SZBQ9,,,
+"Kingston Brass SCC8247 Edenscape Pedestal Steel Construction Towel-Rack, Brushed Brass 25.75 x 14.44 x 32",https://m.media-amazon.com/images/I/31FOa-k+EtL._SS522_.jpg,Modern,Alloy Steel,Brushed Brass,https://www.amazon.com/dp/B0B5VJNZHL,,,
+"Industrial Rolling Bar 3-Tier Kitchen Serving Cart, Wine Storage Rack, Rustic Pipe Dining Cart with Wheels, Restaurant Wine Glass Holder, 15.7x15.7x33.4inches",https://m.media-amazon.com/images/I/51rjiq645tL._SS522_.jpg,,"Solid Wood,Iron",Brown+black,https://www.amazon.com/dp/B07RGDWW5C,,,
+Chill Sack Bean Bag Chair: Giant 5' Memory Foam Furniture Bean Bag - Big Sofa with Soft Micro Fiber Cover - Lime,https://m.media-amazon.com/images/I/51fQFu92tsL._SS522_.jpg,Furniture Foam,,Microsuede - Lime,https://www.amazon.com/dp/B00P21TM2O,,,
+"Caroline's Treasures BB5130JMAT Day of The Dead Red Flowers Skull Doormat 24x36 Front Door Mat Indoor Outdoor Rugs for Entryway, Non Slip Washable Low Pile, 24H X 36W",https://m.media-amazon.com/images/I/41Q15C0DMDL._SS522_.jpg,Day of the Dead Red Flowers Skull,Rubber,Day of the Dead Red Flowers Skull,https://www.amazon.com/dp/B01MR9GSZE,,,
+"glitzhome Adjustable Bar Stool Set of 2 Swivel Mid-Century Modern PU Leather Counter Dining Chairs with Back, Begin",https://m.media-amazon.com/images/I/51OPfpn9ovL._SS522_.jpg,Mid-Century,,Begin,https://www.amazon.com/dp/B08ZC5CYXG,,,
+Symmons 673TR-STN Identity Wall-Mounted Towel Ring in Satin Nickel,https://m.media-amazon.com/images/I/31cLgr4MIBL._SS522_.jpg,Contemporary,Brass,Satin Nickel,https://www.amazon.com/dp/B01LYD3YB1,,,
+"glitzhome Kitchen Island with Storage Kitchen Cart on Wheels Rolling Kitchen Cart Island Table with Tower Holder Spice Rack Drawer for Dining Room Kitchen, 34.25”H, Red",https://m.media-amazon.com/images/I/51wSfraUuhL._SS522_.jpg,Shaker,"Mdf,Metal,Plastic",Red,https://www.amazon.com/dp/B09D2T4GP4,,,
+"Lipper International Child's Toy Chest, 33.25"" W x 17.75"" D x 24.5"" H, Walnut Finish",https://m.media-amazon.com/images/I/41IWlgQ25-L._SS522_.jpg,,"Engineered Wood, Beechwood, Metal",Walnut Finish,https://www.amazon.com/dp/B005H05TWC,,,
+"dnbss LED Nightstand with Charging Station, Swivel Top Bedside Table with Wheels, Smart Night Stand with Laptop Table Workstation for Bedroom, Modern End Side Table (Black)",https://m.media-amazon.com/images/I/41CANS+MiTL._SS522_.jpg,Modern,Wood,1-black,https://www.amazon.com/dp/B0BNWVLYV1,,,
+"Remote Control Holder,TV Remote Caddy/Box with 5 Compartments,Bedside Table Organizer for Controller,Glasses,makeup brushes,jewelry and Media Player,Pen/Pencil Storage(Orange)",https://m.media-amazon.com/images/I/41p58TdmyoL._SS522_.jpg,,Leather,Orange,https://www.amazon.com/dp/B0C2GZNDXF,,,
+"MoNiBloom Foldable Storage Free Standing Shoes Shelf, Bamboo Multifunctional 4-Tier Shoe Organizer for 16-20 Pairs Entryway, Hallway, Corridor, Natural",https://m.media-amazon.com/images/I/41SpDKbBslL._SS522_.jpg,Modern,Bamboo,,https://www.amazon.com/dp/B09JSR3CYZ,,,
+"Walker Edison Furniture Modern Round Nesting Coffee Accent Table Living Room, Walnut/Gold",https://m.media-amazon.com/images/I/51U3y0LRMeL._SS522_.jpg,Coffee Table,Manufactured Wood,Walnut/Gold,https://www.amazon.com/dp/B072P27BTW,,,
+Way Basics Book Shelf 4 Cubby Storage (Tool-free Assembly),https://m.media-amazon.com/images/I/31eEZQKN+rL._SS522_.jpg,Modern,Recycled Material,,https://www.amazon.com/dp/B071HWKHQL,,,
+"Mind Reader Trash Can and Toilet Brush Set, Bathroom Decor, Swivel Lid, Accessories, 8.75"" W x 11.25"" H, 2 Piece Set, Gray",https://m.media-amazon.com/images/I/31ktspfOC9L._SS522_.jpg,,,,https://www.amazon.com/dp/B0BJ7PQ9XH,,,
+"#4203 Adjustable 1/4"" Threaded Non-Skid Leveling Glides Black Pad 4-Pack",https://m.media-amazon.com/images/I/31Oas3rE7sL._SS522_.jpg,,,,https://www.amazon.com/dp/B01M0S28J1,,,
+Funny Welcome Doormat for Entryway Front Porch Mat Welcome Madafakas Bulldog with Gun Doormat for Front Door Decor Personalized Kitchen Mat with Anti-Slip Rubber Back Novelty Gift Mat(23.7 X 15.9 in),https://m.media-amazon.com/images/I/415x2v3cW5L._SS522_.jpg,Farmhouse,Rubber,"Colorful,Funny,Novelty,Personalized",https://www.amazon.com/dp/B09VFPFBND,,,
+"KINGYES Folding Adjustable Backrest Adirondack Chair, Gray",https://m.media-amazon.com/images/I/41RnRNOgDDL._SS522_.jpg,With arms,,Grey,https://www.amazon.com/dp/B0B2JRSBL3,,,
+"Leick Home 10109-GR Oval Condo/Apartment Coffee Table with Shelf, Smoke Gray",https://m.media-amazon.com/images/I/31hgF2KPIJL._SS522_.jpg,Oval Coffee Table,Wood,Smoke Gray,https://www.amazon.com/dp/B08KLBTL5R,,,
+Carter's by DaVinci Colby 3-Drawer Dresser in Grey,https://m.media-amazon.com/images/I/31eTOoDK36L._SS522_.jpg,,"pine, Wood",Grey,https://www.amazon.com/dp/B071DZG655,,,
+Modway Baronet Button-Tufted Vegan Leather Parsons Dining Chair in Gray,https://m.media-amazon.com/images/I/31Um2-NPw3L._SS522_.jpg,Contemporary,Foam,Grey,https://www.amazon.com/dp/B0BR8NVGDL,,,
+"MOOACE Small Side Table, Round End Table Nightstand with Charging Station, Storage Shelves, 2 Tier Sofa Coffee Beside Accent Table for Living Room, Bedroom, Brown",https://m.media-amazon.com/images/I/419Yb6N5yyL._SS522_.jpg,Modern,Wood,Brown,https://www.amazon.com/dp/B0BGL3QXKR,,,
+BYOOTIQUE Makeup Chair Folding Camping Stool Collapsible Seat Telescoping Stool Height Adjustable Round Vanity Stool Chair Portable Makeup Hairstylist Chair for Makeup Nail Artist Hair Stylist Travel,https://m.media-amazon.com/images/I/511N0PuE9EL._SS522_.jpg,,,,https://www.amazon.com/dp/B0CC4X9SS3,,,
+"nimboo Kids Couch - Modular Kids Play Couch Set, Kids Sofa, Toddler Couch, Toddler Sofa, Kid Couch, Foam Playroom Couch for Kids",https://m.media-amazon.com/images/I/51He1KLeOsL._SS522_.jpg,,High Density Comfort Foam,Rainbow Unicorn,https://www.amazon.com/dp/B0CLC3XWR6,,,
+"LOKKHAN Industrial Bar Table 38.6""-48.4"" Height Adjustable Swivel Round Wood Tabletop 23.7"" Dia, Kitchen Dining Office Coffee Bistro Pub Table",https://m.media-amazon.com/images/I/31uVNZMOnXL._SS522_.jpg,,"Wood Tabletop,Wooden Tabletop",Copper,https://www.amazon.com/dp/B0BVT748HV,,,
+"UTONE Gaming Chair Computer Chair Breathable Fabric Office Chair Cloth with Backrest Desk Chair with Footrest, Lumbar Support Swivel Recliner Task Chair Ergonomic Video Game Chair Height Adjustable",https://m.media-amazon.com/images/I/31dCSKQ14YL._SS522_.jpg,Solid Back,Textile,Pink,https://www.amazon.com/dp/B0CF9F4TQD,,,
+"Lexicon Victoria Saddle Wood Bar Stools (Set of 2), 28.5"" SH, Black Sand",https://m.media-amazon.com/images/I/41CPL03Y-WL._SS522_.jpg,Contemporary,Wood,Black Sand,https://www.amazon.com/dp/B08SLPBC36,,,
+ANZORG Behind Door Hanging Kids Shoes Organizer Closet Shoe Organizer Shoe Rack with 12 Mesh Pockets (12 Pockets),https://m.media-amazon.com/images/I/31qQ2tZPv-L._SS522_.jpg,,Non Woven Fabric,12 Pockets,https://www.amazon.com/dp/B09KN5ZTXC,,,
+"Pipishell Full-Motion TV Wall Mount for Most 37–75 Inch TVs up to 100 lbs, Wall Mount TV Bracket with Dual Articulating Arms, Extension, Swivel, Tilt, Fits 16"" Wood Studs, 600 x 400mm Max VESA, PILF8",https://m.media-amazon.com/images/I/41TkLI3K2-L._SS522_.jpg,,,Black,https://www.amazon.com/dp/B0BN7T57NK,,,
+"Noori Rug Home - Lux Collection Modern Ava Round Ivory Velvet Storage Ottoman - 13x13x15 - Livingroom,Bedroom, Kid's Room, Nursery, Medium",https://m.media-amazon.com/images/I/21Uq9uJEE5L._SS522_.jpg,Glam,Engineered Wood,Ivory/Gold Ava,https://www.amazon.com/dp/B097FC9C27,,,
+Modway Parcel Upholstered Fabric Parsons Dining Side Chair in Beige,https://m.media-amazon.com/images/I/41f8WNXejUL._SS522_.jpg,Modern,Foam,Beige,https://www.amazon.com/dp/B00SMM4H98,,,