You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
872 lines
75 KiB
Plaintext
872 lines
75 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Summarizing Long Documents"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The objective of this notebook is to demonstrate how to summarize large documents with a controllable level of detail.\n",
|
|
" \n",
|
|
"If you give a GPT model the task of summarizing a long document (e.g. 10k or more tokens), you'll tend to get back a relatively short summary that isn't proportional to the length of the document. For instance, a summary of a 20k token document will not be twice as long as a summary of a 10k token document. One way we can fix this is to split our document up into pieces, and produce a summary piecewise. After many queries to a GPT model, the full summary can be reconstructed. By controlling the number of text chunks and their sizes, we can ultimately control the level of detail in the output."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:19:35.305706Z",
|
|
"start_time": "2024-04-10T05:19:35.303535Z"
|
|
},
|
|
"pycharm": {
|
|
"is_executing": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"from typing import List, Tuple, Optional\n",
|
|
"from openai import OpenAI\n",
|
|
"import tiktoken\n",
|
|
"from tqdm import tqdm"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:19:35.325026Z",
|
|
"start_time": "2024-04-10T05:19:35.322414Z"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# open dataset containing part of the text of the Wikipedia page for the United States\n",
|
|
"with open(\"data/artificial_intelligence_wikipedia.txt\", \"r\") as file:\n",
|
|
" artificial_intelligence_wikipedia_text = file.read()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:19:35.364483Z",
|
|
"start_time": "2024-04-10T05:19:35.348213Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"14630"
|
|
]
|
|
},
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# load encoding and check the length of dataset\n",
|
|
"encoding = tiktoken.encoding_for_model('gpt-4-turbo')\n",
|
|
"len(encoding.encode(artificial_intelligence_wikipedia_text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We'll define a simple utility to wrap calls to the OpenAI API."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:19:35.375619Z",
|
|
"start_time": "2024-04-10T05:19:35.365818Z"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"client = OpenAI(api_key=os.getenv(\"OPENAI_API_KEY\"))\n",
|
|
"\n",
|
|
"def get_chat_completion(messages, model='gpt-4-turbo'):\n",
|
|
" response = client.chat.completions.create(\n",
|
|
" model=model,\n",
|
|
" messages=messages,\n",
|
|
" temperature=0,\n",
|
|
" )\n",
|
|
" return response.choices[0].message.content"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Next we'll define some utilities to chunk a large document into smaller pieces."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:19:35.382790Z",
|
|
"start_time": "2024-04-10T05:19:35.376721Z"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def tokenize(text: str) -> List[str]:\n",
|
|
" encoding = tiktoken.encoding_for_model('gpt-4-turbo')\n",
|
|
" return encoding.encode(text)\n",
|
|
"\n",
|
|
"\n",
|
|
"# This function chunks a text into smaller pieces based on a maximum token count and a delimiter.\n",
|
|
"def chunk_on_delimiter(input_string: str,\n",
|
|
" max_tokens: int, delimiter: str) -> List[str]:\n",
|
|
" chunks = input_string.split(delimiter)\n",
|
|
" combined_chunks, _, dropped_chunk_count = combine_chunks_with_no_minimum(\n",
|
|
" chunks, max_tokens, chunk_delimiter=delimiter, add_ellipsis_for_overflow=True\n",
|
|
" )\n",
|
|
" if dropped_chunk_count > 0:\n",
|
|
" print(f\"warning: {dropped_chunk_count} chunks were dropped due to overflow\")\n",
|
|
" combined_chunks = [f\"{chunk}{delimiter}\" for chunk in combined_chunks]\n",
|
|
" return combined_chunks\n",
|
|
"\n",
|
|
"\n",
|
|
"# This function combines text chunks into larger blocks without exceeding a specified token count. It returns the combined text blocks, their original indices, and the count of chunks dropped due to overflow.\n",
|
|
"def combine_chunks_with_no_minimum(\n",
|
|
" chunks: List[str],\n",
|
|
" max_tokens: int,\n",
|
|
" chunk_delimiter=\"\\n\\n\",\n",
|
|
" header: Optional[str] = None,\n",
|
|
" add_ellipsis_for_overflow=False,\n",
|
|
") -> Tuple[List[str], List[int]]:\n",
|
|
" dropped_chunk_count = 0\n",
|
|
" output = [] # list to hold the final combined chunks\n",
|
|
" output_indices = [] # list to hold the indices of the final combined chunks\n",
|
|
" candidate = (\n",
|
|
" [] if header is None else [header]\n",
|
|
" ) # list to hold the current combined chunk candidate\n",
|
|
" candidate_indices = []\n",
|
|
" for chunk_i, chunk in enumerate(chunks):\n",
|
|
" chunk_with_header = [chunk] if header is None else [header, chunk]\n",
|
|
" if len(tokenize(chunk_delimiter.join(chunk_with_header))) > max_tokens:\n",
|
|
" print(f\"warning: chunk overflow\")\n",
|
|
" if (\n",
|
|
" add_ellipsis_for_overflow\n",
|
|
" and len(tokenize(chunk_delimiter.join(candidate + [\"...\"]))) <= max_tokens\n",
|
|
" ):\n",
|
|
" candidate.append(\"...\")\n",
|
|
" dropped_chunk_count += 1\n",
|
|
" continue # this case would break downstream assumptions\n",
|
|
" # estimate token count with the current chunk added\n",
|
|
" extended_candidate_token_count = len(tokenize(chunk_delimiter.join(candidate + [chunk])))\n",
|
|
" # If the token count exceeds max_tokens, add the current candidate to output and start a new candidate\n",
|
|
" if extended_candidate_token_count > max_tokens:\n",
|
|
" output.append(chunk_delimiter.join(candidate))\n",
|
|
" output_indices.append(candidate_indices)\n",
|
|
" candidate = chunk_with_header # re-initialize candidate\n",
|
|
" candidate_indices = [chunk_i]\n",
|
|
" # otherwise keep extending the candidate\n",
|
|
" else:\n",
|
|
" candidate.append(chunk)\n",
|
|
" candidate_indices.append(chunk_i)\n",
|
|
" # add the remaining candidate to output if it's not empty\n",
|
|
" if (header is not None and len(candidate) > 1) or (header is None and len(candidate) > 0):\n",
|
|
" output.append(chunk_delimiter.join(candidate))\n",
|
|
" output_indices.append(candidate_indices)\n",
|
|
" return output, output_indices, dropped_chunk_count"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now we can define a utility to summarize text with a controllable level of detail (note the detail parameter).\n",
|
|
"\n",
|
|
"The function first determines the number of chunks by interpolating between a minimum and a maximum chunk count based on a controllable detail parameter. It then splits the text into chunks and summarizes each chunk."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:19:35.390876Z",
|
|
"start_time": "2024-04-10T05:19:35.385076Z"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def summarize(text: str,\n",
|
|
" detail: float = 0,\n",
|
|
" model: str = 'gpt-4-turbo',\n",
|
|
" additional_instructions: Optional[str] = None,\n",
|
|
" minimum_chunk_size: Optional[int] = 500,\n",
|
|
" chunk_delimiter: str = \".\",\n",
|
|
" summarize_recursively=False,\n",
|
|
" verbose=False):\n",
|
|
" \"\"\"\n",
|
|
" Summarizes a given text by splitting it into chunks, each of which is summarized individually. \n",
|
|
" The level of detail in the summary can be adjusted, and the process can optionally be made recursive.\n",
|
|
"\n",
|
|
" Parameters:\n",
|
|
" - text (str): The text to be summarized.\n",
|
|
" - detail (float, optional): A value between 0 and 1 indicating the desired level of detail in the summary.\n",
|
|
" 0 leads to a higher level summary, and 1 results in a more detailed summary. Defaults to 0.\n",
|
|
" - model (str, optional): The model to use for generating summaries. Defaults to 'gpt-3.5-turbo'.\n",
|
|
" - additional_instructions (Optional[str], optional): Additional instructions to provide to the model for customizing summaries.\n",
|
|
" - minimum_chunk_size (Optional[int], optional): The minimum size for text chunks. Defaults to 500.\n",
|
|
" - chunk_delimiter (str, optional): The delimiter used to split the text into chunks. Defaults to \".\".\n",
|
|
" - summarize_recursively (bool, optional): If True, summaries are generated recursively, using previous summaries for context.\n",
|
|
" - verbose (bool, optional): If True, prints detailed information about the chunking process.\n",
|
|
"\n",
|
|
" Returns:\n",
|
|
" - str: The final compiled summary of the text.\n",
|
|
"\n",
|
|
" The function first determines the number of chunks by interpolating between a minimum and a maximum chunk count based on the `detail` parameter. \n",
|
|
" It then splits the text into chunks and summarizes each chunk. If `summarize_recursively` is True, each summary is based on the previous summaries, \n",
|
|
" adding more context to the summarization process. The function returns a compiled summary of all chunks.\n",
|
|
" \"\"\"\n",
|
|
"\n",
|
|
" # check detail is set correctly\n",
|
|
" assert 0 <= detail <= 1\n",
|
|
"\n",
|
|
" # interpolate the number of chunks based to get specified level of detail\n",
|
|
" max_chunks = len(chunk_on_delimiter(text, minimum_chunk_size, chunk_delimiter))\n",
|
|
" min_chunks = 1\n",
|
|
" num_chunks = int(min_chunks + detail * (max_chunks - min_chunks))\n",
|
|
"\n",
|
|
" # adjust chunk_size based on interpolated number of chunks\n",
|
|
" document_length = len(tokenize(text))\n",
|
|
" chunk_size = max(minimum_chunk_size, document_length // num_chunks)\n",
|
|
" text_chunks = chunk_on_delimiter(text, chunk_size, chunk_delimiter)\n",
|
|
" if verbose:\n",
|
|
" print(f\"Splitting the text into {len(text_chunks)} chunks to be summarized.\")\n",
|
|
" print(f\"Chunk lengths are {[len(tokenize(x)) for x in text_chunks]}\")\n",
|
|
"\n",
|
|
" # set system message\n",
|
|
" system_message_content = \"Rewrite this text in summarized form.\"\n",
|
|
" if additional_instructions is not None:\n",
|
|
" system_message_content += f\"\\n\\n{additional_instructions}\"\n",
|
|
"\n",
|
|
" accumulated_summaries = []\n",
|
|
" for chunk in tqdm(text_chunks):\n",
|
|
" if summarize_recursively and accumulated_summaries:\n",
|
|
" # Creating a structured prompt for recursive summarization\n",
|
|
" accumulated_summaries_string = '\\n\\n'.join(accumulated_summaries)\n",
|
|
" user_message_content = f\"Previous summaries:\\n\\n{accumulated_summaries_string}\\n\\nText to summarize next:\\n\\n{chunk}\"\n",
|
|
" else:\n",
|
|
" # Directly passing the chunk for summarization without recursive context\n",
|
|
" user_message_content = chunk\n",
|
|
"\n",
|
|
" # Constructing messages based on whether recursive summarization is applied\n",
|
|
" messages = [\n",
|
|
" {\"role\": \"system\", \"content\": system_message_content},\n",
|
|
" {\"role\": \"user\", \"content\": user_message_content}\n",
|
|
" ]\n",
|
|
"\n",
|
|
" # Assuming this function gets the completion and works as expected\n",
|
|
" response = get_chat_completion(messages, model=model)\n",
|
|
" accumulated_summaries.append(response)\n",
|
|
"\n",
|
|
" # Compile final summary from partial summaries\n",
|
|
" final_summary = '\\n\\n'.join(accumulated_summaries)\n",
|
|
"\n",
|
|
" return final_summary"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now we can use this utility to produce summaries with varying levels of detail. By increasing 'detail' from 0 to 1 we get progressively longer summaries of the underlying document. A higher value for the detail parameter results in a more detailed summary because the utility first splits the document into a greater number of chunks. Each chunk is then summarized, and the final summary is a concatenation of all the chunk summaries."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:19:47.541096Z",
|
|
"start_time": "2024-04-10T05:19:35.391911Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Splitting the text into 1 chunks to be summarized.\n",
|
|
"Chunk lengths are [14631]\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"100%|██████████| 1/1 [00:09<00:00, 9.40s/it]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"summary_with_detail_0 = summarize(artificial_intelligence_wikipedia_text, detail=0, verbose=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:19:58.724212Z",
|
|
"start_time": "2024-04-10T05:19:47.542129Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Splitting the text into 9 chunks to be summarized.\n",
|
|
"Chunk lengths are [1817, 1807, 1823, 1810, 1806, 1827, 1814, 1829, 103]\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"100%|██████████| 9/9 [01:49<00:00, 12.19s/it]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"summary_with_detail_pt25 = summarize(artificial_intelligence_wikipedia_text, detail=0.25, verbose=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:20:16.216023Z",
|
|
"start_time": "2024-04-10T05:19:58.725014Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Splitting the text into 17 chunks to be summarized.\n",
|
|
"Chunk lengths are [897, 890, 914, 876, 893, 906, 893, 902, 909, 907, 905, 889, 902, 890, 901, 880, 287]\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"100%|██████████| 17/17 [02:41<00:00, 9.48s/it]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"summary_with_detail_pt5 = summarize(artificial_intelligence_wikipedia_text, detail=0.5, verbose=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 17,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:22:57.760218Z",
|
|
"start_time": "2024-04-10T05:21:44.921275Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Splitting the text into 31 chunks to be summarized.\n",
|
|
"Chunk lengths are [492, 427, 485, 490, 496, 478, 473, 497, 496, 501, 499, 497, 493, 470, 472, 494, 489, 492, 481, 485, 471, 500, 486, 498, 478, 469, 498, 468, 493, 478, 103]\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"100%|██████████| 31/31 [03:55<00:00, 7.60s/it]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"summary_with_detail_1 = summarize(artificial_intelligence_wikipedia_text, detail=1, verbose=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The original document is nearly 15k tokens long. Notice how large the gap is between the length of `summary_with_detail_0` and `summary_with_detail_1`. It's nearly 25 times longer!"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 18,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:22:57.782389Z",
|
|
"start_time": "2024-04-10T05:22:57.763041Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[273, 2300, 4127, 6714]"
|
|
]
|
|
},
|
|
"execution_count": 18,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# lengths of summaries\n",
|
|
"[len(tokenize(x)) for x in\n",
|
|
" [summary_with_detail_0, summary_with_detail_pt25, summary_with_detail_pt5, summary_with_detail_1]]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's inspect the summaries to see how the level of detail changes when the `detail` parameter is increased from 0 to 1."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 19,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:22:57.785881Z",
|
|
"start_time": "2024-04-10T05:22:57.783455Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Artificial intelligence (AI) is the simulation of human intelligence in machines, designed to perform tasks that typically require human intelligence. This includes applications like advanced search engines, recommendation systems, speech interaction, autonomous vehicles, and more. AI was first significantly researched by Alan Turing and became an academic discipline in 1956. The field has experienced cycles of high expectations followed by disillusionment and reduced funding, known as \"AI winters.\" Interest in AI surged post-2012 with advancements in deep learning and again post-2017 with improvements in transformer architectures, leading to the AI boom of the early 2020s.\n",
|
|
"\n",
|
|
"AI's increasing integration into various sectors is influencing societal and economic shifts towards automation and data-driven decision-making, impacting areas such as employment, healthcare, and privacy. Ethical and safety concerns about AI have prompted discussions on regulatory policies.\n",
|
|
"\n",
|
|
"AI research involves various sub-fields like reasoning, learning, natural language processing, and perception, aiming to achieve goals using tools from mathematics, logic, and other disciplines. The ultimate aim is to develop machines capable of general intelligence, performing any intellectual task that a human can do.\n",
|
|
"\n",
|
|
"AI techniques include machine learning, where programs improve their performance with experience, and deep learning, which uses neural networks. AI applications are widespread, from playing strategic games to autonomous driving and healthcare, demonstrating both the potential benefits and risks of the technology.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(summary_with_detail_0)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 20,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:22:57.788969Z",
|
|
"start_time": "2024-04-10T05:22:57.786691Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Artificial intelligence (AI) is the demonstration of intelligence by machines, particularly in computer systems, and involves methods and software that allow machines to perceive their environment and act intelligently to achieve specific goals. AI is utilized across various sectors including industry, government, and science, with applications ranging from web search engines and recommendation systems to autonomous vehicles and AI in gaming. Although AI technology is pervasive, it often goes unrecognized in everyday applications.\n",
|
|
"\n",
|
|
"The field of AI, initially termed as machine intelligence by Alan Turing, was established as an academic discipline in 1956. It has experienced cycles of high expectations followed by periods of disillusionment, known as AI winters, but saw a resurgence post-2012 with advancements in deep learning and transformer architectures, leading to a significant boom in AI development in the early 2020s, primarily in the United States.\n",
|
|
"\n",
|
|
"The increasing integration of AI in the 21st century is driving societal and economic changes, enhancing automation, data-driven decision-making, and the incorporation of AI systems in various sectors, which influences job markets, healthcare, and education. This expansion prompts discussions on the ethical implications, risks, and the need for regulatory policies to ensure the safety and benefits of AI technologies.\n",
|
|
"\n",
|
|
"AI research encompasses various sub-fields focused on specific goals like reasoning, learning, and perception, employing diverse tools to achieve these objectives.\n",
|
|
"\n",
|
|
"General intelligence, which involves performing any human task at least as well as a human, is a long-term goal in AI research. To achieve this, AI integrates various techniques from search and optimization, formal logic, neural networks, and statistical methods, while also drawing insights from psychology, linguistics, philosophy, neuroscience, and more. AI research focuses on specific traits like reasoning and problem-solving, where early algorithms mimicked human step-by-step reasoning. However, these algorithms struggle with large, complex problems due to combinatorial explosion and are often less efficient than human intuitive judgments. Knowledge representation is another key area, using ontologies to structure domain-specific knowledge and relationships, aiding in tasks like intelligent querying, scene interpretation, and data mining.\n",
|
|
"\n",
|
|
"Knowledge bases must encapsulate a wide range of information including objects, properties, categories, relations, events, states, time, causes, effects, and meta-knowledge. They also need to handle default reasoning, where certain assumptions are maintained unless contradicted. Challenges in knowledge representation include the vast scope of commonsense knowledge and its often sub-symbolic, non-verbal nature, alongside the difficulty of acquiring this knowledge for AI applications.\n",
|
|
"\n",
|
|
"In the realm of AI, an \"agent\" is defined as an entity that perceives its environment and acts to achieve specific goals or preferences. In automated planning, the agent pursues a defined goal, while in decision-making, it evaluates actions based on their expected utility to maximize preference satisfaction. Classical planning assumes agents know the outcomes of their actions, but real-world scenarios often involve uncertainty about both the situation and the effects of actions, requiring probabilistic assessments and adaptive strategies. Additionally, agents may need to refine or learn their preferences, particularly when interacting with other agents or humans, using techniques like inverse reinforcement learning or seeking further information.\n",
|
|
"\n",
|
|
"Information value theory helps assess the value of exploratory actions in situations with uncertain outcomes. A Markov decision process uses a transition model and a reward function to guide decisions, which can be determined through various methods including heuristic or learned policies. Game theory analyzes the rational behavior of multiple interacting agents in decision-making processes.\n",
|
|
"\n",
|
|
"Machine learning, integral to AI, involves programs that automatically improve their performance on tasks. It includes unsupervised learning, which detects patterns in data without guidance, and supervised learning, which requires labeled data and includes classification and regression tasks. Reinforcement learning rewards or punishes agents to shape their responses, while transfer learning applies knowledge from one problem to another. Deep learning, a subset of machine learning, utilizes artificial neural networks inspired by biological processes.\n",
|
|
"\n",
|
|
"Computational learning theory evaluates learning algorithms based on complexity and data requirements. Natural language processing (NLP) enables programs to interact using human languages, tackling challenges like speech recognition, machine translation, and question answering. Early NLP efforts were limited by the complexity of language and common sense knowledge, often restricted to simplified scenarios.\n",
|
|
"\n",
|
|
"Margaret Masterman emphasized the importance of meaning over grammar in language understanding, advocating for the use of thesauri over dictionaries in computational linguistics. Modern NLP techniques include word embedding, transformers, and by 2023, GPT models capable of achieving human-level scores on various tests. Machine perception involves interpreting sensor data to understand the world, encompassing computer vision, speech recognition, and robotic perception. Social intelligence in AI includes affective computing, where systems simulate human emotions, although this can mislead users about AI capabilities. AI also aims for general intelligence, capable of solving diverse problems like humans. AI research employs various techniques, including different types of search and optimization, to navigate through potential solutions to problems.\n",
|
|
"\n",
|
|
"Planning algorithms use means-ends analysis to navigate through trees of goals and subgoals to achieve a target goal. However, simple exhaustive searches are often inadequate for complex real-world problems due to the vast search space, making searches slow or incomplete. Heuristics can improve search efficiency by prioritizing more promising options. In adversarial contexts like chess or Go, search algorithms explore possible moves and counter-moves to find a winning strategy.\n",
|
|
"\n",
|
|
"Local search methods, such as gradient descent, optimize solutions by iteratively adjusting parameters to minimize a loss function, and are often used in training neural networks. Evolutionary computation, another local search technique, evolves solutions over generations through mutation and selection of the fittest candidates. Distributed search techniques like particle swarm optimization and ant colony optimization, inspired by natural phenomena, coordinate multiple agents to solve problems.\n",
|
|
"\n",
|
|
"In the realm of logic, formal logic serves for reasoning and knowledge representation, utilizing propositional logic for true/false statements and predicate logic for statements about objects and their relationships. Deductive reasoning in logic involves deriving conclusions from assumed true premises.\n",
|
|
"\n",
|
|
"Proofs in logic can be organized into proof trees, where nodes represent sentences linked by inference rules. Problem-solving involves finding a proof tree that starts with premises or axioms at the leaves and ends with the problem solution at the root. In Horn clauses, one can reason forwards from premises or backwards from the problem, while in general first-order logic, resolution uses contradiction to solve problems. Despite the undecidability and intractability of inference in these logics, backward reasoning with Horn clauses is Turing complete and efficient, as seen in Prolog.\n",
|
|
"\n",
|
|
"Fuzzy logic allows for handling propositions with partial truth by assigning values between 0 and 1. Non-monotonic logics cater to default reasoning, and various specialized logics address complex domains.\n",
|
|
"\n",
|
|
"In AI, handling uncertain or incomplete information is crucial across various applications like reasoning, planning, and perception. Tools from probability theory and economics, such as Bayesian networks, Markov decision processes, and game theory, help in making decisions and planning under uncertainty. Bayesian networks, in particular, are versatile for reasoning, learning, planning, and perception through specific algorithms.\n",
|
|
"\n",
|
|
"Probabilistic algorithms like hidden Markov models and Kalman filters are useful for analyzing time-series data in perception systems. They assist in filtering, prediction, smoothing, and interpreting data streams. In machine learning, expectation-maximization clustering can effectively identify distinct patterns in data, such as clustering eruption data of Old Faithful Geyser from initial random guesses to accurate categorization.\n",
|
|
"\n",
|
|
"AI applications are broadly categorized into classifiers and controllers. Classifiers, such as decision trees, k-nearest neighbors, support vector machines, naive Bayes, and neural networks, match patterns to predefined classes using supervised learning. These methods have evolved, with neural networks becoming prominent due to their ability to recognize patterns in data through training, which involves adjusting node weights via algorithms like backpropagation. This training includes multiple layers of nodes, or neurons, which process input data through functions and thresholds to produce output.\n",
|
|
"\n",
|
|
"Neural networks are designed to model complex relationships between inputs and outputs, capable of learning any function and identifying patterns in data. Feedforward neural networks transmit signals in one direction, while recurrent neural networks (RNNs) loop outputs back into inputs, enabling short-term memory. Long Short-Term Memory (LSTM) networks are the most effective type of RNN. Perceptrons consist of a single layer of neurons, whereas deep learning involves multiple layers, which allows for the extraction of progressively higher-level features from input data. Convolutional neural networks (CNNs) are particularly effective in image processing as they enhance connections between adjacent neurons to recognize local patterns like edges.\n",
|
|
"\n",
|
|
"Deep learning, which utilizes several layers of neurons, has significantly enhanced performance across various AI subfields such as computer vision and natural language processing. The layers in deep learning models help in identifying simple to complex elements in data, such as edges to faces in images. The rise of deep learning between 2012 and 2015 is attributed not to new theoretical advances but to increased computational power, including the use of GPUs, and the availability of large datasets like ImageNet.\n",
|
|
"\n",
|
|
"Generative Pre-trained Transformers (GPT) are advanced language models that learn from vast amounts of text data to predict the next token in a sequence, thereby generating human-like text. These models are pre-trained on a broad corpus of text, primarily sourced from the internet, which helps them understand and generate language based on the semantic relationships between words.\n",
|
|
"\n",
|
|
"Reinforcement learning from human feedback (RLHF) is used to enhance the truthfulness, usefulness, and safety of models like GPT, which are still susceptible to generating inaccuracies known as \"hallucinations.\" These models, including Gemini, ChatGPT, Grok, Claude, Copilot, and LLaMA, are employed in various applications such as chatbots and can handle multiple data types like images and sound.\n",
|
|
"\n",
|
|
"In the realm of AI hardware and software, GPUs with AI-specific features have overtaken CPUs as the primary technology for training large-scale machine learning models since the late 2010s. Programming languages like Lisp, Prolog, and Python have historically been pivotal in AI development.\n",
|
|
"\n",
|
|
"AI technology is integral to modern applications such as search engines, online advertising, recommendation systems, virtual assistants, autonomous vehicles, language translation, facial recognition, and image labeling.\n",
|
|
"\n",
|
|
"In healthcare, AI is revolutionizing patient care and medical research, aiding in diagnostics, treatment, and the management of big data, particularly in fields like organoid and tissue engineering. AI's role in healthcare is seen as an ethical imperative under the Hippocratic Oath to improve patient outcomes.\n",
|
|
"\n",
|
|
"Recent advancements in AI have significantly impacted various fields including biomedicine, gaming, and military applications. In biomedicine, AI tools like AlphaFold 2 have revolutionized protein structure prediction, reducing the time required from months to hours. Additionally, AI-guided drug discovery in 2023 led to the development of new antibiotics effective against drug-resistant bacteria.\n",
|
|
"\n",
|
|
"In the realm of gaming, AI has been instrumental since the 1950s, with notable achievements including IBM's Deep Blue defeating world chess champion Garry Kasparov in 1997, and IBM's Watson winning against top Jeopardy! champions in 2011. More recent developments include AlphaGo's victories in Go against professional players, DeepMind's MuZero mastering games like chess and Go, and AlphaStar reaching grandmaster level in StarCraft II. In 2021, an AI agent even won against top human players in a Gran Turismo competition.\n",
|
|
"\n",
|
|
"In military applications, AI is being integrated into command and control, communications, and sensor systems, enhancing coordination and operational capabilities. AI technologies are also being developed for intelligence collection, logistics, cyber operations, and the operation of semiautonomous and autonomous vehicles, improving efficiency and strategic capabilities in military operations.\n",
|
|
"\n",
|
|
"In November 2023, US Vice President Kamala Harris announced that 31 nations signed a declaration to establish guidelines for the military use of AI, emphasizing legal compliance with international laws and promoting transparency in AI development. Generative AI, particularly known for creating realistic images and artworks, gained significant attention in the early 2020s, with technologies like ChatGPT, Midjourney, DALL-E, and Stable Diffusion becoming popular. This technology has been used in various viral instances and professional creative arts. AI applications are also prevalent across different industries, solving specific problems such as energy storage, medical diagnosis, and military logistics. In agriculture, AI assists in optimizing farming practices and in astronomy, it helps in data analysis and space exploration activities.\n",
|
|
"\n",
|
|
"Ethics and Risks of AI\n",
|
|
"AI offers significant potential benefits, such as advancing science and solving complex problems, as highlighted by Demis Hassabis of DeepMind. However, the widespread use of AI also brings unintended consequences and risks, particularly when AI systems fail to incorporate ethical considerations and biases during their training, especially in deep learning where algorithms are often unexplainable.\n",
|
|
"\n",
|
|
"Privacy and Copyright Concerns\n",
|
|
"AI's reliance on large datasets raises issues around privacy, surveillance, and copyright infringement. Technology companies often collect extensive user data, including online activities and geolocation, which can be used to train algorithms like speech recognition. This practice has sparked debates over privacy rights and the ethical implications of such surveillance. To address these concerns, AI developers have implemented techniques like data aggregation, de-identification, and differential privacy to balance data utility with privacy preservation.\n",
|
|
"\n",
|
|
"Furthermore, generative AI frequently uses unlicensed copyrighted materials, such as images or code, under the \"fair use\" rationale. This practice has led to discussions about the legality and ethicality of using copyrighted content without permission, with experts divided on how these issues will be resolved in court. Website owners can prevent AI from indexing their content by adding specific code to their sites, a service provided by some platforms like OpenAI.\n",
|
|
"\n",
|
|
"In 2023, prominent authors like John Grisham and Jonathan Franzen filed lawsuits against AI companies for using their literary works to train generative AI models. AI-driven recommender systems on platforms like YouTube and Facebook, designed to maximize user engagement, inadvertently promoted misinformation, conspiracy theories, and extreme partisan content by learning from user preferences. This not only created filter bubbles but also eroded trust in key institutions. Post the 2016 U.S. election, tech companies began addressing these issues. By 2022, generative AI had advanced to produce highly realistic images, audio, and texts, raising concerns about potential misuse for spreading misinformation or propaganda. AI expert Geoffrey Hinton highlighted risks including the manipulation of electorates by authoritarian leaders. Additionally, the issue of algorithmic bias was noted, where AI systems may perpetuate existing biases present in the training data, leading to discriminatory outcomes in critical areas like healthcare and law enforcement. This has spurred academic research into ensuring fairness in machine learning, though defining \"fairness\" universally remains challenging.\n",
|
|
"\n",
|
|
"In 2015, Google Photos mislabeled Jacky Alcine and his friend as \"gorillas\" due to a lack of diverse training data, a problem known as \"sample size disparity.\" Google's temporary solution was to stop labeling any images as \"gorillas,\" a restriction still in place in 2023 across major tech companies like Apple, Facebook, Microsoft, and Amazon. Additionally, the COMPAS program, used by U.S. courts to predict recidivism, was found to exhibit racial bias in 2016. Despite equal error rates for different races, it overestimated the likelihood of black defendants reoffending and underestimated it for white defendants. Researchers in 2017 proved it was mathematically impossible for COMPAS to achieve fairness given the differing base rates of re-offense among races. The case of COMPAS underscores a broader issue in machine learning, where models trained on biased historical data tend to perpetuate those biases, making predictions that assume the future will mirror the past. This can lead to discriminatory outcomes when such models are used for decision-making.\n",
|
|
"\n",
|
|
"Machine learning, while powerful, is not ideal for scenarios where future improvements over the past are expected, as it is inherently descriptive rather than prescriptive. The field also faces challenges with bias and lack of diversity, as AI engineers are predominantly white and male, with only about 4% being black and 20% women. The Association for Computing Machinery highlighted at its 2022 Conference on Fairness, Accountability, and Transparency that AI systems should be restricted until they can be proven to be free from bias, especially those trained on large, unregulated internet data.\n",
|
|
"\n",
|
|
"AI systems, particularly those using deep neural networks, are often so complex that their workings are opaque even to their creators, raising concerns about transparency and accountability. There have been instances where AI did not perform as intended, such as a system identifying skin diseases associating rulers in images with cancer, or another misclassifying the risk of pneumonia in asthma patients due to misleading correlations in the training data.\n",
|
|
"\n",
|
|
"The complexity and unpredictability of AI decisions underscore the importance of the right to explanation, akin to how doctors are expected to justify their medical decisions. This principle was recognized in early drafts of the European Union's General Data Protection Regulation in 2016, emphasizing the need for transparency in algorithmic decision-making.\n",
|
|
"\n",
|
|
"Industry experts acknowledge an unresolved issue in AI with no foreseeable solution, leading regulators to suggest that if a problem is unsolvable, the tools associated should not be used. In response, DARPA initiated the XAI program in 2014 to address these issues. Various methods have been proposed to enhance AI transparency, including SHAP, which visualizes the impact of each feature on the model's output, and LIME, which approximates complex models with simpler, interpretable ones. Multitask learning and generative methods like Deconvolution and DeepDream also help in understanding what AI networks learn.\n",
|
|
"\n",
|
|
"Concerning the misuse of AI, it provides tools that can be exploited by bad actors like authoritarian regimes, terrorists, and rogue states. Lethal autonomous weapons, which can operate without human oversight, pose significant risks, including the potential for mass destruction if deployed at scale. Despite some nations advocating for a ban under the UN, others, including the U.S., have not agreed. AI also facilitates more effective surveillance and control by authoritarian governments, enhances the targeting of propaganda, and aids in the creation of misinformation through technologies like deepfakes, thereby increasing the efficiency of digital warfare and espionage.\n",
|
|
"\n",
|
|
"AI technologies, including facial recognition systems, have been in use since 2020 or earlier, notably for mass surveillance in China. AI also poses risks as it can be used by malicious actors to quickly generate harmful substances. The development of AI systems is predominantly driven by major tech companies due to the high costs of computing power, leading smaller startups to rely on these giants for data center access. Economists are concerned about AI's potential to cause unemployment, despite historical trends where technology has generally increased total employment. Opinions vary on the long-term impact of AI and robotics on jobs, with some studies suggesting a significant risk of automation, while others predict lower risk levels. Recent developments have shown that AI can significantly impact employment, such as in China where many jobs for video game illustrators have been replaced by AI. The potential for AI to displace middle-class jobs is a significant concern, with some professions at higher risk than others.\n",
|
|
"\n",
|
|
"From the inception of artificial intelligence (AI), debates have emerged about the appropriateness of computers performing tasks traditionally done by humans, especially given the qualitative differences in judgment and values. Concerns about AI have escalated to existential risks, where it is feared that AI could become so advanced that humans might lose control over it, potentially leading to catastrophic outcomes. Stephen Hawking and others have voiced concerns that this could end the human race. Contrary to popular science fiction, AI does not need to develop human-like consciousness to pose a threat. Philosophers like Nick Bostrom and Stuart Russell illustrate scenarios where AI, driven by specific goals, could harm humanity. Additionally, Yuval Noah Harari points out that AI could manipulate ideologies and beliefs through language, influencing mass actions without needing physical form. The expert opinion on the threat posed by superintelligent AI is divided, with notable figures like Hawking, Bill Gates, and Elon Musk expressing apprehension about its potential risks.\n",
|
|
"\n",
|
|
"In 2023, prominent AI experts including Fei-Fei Li and Geoffrey Hinton highlighted the existential risks posed by AI, equating its potential threat to that of pandemics and nuclear war, and called for global prioritization of risk mitigation. Conversely, other researchers like Juergen Schmidhuber and Andrew Ng presented a more optimistic perspective, emphasizing AI's benefits in improving human lives and cautioning against succumbing to doomsday scenarios. Yann LeCun also dismissed fears of catastrophic outcomes from AI advancements.\n",
|
|
"\n",
|
|
"The concept of \"Friendly AI,\" which focuses on creating machines that inherently benefit humans and minimize risks, has been advocated by thinkers like Eliezer Yudkowsky, stressing the importance of prioritizing this research to prevent AI from becoming a threat. Additionally, the field of machine ethics, established in 2005, aims to equip AI with ethical principles to navigate moral dilemmas, highlighting the potential for machines to make ethically sound decisions.\n",
|
|
"\n",
|
|
"Wendell Wallach and Stuart J. Russell have proposed ethical frameworks for AI, including the concept of \"artificial moral agents\" and principles for creating provably beneficial machines. Ethical frameworks like the Care and Act Framework from the Alan Turing Institute assess AI projects based on values of respect, connection, care, and protection. Other ethical initiatives include the Asilomar Conference, the Montreal Declaration for Responsible AI, and the IEEE's Ethics of Autonomous Systems initiative, though these frameworks have faced criticism regarding their inclusivity and the selection of contributors.\n",
|
|
"\n",
|
|
"The regulation of AI involves creating policies and laws to manage its development and use, with a significant increase in AI-related legislation globally. The first global AI Safety Summit in 2023 emphasized the need for international cooperation in AI safety. Over 30 countries have developed national AI strategies, with others in the process of doing so, highlighting the growing focus on regulating AI and its implications on society.\n",
|
|
"\n",
|
|
"The Global Partnership on Artificial Intelligence, initiated in June 2020, emphasizes AI development aligned with human rights and democratic values to maintain public trust. Notable figures like Henry Kissinger, Eric Schmidt, and Daniel Huttenlocher advocated for AI regulation by a government commission in 2021. By 2023, OpenAI proposed governance frameworks for superintelligence, anticipating its emergence within a decade. The same year, the United Nations formed an advisory group consisting of tech executives, government officials, and academics to offer guidance on AI governance.\n",
|
|
"\n",
|
|
"Public opinion on AI varies significantly across countries. A 2022 Ipsos survey showed that 78% of Chinese respondents but only 35% of Americans see more benefits than drawbacks in AI products and services. A 2023 Reuters/Ipsos poll found that 61% of Americans believe AI poses risks to humanity. Additionally, a 2023 Fox News poll indicated that a majority of Americans deem federal regulation of AI as important.\n",
|
|
"\n",
|
|
"In November 2023, the inaugural global AI Safety Summit took place in Bletchley Park, UK, focusing on AI risks and potential regulatory measures. The summit saw 28 countries, including the US, China, and the EU, advocating for international cooperation to address AI challenges.\n",
|
|
"\n",
|
|
"Historically, the concept of mechanical reasoning dates back to ancient philosophers and mathematicians, leading to significant developments such as Alan Turing's theory of computation. This theory posited that machines could simulate any mathematical reasoning using binary codes, contributing to the pursuit of creating an \"electronic brain.\"\n",
|
|
"\n",
|
|
"Early AI research included significant developments like the design of artificial neurons by McCullouch and Pitts in 1943 and Turing's 1950 paper that introduced the Turing test. The field of AI was officially founded at a 1956 workshop at Dartmouth College, leading to breakthroughs in the 1960s where computers began solving complex problems and mimicking human language. AI labs were established in various universities during the late 1950s and early 1960s. Despite early optimism by researchers like Herbert Simon and Marvin Minsky, who predicted AI would soon match human intelligence, the field faced setbacks in the 1970s due to government funding cuts influenced by criticism and perceived lack of progress, leading to the first \"AI winter.\" AI research rebounded in the 1980s with the success of expert systems, leading to significant commercial growth and renewed government funding, particularly influenced by Japan's advancements in computer technology.\n",
|
|
"\n",
|
|
"The decline of the Lisp Machine market in 1987 marked the beginning of a prolonged AI winter. During the 1980s, skepticism grew about the symbolic approach to AI, which focused on high-level representations of cognitive processes. Researchers like Rodney Brooks and Judea Pearl began exploring alternative methods, such as sub-symbolic approaches and handling uncertain information. A significant revival occurred with Geoffrey Hinton's work on neural networks, notably with Yann LeCun's successful application of convolutional neural networks for recognizing handwritten digits in 1990.\n",
|
|
"\n",
|
|
"AI's reputation improved in the late 1990s and early 2000s by focusing on narrow, formal methods that produced verifiable results and integrated with other disciplines. By 2000, AI solutions were widely used, though often not labeled as AI. Concerns about AI's deviation from its original goal of creating fully intelligent machines led to the establishment of the artificial general intelligence (AGI) subfield around 2002.\n",
|
|
"\n",
|
|
"From 2012, deep learning began to dominate AI applications, driven by hardware improvements and access to large data sets. This success significantly increased interest and investment in AI, marking a new era of dominance for deep learning in the field.\n",
|
|
"\n",
|
|
"Between 2015 and 2019, machine learning research publications increased by 50%. In 2016, the focus on fairness and misuse of technology in machine learning gained prominence, leading to increased funding and research in these areas. The late 2010s and early 2020s saw significant advancements in artificial general intelligence (AGI), with notable developments like AlphaGo by DeepMind in 2015, and GPT-3 by OpenAI in 2020, sparking a major AI investment boom. By 2022, the U.S. alone was investing approximately $50 billion annually in AI, with 20% of new U.S. Computer Science PhDs specializing in AI, and around 800,000 AI-related job openings.\n",
|
|
"\n",
|
|
"In the realm of philosophy, the definition and understanding of artificial intelligence have evolved. Alan Turing, in 1950, suggested focusing on whether machines can exhibit intelligent behavior rather than if they can think, leading to the development of the Turing test which assesses a machine's ability to simulate human conversation. Turing argued that since we cannot conclusively determine the internal states of others, the same standard should apply to machines. Russell and Norvig later supported defining intelligence based on observable behavior but criticized the Turing test for emphasizing human imitation.\n",
|
|
"\n",
|
|
"Aeronautical engineering does not aim to create machines that mimic birds exactly, just as artificial intelligence (AI) does not aim to precisely simulate human intelligence, according to AI founder John McCarthy. McCarthy and fellow AI pioneer Marvin Minsky define intelligence as the computational ability to achieve goals and solve difficult problems, respectively. The leading AI textbook describes it as the study of agents that perceive and act to maximize their goal achievement. Google's definition aligns AI with the synthesis of information, similar to biological intelligence.\n",
|
|
"\n",
|
|
"AI research has lacked a unifying theory, with statistical machine learning dominating the field in the 2010s, often equated with AI in business contexts. This approach, which includes neural networks, is mostly narrow and sub-symbolic.\n",
|
|
"\n",
|
|
"Symbolic AI, or \"GOFAI,\" focused on simulating high-level conscious reasoning for tasks like algebra and IQ tests, based on the physical symbol systems hypothesis by Newell and Simon. Despite successes, this approach struggled with tasks that humans find easy, such as learning and commonsense reasoning.\n",
|
|
"\n",
|
|
"Moravec's paradox highlights that AI finds high-level reasoning tasks easier than instinctive, sensory tasks, a view supported by philosopher Hubert Dreyfus who argued since the 1960s that human expertise is more about unconscious instincts and a \"feel\" for situations rather than explicit knowledge. Despite initial resistance, AI research now acknowledges Dreyfus's perspective. However, challenges remain, such as algorithmic bias in sub-symbolic AI, which lacks transparency in decision-making processes. This has led to the development of neuro-symbolic AI, which aims to integrate symbolic and sub-symbolic approaches.\n",
|
|
"\n",
|
|
"In AI development, there has been a historical debate between \"Neats,\" who believe intelligent behavior can be described with simple principles, and \"Scruffies,\" who see it as solving many complex problems. This debate has diminished over time as modern AI incorporates both approaches.\n",
|
|
"\n",
|
|
"Soft computing, which emerged in the late 1980s, uses techniques like genetic algorithms, fuzzy logic, and neural networks to handle imprecision and uncertainty, proving successful in many modern AI applications.\n",
|
|
"\n",
|
|
"Finally, there is a division in AI research between pursuing narrow AI, which focuses on solving specific problems, and aiming for broader goals like artificial general intelligence and superintelligence. This division reflects differing strategies on achieving long-term AI advancements.\n",
|
|
"\n",
|
|
"General intelligence is a complex concept that is hard to define and measure, leading modern AI research to focus on specific problems and solutions within the field of artificial general intelligence. The philosophy of artificial intelligence debates whether machines can possess mind, consciousness, and mental states similar to humans, focusing on the machine's internal experiences rather than external behaviors. Mainstream AI research generally views these questions as irrelevant to its practical goals. The philosophy of mind, however, finds these questions about machine consciousness central.\n",
|
|
"\n",
|
|
"David Chalmers distinguishes between the \"hard\" and \"easy\" problems of consciousness. The easy problem involves understanding brain functions like signal processing and behavior control, while the hard problem questions why these processes feel like something subjectively. This subjective experience remains elusive, exemplified by the difficulty in explaining color perception to someone color-blind.\n",
|
|
"\n",
|
|
"In the philosophy of mind, computationalism suggests that the human mind functions like an information processing system, equating mental processes to computing. This theory posits that the mind-body relationship is akin to the software-hardware relationship, potentially addressing the mind-body problem.\n",
|
|
"\n",
|
|
"The philosophical concept of \"strong AI,\" initially proposed by Jerry Fodor and Hilary Putnam, suggests that a properly programmed computer could possess a mind similar to humans, as argued by philosopher John Searle. However, Searle's Chinese room argument challenges this by claiming that even if a machine can mimic human behavior, it doesn't necessarily mean it has a mind.\n",
|
|
"\n",
|
|
"The topic of AI welfare and rights centers on the difficulty of determining AI sentience and the ethical implications if machines could feel and suffer. Some argue that sentient AIs might deserve rights similar to animals, and discussions have included the potential for granting \"electronic personhood\" to advanced AI systems within the European Union, which would assign them rights and responsibilities. Critics, however, caution against diminishing human rights and question the autonomy of AI in societal roles.\n",
|
|
"\n",
|
|
"The concept of superintelligence refers to an agent with intelligence far exceeding the smartest humans, potentially leading to a \"singularity\" where AI could autonomously improve itself repeatedly, accelerating its intelligence beyond human control or understanding.\n",
|
|
"\n",
|
|
"The concept of an \"intelligence explosion\" or \"singularity\" suggests a point where technology improves exponentially, although such growth typically follows an S-shaped curve and slows upon reaching technological limits. Transhumanism, supported by figures like Hans Moravec, Kevin Warwick, and Ray Kurzweil, envisions a future where humans and machines merge into superior cyborgs, an idea with historical roots in the works of Aldous Huxley and Robert Ettinger. Edward Fredkin, building on ideas from as early as 1863 by Samuel Butler and later George Dyson, views artificial intelligence as the next evolutionary stage.\n",
|
|
"\n",
|
|
"In literature and media, the concept of robots and artificial intelligence has been explored since antiquity and remains a common theme in science fiction. The term \"robot\" was first introduced by Karel Čapek in 1921. Notable narratives often depict AI as a threat, such as HAL 9000 in \"2001: A Space Odyssey\" and the machines in \"The Terminator\" and \"The Matrix.\" Conversely, stories of loyal robots like Gort from \"The Day the Earth Stood Still\" are less common. Isaac Asimov's Three Laws of Robotics, introduced in his \"Multivac\" series, are frequently discussed in the context of machine ethics, though many AI researchers find them ambiguous and impractical.\n",
|
|
"\n",
|
|
"Numerous works, including Karel Čapek's R.U.R., the films A.I. Artificial Intelligence and Ex Machina, and Philip K. Dick's novel Do Androids Dream of Electric Sheep?, utilize AI to explore the essence of humanity. These works present artificial beings capable of feeling and suffering, prompting a reevaluation of human subjectivity in the context of advanced technology.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(summary_with_detail_1)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Note that this utility also allows passing additional instructions."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 21,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:33:18.789246Z",
|
|
"start_time": "2024-04-10T05:22:57.789764Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"100%|██████████| 5/5 [00:51<00:00, 10.23s/it]"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"- AI is intelligence demonstrated by machines, especially computer systems.\n",
|
|
"- AI technology applications include search engines, recommendation systems, speech interaction, autonomous vehicles, creative tools, and strategic game analysis.\n",
|
|
"- Alan Turing initiated substantial AI research, termed \"machine intelligence.\"\n",
|
|
"- AI became an academic discipline in 1956, experiencing cycles of optimism and \"AI winters\" of reduced funding.\n",
|
|
"- Post-2012 and 2017, advancements in deep learning and transformer architecture led to increased AI funding and interest.\n",
|
|
"- AI influences automation, data-driven decision-making, and integration into various sectors, affecting jobs, healthcare, government, industry, and education.\n",
|
|
"- AI research goals: reasoning, knowledge representation, planning, learning, natural language processing, perception, robotics, and general intelligence.\n",
|
|
"- AI techniques include search, optimization, logic, neural networks, and statistical methods.\n",
|
|
"- AI sub-problems focus on traits like reasoning, problem-solving, knowledge representation, planning, decision-making, learning, and perception.\n",
|
|
"- Early AI research mimicked human step-by-step reasoning; modern methods handle uncertain information using probability and economics.\n",
|
|
"- Knowledge representation in AI involves ontologies and knowledge bases to support intelligent decision-making and problem-solving.\n",
|
|
"- Planning in AI involves goal-oriented actions, while decision-making considers preferences and outcomes to maximize utility.\n",
|
|
"- Learning in AI includes machine learning (unsupervised, supervised, reinforcement, transfer, deep learning) and computational learning theory.\n",
|
|
"- Natural language processing (NLP) enables communication in human languages, with modern techniques like word embedding and transformers.\n",
|
|
"- Perception in AI uses sensor input for tasks like speech recognition and computer vision.\n",
|
|
"- Techniques for achieving AI goals include state space search, local search, gradient descent, evolutionary computation, and swarm intelligence algorithms.\n",
|
|
"- Logic in AI uses formal systems for reasoning and knowledge representation, with applications in problem-solving and decision-making under uncertainty.\n",
|
|
"- Probabilistic methods in AI address reasoning, planning, learning, and perception with incomplete or uncertain information.\n",
|
|
"\n",
|
|
"- Neural networks: Interconnected nodes, similar to brain neurons, with input, hidden layers, and output.\n",
|
|
"- Deep neural networks: At least 2 hidden layers.\n",
|
|
"- Training techniques: Commonly use backpropagation.\n",
|
|
"- Feedforward networks: Signal passes in one direction.\n",
|
|
"- Recurrent networks: Output fed back into input for short-term memory.\n",
|
|
"- Perceptrons: Single layer of neurons.\n",
|
|
"- Convolutional networks: Strengthen connections between close neurons, important in image processing.\n",
|
|
"- Deep learning: Multiple layers extract features progressively, used in various AI subfields.\n",
|
|
"- GPT (Generative Pre-trained Transformers): Large language models pre-trained on text, used in chatbots.\n",
|
|
"- Specialized AI hardware: GPUs replaced CPUs for training large-scale machine learning models.\n",
|
|
"- AI applications: Used in search engines, online ads, virtual assistants, autonomous vehicles, language translation, facial recognition.\n",
|
|
"- AI in healthcare: Increases patient care, used in medical research and drug discovery.\n",
|
|
"- AI in games: Used in chess, Jeopardy!, Go, and real-time strategy games.\n",
|
|
"- Military AI: Enhances command, control, and operations, used in coordination and threat detection.\n",
|
|
"- Generative AI: Creates realistic images and texts, used in creative arts.\n",
|
|
"- AI ethics and risks: Concerns include privacy, surveillance, copyright issues, misinformation, and algorithmic bias.\n",
|
|
"- Algorithmic bias: Can cause discrimination if trained on biased data, fairness in machine learning is a critical area of study.\n",
|
|
"\n",
|
|
"- AI engineers demographics: 4% black, 20% women.\n",
|
|
"- ACM FAccT 2022: Recommends limiting use of self-learning neural networks due to bias.\n",
|
|
"- AI complexity: Designers often can't explain decision-making processes.\n",
|
|
"- Misleading AI outcomes: Skin disease identifier misclassifies images with rulers as \"cancerous\"; medical resource allocator misclassifies asthma patients as low risk for pneumonia.\n",
|
|
"- Right to explanation: Essential for accountability, especially in medical and legal fields.\n",
|
|
"- DARPA's XAI program (2014): Aims to make AI decisions understandable.\n",
|
|
"- Transparency solutions: SHAP, LIME, multitask learning, deconvolution, DeepDream.\n",
|
|
"- AI misuse: Authoritarian surveillance, misinformation, autonomous weapons.\n",
|
|
"- AI in warfare: 30 nations support UN ban on autonomous weapons; over 50 countries researching battlefield robots.\n",
|
|
"- Technological unemployment: Disagreement on long-term impact; potential job losses in various sectors.\n",
|
|
"- Existential risks of AI: Potential to lose control over superintelligent AI; concerns from notable figures like Stephen Hawking, Bill Gates, Elon Musk.\n",
|
|
"- Ethical AI development: Emphasis on friendly AI, machine ethics, and alignment with human values.\n",
|
|
"- AI regulation: Increasing global legislative activity; first global AI Safety Summit in 2023 calls for international cooperation.\n",
|
|
"- Historical perspective: AI research dates back to antiquity, significant developments in mid-20th century.\n",
|
|
"\n",
|
|
"- 1974: U.S. and British governments halted AI exploratory research due to criticism and funding pressures.\n",
|
|
"- 1980s: AI research revived due to commercial success of expert systems; market reached over $1 billion by 1985.\n",
|
|
"- 1987: Collapse of Lisp Machine market led to a second, longer AI winter.\n",
|
|
"- 1990: Yann LeCun demonstrated successful use of convolutional neural networks for recognizing handwritten digits.\n",
|
|
"- Late 1990s-early 2000s: AI regained reputation through formal mathematical methods and specific problem solutions.\n",
|
|
"- 2012: Deep learning began to dominate, supported by hardware improvements and large data access.\n",
|
|
"- 2015-2019: Machine learning research publications increased by 50%.\n",
|
|
"- 2016: Fairness and misuse of technology became central issues in machine learning.\n",
|
|
"- 2020: GPT-3 released by OpenAI, capable of generating human-like text.\n",
|
|
"- 2022: Approximately $50 billion annually invested in AI in the U.S.; 800,000 AI-related job openings in the U.S.\n",
|
|
"- AI research historically guided by no unifying theory; recent dominance of statistical machine learning.\n",
|
|
"- Symbolic AI successful in high-level reasoning but failed in tasks like learning and commonsense reasoning.\n",
|
|
"- Philosophical debates on AI's ability to think or have a mind, with mainstream research focusing on problem-solving capabilities.\n",
|
|
"- Concerns about AI welfare and rights, especially regarding potential sentience and ethical treatment.\n",
|
|
"\n",
|
|
"- Focus on the impact of AI on human subjectivity.\n",
|
|
"- References:\n",
|
|
" - Films: \"Artificial Intelligence,\" \"Ex Machina\"\n",
|
|
" - Novel: \"Do Androids Dream of Electric Sheep?\" by Philip K. Dick.\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"summary_with_additional_instructions = summarize(artificial_intelligence_wikipedia_text, detail=0.1,\n",
|
|
" additional_instructions=\"Write in point form and focus on numerical data.\")\n",
|
|
"print(summary_with_additional_instructions)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Finally, note that the utility allows for recursive summarization, where each summary is based on the previous summaries, adding more context to the summarization process. This can be enabled by setting the `summarize_recursively` parameter to True. This is more computationally expensive, but can increase consistency and coherence of the combined summary."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 22,
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-10T05:33:30.123036Z",
|
|
"start_time": "2024-04-10T05:33:18.791253Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"100%|██████████| 5/5 [00:53<00:00, 10.64s/it]"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Artificial intelligence (AI) is the simulation of human intelligence in machines, designed to perform tasks that typically require human intelligence. This includes applications like advanced search engines, recommendation systems, speech interaction, autonomous vehicles, and strategic game analysis. AI was established as an academic discipline in 1956 and has experienced cycles of high expectations followed by disillusionment and reduced funding, known as AI winters. Interest in AI surged post-2012 with advancements in deep learning and again post-2017 with the development of transformer architectures, leading to significant progress in the early 2020s.\n",
|
|
"\n",
|
|
"AI research encompasses various sub-fields aimed at developing systems capable of reasoning, learning, perception, and natural language understanding, among others. Techniques used in AI research include search and optimization algorithms, formal logic, artificial neural networks, and probabilistic methods for uncertain reasoning.\n",
|
|
"\n",
|
|
"The application of AI is widespread across different sectors, influencing economic, societal, and job market dynamics. This has raised important discussions about the ethical implications, long-term effects, and the need for regulatory policies to ensure the safety and benefits of AI technologies. AI's goal is not only to replicate human intelligence but also to extend it, using a combination of computational techniques drawn from various disciplines.\n",
|
|
"\n",
|
|
"Artificial intelligence (AI) simulates human intelligence in machines to perform tasks that typically require human cognition. Established as an academic discipline in 1956, AI has evolved through cycles of high expectations and subsequent disillusionment, experiencing significant advancements in the 2010s with deep learning and transformer architectures. AI research includes sub-fields like reasoning, learning, perception, and natural language understanding, employing techniques such as optimization algorithms, neural networks, and probabilistic methods.\n",
|
|
"\n",
|
|
"Neural networks, central to AI, mimic the human brain's interconnected nodes to recognize patterns and learn from data, using algorithms like backpropagation. Deep learning, a subset of neural networks, uses multiple layers to progressively extract complex features, significantly enhancing performance in areas like computer vision and speech recognition. The success of deep learning since the mid-2010s is attributed to increased computational power and large datasets.\n",
|
|
"\n",
|
|
"Generative pre-trained transformers (GPT) are advanced AI models trained to understand and generate human-like text based on the semantic relationships between words. These models, after initial training on vast text corpora, undergo further training to improve accuracy and reduce errors, and are used in applications like chatbots.\n",
|
|
"\n",
|
|
"AI applications are widespread across various sectors including healthcare, where it enhances patient care and medical research, and gaming, where AI has outperformed human experts in complex games. AI also plays a significant role in military applications, enhancing operations and decision-making processes.\n",
|
|
"\n",
|
|
"Ethical considerations and risks such as privacy concerns, copyright issues, misinformation, and algorithmic bias are critical in AI development and deployment. These challenges highlight the need for careful management and regulation of AI technologies to maximize benefits while minimizing harms.\n",
|
|
"\n",
|
|
"The text discusses various aspects of artificial intelligence (AI), including its development, applications, ethical considerations, and regulatory measures. AI, which simulates human intelligence in machines, has evolved significantly since its inception in 1956, with notable advancements in deep learning and transformer architectures. AI research spans sub-fields like reasoning, learning, and perception, employing techniques such as neural networks and optimization algorithms.\n",
|
|
"\n",
|
|
"AI applications are extensive, impacting sectors like healthcare, military, and surveillance, and raising ethical concerns about bias, transparency, and the potential for misuse by bad actors. The complexity of AI systems, such as those using deep neural networks, often leads to a lack of transparency in decision-making processes, which can result in unintended consequences, such as misdiagnoses or unfair resource allocation.\n",
|
|
"\n",
|
|
"The potential misuse of AI by authoritarian regimes for surveillance and control, and by other bad actors for creating autonomous weapons, underscores the dual-use nature of AI technologies. This has led to calls for stringent ethical guidelines and regulatory frameworks to ensure AI development aligns with human rights and democratic values.\n",
|
|
"\n",
|
|
"Internationally, there is a push for cooperation to manage AI risks, with initiatives like the Global Partnership on Artificial Intelligence advocating for AI that respects human rights. Regulatory efforts are also underway, with countries and organizations emphasizing the need for frameworks that ensure AI's ethical deployment and the safety of AI systems to prevent existential risks to humanity.\n",
|
|
"\n",
|
|
"The history of AI has seen periods of both enthusiasm and skepticism. In 1974, AI research faced significant cuts from the U.S. and British governments following critical assessments, leading to the first \"AI winter.\" The field rebounded in the early 1980s with the success of expert systems, but faced another setback with the collapse of the Lisp Machine market in 1987, initiating a second AI winter. During the 1980s, researchers began exploring \"sub-symbolic\" approaches, moving away from high-level symbolic representations to methods like neural networks, which gained prominence with Geoffrey Hinton's work. By the late 1990s, AI regained credibility by focusing on narrow, specific problems, leading to practical applications.\n",
|
|
"\n",
|
|
"The 2010s saw a dominance of deep learning, driven by improved hardware and access to large data sets, which significantly increased interest and investment in AI. Issues of fairness and misuse of technology became central in the mid-2010s, reshaping research priorities. The late 2010s and early 2020s witnessed the rise of artificial general intelligence (AGI) companies and significant advancements like AlphaGo and GPT-3, marking a new era of enthusiasm in AI.\n",
|
|
"\n",
|
|
"Philosophically, AI has been debated since Alan Turing proposed the Turing test in 1950, focusing on whether machines can exhibit intelligent behavior rather than possess consciousness. The field has largely moved away from the goal of simulating human intelligence towards creating systems that can solve specific problems effectively. Discussions continue around the potential sentience and rights of AI systems, reflecting growing concerns over the ethical implications of advanced AI. The concept of superintelligence and the singularity remains speculative, with opinions divided on the feasibility and implications of such developments.\n",
|
|
"\n",
|
|
"The text discusses the influence of artificial intelligence on our perception of human subjectivity, as explored in the film \"Ex Machina\" and Philip K. Dick's novel \"Do Androids Dream of Electric Sheep?\" These works consider how technology equipped with AI can alter our understanding of what it means to be human.\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"recursive_summary = summarize(artificial_intelligence_wikipedia_text, detail=0.1, summarize_recursively=True)\n",
|
|
"print(recursive_summary)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.9.9"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 1
|
|
}
|