adds two Whisper guides (#628)
* adds two Whisper guides * polishing * fixes function argument misordering --------- Co-authored-by: Ted Sanders <ted@openai.com>pull/633/head
prestontuggle
10 months ago
committed by
GitHub
parent
8e6e058c6a
commit
0933dd1569
@ -0,0 +1,330 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Addressing transcription misspellings: prompt vs post-processing\n",
|
||||
"\n",
|
||||
"We are addressing the problem of enhancing the precision of transcriptions, particularly when it comes to company names and product references. Our solution involves a dual strategy that utilizes both the Whisper prompt parameter and GPT-4's post-processing capabilities. \n",
|
||||
"\n",
|
||||
"Two approaches to correct inaccuracies are:\n",
|
||||
"\n",
|
||||
"- We input a list of correct spellings directly into Whisper's prompt parameter to guide the initial transcription.\n",
|
||||
"\n",
|
||||
"- We utilized GPT-4 to fix misspellings post transcription, again using the same list of correct spellings in the prompt.\n",
|
||||
"\n",
|
||||
"These strategies aimed at ensuring precise transcription of unfamilar proper nouns.\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"To get started, let's:\n",
|
||||
"\n",
|
||||
"- Import the OpenAI Python library (if you don't have it, you'll need to install it with ```pip install openai```)\n",
|
||||
"- Download the audio file example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# imports\n",
|
||||
"import openai # for making OpenAI API calls\n",
|
||||
"import urllib # for downloading example audio files\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"('data/ZyntriQix.wav', <http.client.HTTPMessage at 0x11ac6a2b0>)"
|
||||
]
|
||||
},
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# set download paths\n",
|
||||
"ZyntriQix_remote_filepath = \"https://cdn.openai.com/API/examples/data/ZyntriQix.wav\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# set local save locations\n",
|
||||
"ZyntriQix_filepath = \"data/ZyntriQix.wav\"\n",
|
||||
"\n",
|
||||
"# download example audio files and save locally\n",
|
||||
"urllib.request.urlretrieve(ZyntriQix_remote_filepath, ZyntriQix_filepath)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setting our baseline with a fictitious audio recording\n",
|
||||
"\n",
|
||||
"Our reference point is a monologue, which was generated by ChatGPT from prompts given by the author. The author then voiced this content. So, the author both guided the ChatGPT's output with prompts and brought it to life by speaking it."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Our fictitious company, ZyntriQix, offers a range of tech products. These include Digique Plus, CynapseFive, VortiQore V8, EchoNix Array, OrbitalLink Seven, and DigiFractal Matrix. We also spearhead several initiatives such as PULSE, RAPT, B.R.I.C.K., Q.U.A.R.T.Z., and F.L.I.N.T."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# define a wrapper function for seeing how prompts affect transcriptions\n",
|
||||
"def transcribe(prompt: str, audio_filepath) -> str:\n",
|
||||
" \"\"\"Given a prompt, transcribe the audio file.\"\"\"\n",
|
||||
" transcript = openai.Audio.transcribe(\n",
|
||||
" file=open(audio_filepath, \"rb\"),\n",
|
||||
" model=\"whisper-1\",\n",
|
||||
" prompt=prompt,\n",
|
||||
" )\n",
|
||||
" return transcript[\"text\"]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"\"Have you heard of ZentricX? This tech giant boasts products like Digi-Q+, Synapse 5, VortiCore V8, Echo Nix Array, and not to forget the latest Orbital Link 7 and Digifractal Matrix. Their innovation arsenal also includes the Pulse framework, Wrapped system, they've developed a brick infrastructure court system, and launched the Flint initiative, all highlighting their commitment to relentless innovation. ZentricX, in just 30 years, has soared from a startup to a tech titan, serving us tech marvels alongside a stimulating linguistic challenge. Quite an adventure, wouldn't you agree?\""
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# baseline transcription with no prompt\n",
|
||||
"transcribe(prompt=\"\", audio_filepath=ZyntriQix_filepath)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Whisper transcribed our company name, product names, and miscapitalized our acronyms incorrectly. Let's pass the correct names as a list in the prompt. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"\"Have you heard of ZyntriQix? This tech giant boasts products like Digique Plus, CynapseFive, VortiQore V8, EchoNix Array, and not to forget the latest OrbitalLink Seven and DigiFractal Matrix. Their innovation arsenal also includes the PULSE framework, RAPT system. They've developed a B.R.I.C.K. infrastructure, Q.U.A.R.T. system, and launched the F.L.I.N.T. initiative, all highlighting their commitment to relentless innovation. ZyntriQix in just 30 years has soared from a startup to a tech titan, serving us tech marvels alongside a stimulating linguistic challenge. Quite an adventure, wouldn't you agree?\""
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# add the correct spelling names to the prompt\n",
|
||||
"transcribe(\n",
|
||||
" prompt=\"ZyntriQix, Digique Plus, CynapseFive, VortiQore V8, EchoNix Array, OrbitalLink Seven, DigiFractal Matrix, PULSE, RAPT, B.R.I.C.K., Q.U.A.R.T.Z., F.L.I.N.T.\",\n",
|
||||
" audio_filepath=ZyntriQix_filepath,\n",
|
||||
")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"When passing the list of product names, some of the product names are transcribed correctly while others are still misspelled. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"\"Have you heard of ZentricX? This tech giant boasts products like DigiCube Plus, Synapse 5, VortiCore V8, EchoNix Array, and not to forget the latest Orbital Link 7 and Digifractal Matrix. Their innovation arsenal also includes the PULSE framework, RAPT system. They've developed a brick infrastructure court system and launched the F.L.I.N.T. initiative, all highlighting their commitment to relentless innovation. ZentricX in just 30 years has soared from a startup to a tech titan, serving us tech marvels alongside a stimulating linguistic challenge. Quite an adventure, wouldn't you agree?\""
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# add a full product list to the prompt\n",
|
||||
"transcribe(\n",
|
||||
" prompt=\"ZyntriQix, Digique Plus, CynapseFive, VortiQore V8, EchoNix Array, OrbitalLink Seven, DigiFractal Matrix, PULSE, RAPT, AstroPixel Array, QuantumFlare Five, CyberPulse Six, VortexDrive Matrix, PhotonLink Ten, TriCircuit Array, PentaSync Seven, UltraWave Eight, QuantumVertex Nine, HyperHelix X, DigiSpiral Z, PentaQuark Eleven, TetraCube Twelve, GigaPhase Thirteen, EchoNeuron Fourteen, FusionPulse V15, MetaQuark Sixteen, InfiniCircuit Seventeen, TeraPulse Eighteen, ExoMatrix Nineteen, OrbiSync Twenty, QuantumHelix TwentyOne, NanoPhase TwentyTwo, TeraFractal TwentyThree, PentaHelix TwentyFour, ExoCircuit TwentyFive, HyperQuark TwentySix, B.R.I.C.K., Q.U.A.R.T.Z., F.L.I.N.T.\",\n",
|
||||
" audio_filepath=ZyntriQix_filepath,\n",
|
||||
")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## You can use GPT-4 to fix spelling mistakes\n",
|
||||
"\n",
|
||||
"Leveraging GPT-4 proves especially useful when the speech content is unknown beforehand and we have a list of product names readily available.\n",
|
||||
"\n",
|
||||
"The post-processing technique using GPT-4 is notably more scalable than depending solely on Whisper's prompt parameter, which has a token limit of 244. GPT-4 allows us to process larger lists of correct spellings, making it a more robust method for handling extensive product lists.\n",
|
||||
"\n",
|
||||
"However, this post-processing technique isn't without limitations. It's constrained by the context window of the chosen model, which may pose challenges when dealing with vast numbers of unique terms. For instance, companies with thousands of SKUs may find that the context window of GPT-4 is insufficient to handle their requirements, and they might need to explore alternative solutions.\n",
|
||||
"\n",
|
||||
"Interestingly, the GPT-4 post-processing technique seems more reliable than using Whisper alone. This method, which leverages a product list, enhances the reliability of our results. However, this increased reliability comes at a price, as using this approach can increase costs and can result in higher latency."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# define a wrapper function for seeing how prompts affect transcriptions\n",
|
||||
"def transcribe_with_spellcheck(system_message, audio_filepath):\n",
|
||||
" response = openai.ChatCompletion.create(\n",
|
||||
" model=\"gpt-4\",\n",
|
||||
" temperature=0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"system\", \"content\": system_message},\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": transcribe(prompt=\"\", audio_filepath=audio_filepath),\n",
|
||||
" },\n",
|
||||
" ],\n",
|
||||
" )\n",
|
||||
" return response[\"choices\"][0][\"message\"][\"content\"]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now, let's input the original product list into GPT-4 and evaluate its performance. By doing so, we aim to assess the AI model's ability to correctly spell the proprietary product names, even with no prior knowledge of the exact terms to appear in the transcription. In our experiment, GPT-4 was successful in correctly spelling our product names, confirming its potential as a reliable tool for ensuring transcription accuracy."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Have you heard of ZyntriQix? This tech giant boasts products like Digique Plus, CynapseFive, VortiQore V8, EchoNix Array, and not to forget the latest OrbitalLink Seven and DigiFractal Matrix. Their innovation arsenal also includes the PULSE framework, RAPT system, they've developed a B.R.I.C.K. infrastructure court system, and launched the F.L.I.N.T. initiative, all highlighting their commitment to relentless innovation. ZyntriQix, in just 30 years, has soared from a startup to a tech titan, serving us tech marvels alongside a stimulating linguistic challenge. Quite an adventure, wouldn't you agree?\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"system_prompt = \"You are a helpful assistant for the company ZyntriQix. Your task is to correct any spelling discrepancies in the transcribed text. Make sure that the names of the following products are spelled correctly: ZyntriQix, Digique Plus, CynapseFive, VortiQore V8, EchoNix Array, OrbitalLink Seven, DigiFractal Matrix, PULSE, RAPT, B.R.I.C.K., Q.U.A.R.T.Z., F.L.I.N.T.\"\n",
|
||||
"new_text = transcribe_with_spellcheck(system_prompt, audio_filepath=ZyntriQix_filepath)\n",
|
||||
"print(new_text)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In this case, we supplied a comprehensive product list that included all the previously used spellings, along with additional new names. This scenario simulates a real-life situation where we have a substantial SKU list and uncertain about the exact terms to appear in the transcription. Feeding this extensive list of product names into the system resulted in a correctly transcribed output."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Have you heard of ZyntriQix? This tech giant boasts products like Digique Plus, CynapseFive, VortiQore V8, EchoNix Array, and not to forget the latest OrbitalLink Seven and DigiFractal Matrix. Their innovation arsenal also includes the PULSE framework, RAPT system, they've developed a B.R.I.C.K. infrastructure court system, and launched the F.L.I.N.T. initiative, all highlighting their commitment to relentless innovation. ZyntriQix, in just 30 years, has soared from a startup to a tech titan, serving us tech marvels alongside a stimulating linguistic challenge. Quite an adventure, wouldn't you agree?\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"system_prompt = \"You are a helpful assistant for the company ZyntriQix. Your task is to correct any spelling discrepancies in the transcribed text. Make sure that the names of the following products are spelled correctly: ZyntriQix, Digique Plus, CynapseFive, VortiQore V8, EchoNix Array, OrbitalLink Seven, DigiFractal Matrix, PULSE, RAPT, AstroPixel Array, QuantumFlare Five, CyberPulse Six, VortexDrive Matrix, PhotonLink Ten, TriCircuit Array, PentaSync Seven, UltraWave Eight, QuantumVertex Nine, HyperHelix X, DigiSpiral Z, PentaQuark Eleven, TetraCube Twelve, GigaPhase Thirteen, EchoNeuron Fourteen, FusionPulse V15, MetaQuark Sixteen, InfiniCircuit Seventeen, TeraPulse Eighteen, ExoMatrix Nineteen, OrbiSync Twenty, QuantumHelix TwentyOne, NanoPhase TwentyTwo, TeraFractal TwentyThree, PentaHelix TwentyFour, ExoCircuit TwentyFive, HyperQuark TwentySix, GigaLink TwentySeven, FusionMatrix TwentyEight, InfiniFractal TwentyNine, MetaSync Thirty, B.R.I.C.K., Q.U.A.R.T.Z., F.L.I.N.T. Only add necessary punctuation such as periods, commas, and capitalization, and use only the context provided.\"\n",
|
||||
"new_text = transcribe_with_spellcheck(system_prompt, audio_filepath=ZyntriQix_filepath)\n",
|
||||
"print(new_text)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We are employing GPT-4 as a spell checker, using the same list of correct spellings that was previously used in the prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"The misspelled words are: ZentricX, Digi-Q+, Synapse 5, VortiCore V8, Echo Nix Array, Orbital Link 7, Digifractal Matrix, Pulse, Wrapped, brick, Flint, and 30. The total number of misspelled words is 12.\n",
|
||||
"\n",
|
||||
"The corrected paragraph is:\n",
|
||||
"\n",
|
||||
"Have you heard of ZyntriQix? This tech giant boasts products like Digique Plus, CynapseFive, VortiQore V8, EchoNix Array, and not to forget the latest OrbitalLink Seven and DigiFractal Matrix. Their innovation arsenal also includes the PULSE framework, RAPT system, they've developed a B.R.I.C.K. infrastructure court system, and launched the F.L.I.N.T. initiative, all highlighting their commitment to relentless innovation. ZyntriQix, in just MetaSync Thirty years, has soared from a startup to a tech titan, serving us tech marvels alongside a stimulating linguistic challenge. Quite an adventure, wouldn't you agree?\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"system_prompt = \"You are a helpful assistant for the company ZyntriQix. Your first task is to list the words that are not spelled correctly according to the list provided to you and to tell me the number of misspelled words. Your next task is to insert those correct words in place of the misspelled ones. List: ZyntriQix, Digique Plus, CynapseFive, VortiQore V8, EchoNix Array, OrbitalLink Seven, DigiFractal Matrix, PULSE, RAPT, AstroPixel Array, QuantumFlare Five, CyberPulse Six, VortexDrive Matrix, PhotonLink Ten, TriCircuit Array, PentaSync Seven, UltraWave Eight, QuantumVertex Nine, HyperHelix X, DigiSpiral Z, PentaQuark Eleven, TetraCube Twelve, GigaPhase Thirteen, EchoNeuron Fourteen, FusionPulse V15, MetaQuark Sixteen, InfiniCircuit Seventeen, TeraPulse Eighteen, ExoMatrix Nineteen, OrbiSync Twenty, QuantumHelix TwentyOne, NanoPhase TwentyTwo, TeraFractal TwentyThree, PentaHelix TwentyFour, ExoCircuit TwentyFive, HyperQuark TwentySix, GigaLink TwentySeven, FusionMatrix TwentyEight, InfiniFractal TwentyNine, MetaSync Thirty, B.R.I.C.K., Q.U.A.R.T.Z., F.L.I.N.T.\"\n",
|
||||
"new_text = transcribe_with_spellcheck(system_prompt, audio_filepath=ZyntriQix_filepath)\n",
|
||||
"print(new_text)\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.9"
|
||||
},
|
||||
"orig_nbformat": 4
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@ -0,0 +1,465 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Enhancing Whisper transcriptions: pre- & post-processing techniques\n",
|
||||
"\n",
|
||||
"This notebook offers a guide to improve the Whisper's transcriptions. We'll streamline your audio data via trimming and segmentation, enhancing Whisper's transcription quality. After transcriptions, we'll refine the output by adding punctuation, adjusting product terminology (e.g., 'five two nine' to '529'), and mitigating Unicode issues. These strategies will help improve the clarity of your transcriptions, but remember, customization based on your unique use-case may be beneficial.\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"To get started let's import a few different libraries:\n",
|
||||
"\n",
|
||||
"- [PyDub](http://pydub.com/) is a simple and easy-to-use Python library for audio processing tasks such as slicing, concatenating, and exporting audio files.\n",
|
||||
"\n",
|
||||
"- The `Audio` class from the `IPython.display` module allows you to create an audio control that can play sound in Jupyter notebooks, providing a straightforward way to play audio data directly in your notebook.\n",
|
||||
"\n",
|
||||
"- For our audio file, we'll use a fictional earnings call written by ChatGPT and read aloud by the author.This audio file is relatively short, but hopefully provides you with an illustrative idea of how these pre and post processing steps can be applied to any audio file. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import openai\n",
|
||||
"import os\n",
|
||||
"import urllib\n",
|
||||
"from IPython.display import Audio\n",
|
||||
"from pathlib import Path\n",
|
||||
"from pydub import AudioSegment"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"('data/EarningsCall.wav', <http.client.HTTPMessage at 0x11a92fbe0>)"
|
||||
]
|
||||
},
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# set download paths\n",
|
||||
"EarningsCall_remote_filepath = \"https://cdn.openai.com/API/examples/data/EarningsCall.wav\"\n",
|
||||
"\n",
|
||||
"# set local save locations\n",
|
||||
"EarningsCall_filepath = \"data/EarningsCall.wav\"\n",
|
||||
"\n",
|
||||
"# download example audio files and save locally\n",
|
||||
"urllib.request.urlretrieve(EarningsCall_remote_filepath, EarningsCall_filepath)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"At times, files with long silences at the beginning can cause Whisper to transcribe the audio incorrectly. We'll use Pydub to detect and trim the silence. \n",
|
||||
"\n",
|
||||
"Here, we've set the decibel threshold of 20. You can change this if you would like."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Function to detect leading silence\n",
|
||||
"# Returns the number of milliseconds until the first sound (chunk averaging more than X decibels)\n",
|
||||
"def milliseconds_until_sound(sound, silence_threshold_in_decibels=-20.0, chunk_size=10):\n",
|
||||
" trim_ms = 0 # ms\n",
|
||||
"\n",
|
||||
" assert chunk_size > 0 # to avoid infinite loop\n",
|
||||
" while sound[trim_ms:trim_ms+chunk_size].dBFS < silence_threshold_in_decibels and trim_ms < len(sound):\n",
|
||||
" trim_ms += chunk_size\n",
|
||||
"\n",
|
||||
" return trim_ms"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def trim_start(filepath):\n",
|
||||
" path = Path(filepath)\n",
|
||||
" directory = path.parent\n",
|
||||
" filename = path.name\n",
|
||||
" audio = AudioSegment.from_file(filepath, format=\"wav\")\n",
|
||||
" start_trim = milliseconds_until_sound(audio)\n",
|
||||
" trimmed = audio[start_trim:]\n",
|
||||
" new_filename = directory / f\"trimmed_{filename}\"\n",
|
||||
" trimmed.export(new_filename, format=\"wav\")\n",
|
||||
" return trimmed, new_filename"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def transcribe_audio(file,output_dir):\n",
|
||||
" audio_path = os.path.join(output_dir, file)\n",
|
||||
" with open(audio_path, 'rb') as audio_data:\n",
|
||||
" transcription = openai.Audio.transcribe(\"whisper-1\", audio_data)\n",
|
||||
" return transcription['text']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"At times, we've seen unicode character injection in transcripts, removing any non-ASCII characters should help mitigate this issue.\n",
|
||||
"\n",
|
||||
"Keep in mind you should not use this function if you are transcribing in Greek, Cyrillic, Arabic, Chinese, etc"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Define function to remove non-ascii characters\n",
|
||||
"def remove_non_ascii(text):\n",
|
||||
" return ''.join(i for i in text if ord(i)<128)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This function will add formatting and punctuation to our transcript. Whisper generates a transcript with punctuation but without formatting."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Define function to add punctuation\n",
|
||||
"def punctuation_assistant(ascii_transcript):\n",
|
||||
" response = openai.ChatCompletion.create(\n",
|
||||
" model=\"gpt-3.5-turbo\",\n",
|
||||
" temperature=0,\n",
|
||||
" messages=[\n",
|
||||
" {\n",
|
||||
" \"role\": \"system\", \n",
|
||||
" \"content\": \"You are a helpful assistant that adds punctuation to text. Preserve the original words and only insert necessary punctuation such as periods, commas, capialization, symbols like dollar sings or percentage signs, and formatting. Use only the context provided. If there is no context provided say, 'No context provided'\\n\"\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\", \n",
|
||||
" \"content\": ascii_transcript \n",
|
||||
" }\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" return response"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Our audio file is a recording from a fake earnings call that includes a lot of financial products. This function can help ensure that if Whisper transcribes these financial product names incorrectly, that they can be corrected. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Define function to fix product mispellings\n",
|
||||
"def product_assistant(ascii_transcript):\n",
|
||||
" response = openai.ChatCompletion.create(\n",
|
||||
" model=\"gpt-4\",\n",
|
||||
" temperature=0,\n",
|
||||
" messages=[\n",
|
||||
" {\n",
|
||||
" \"role\": \"system\", \n",
|
||||
" \"content\": \"You are an intelligent assistant specializing in financial products; your task is to process transcripts of earnings calls, ensuring that all references to financial products and common financial terms are in the correct format. For each financial product or common term that is typically abbreviated as an acronym, the full term should be spelled out followed by the acronym in parentheses. For example, '401k' should be transformed to '401(k) retirement savings plan', 'HSA' should be transformed to 'Health Savings Account (HSA)', 'ROA' should be transformed to 'Return on Assets (ROA)', 'VaR' should be transformed to 'Value at Risk (VaR)', and 'PB' should be transformed to 'Price to Book (PB) ratio'. Similarly, transform spoken numbers representing financial products into their numeric representations, followed by the full name of the product in parentheses. For instance, 'five two nine' to '529 (Education Savings Plan)' and 'four zero one k' to '401(k) (Retirement Savings Plan)'. However, be aware that some acronyms can have different meanings based on the context (e.g., 'LTV' can stand for 'Loan to Value' or 'Lifetime Value'). You will need to discern from the context which term is being referred to and apply the appropriate transformation. In cases where numerical figures or metrics are spelled out but do not represent specific financial products (like 'twenty three percent'), these should be left as is. Your role is to analyze and adjust financial product terminology in the text. Once you've done that, produce the adjusted transcript and a list of the words you've changed\"\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\", \n",
|
||||
" \"content\": ascii_transcript \n",
|
||||
" }\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" return response"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This function will create a new file with 'trimmed' appended to the original file name"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Trim the start of the original audio file\n",
|
||||
"trimmed_audio = trim_start(EarningsCall_filepath)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"trimmed_audio, trimmed_filename = trim_start(EarningsCall_filepath)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Our fake earnings report audio file is fairly short in length, so we'll adjust the segments accordingly. Keep in mind you can adjust the segment length as you need."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Segment audio\n",
|
||||
"trimmed_audio = AudioSegment.from_wav(trimmed_filename) # Load the trimmed audio file\n",
|
||||
"\n",
|
||||
"one_minute = 1 * 60 * 1000 # Duration for each segment (in milliseconds)\n",
|
||||
"\n",
|
||||
"start_time = 0 # Start time for the first segment\n",
|
||||
"\n",
|
||||
"i = 0 # Index for naming the segmented files\n",
|
||||
"\n",
|
||||
"output_dir_trimmed = \"TrimmedEarningsDirectory\" # Output directory for the segmented files\n",
|
||||
"\n",
|
||||
"if not os.path.isdir(output_dir_trimmed): # Create the output directory if it does not exist\n",
|
||||
" os.makedirs(output_dir_trimmed)\n",
|
||||
"\n",
|
||||
"while start_time < len(trimmed_audio): # Loop over the trimmed audio file\n",
|
||||
" segment = trimmed_audio[start_time:start_time + one_minute] # Extract a segment\n",
|
||||
" segment.export(os.path.join(output_dir_trimmed, f\"trimmed_{i:02d}.wav\"), format=\"wav\") # Save the segment\n",
|
||||
" start_time += one_minute # Update the start time for the next segment\n",
|
||||
" i += 1 # Increment the index for naming the next file\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get list of trimmed and segmented audio files and sort them numerically\n",
|
||||
"audio_files = sorted(\n",
|
||||
" (f for f in os.listdir(output_dir_trimmed) if f.endswith(\".wav\")),\n",
|
||||
" key=lambda f: int(''.join(filter(str.isdigit, f)))\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Use a loop to apply the transcribe function to all audio files\n",
|
||||
"transcriptions = [transcribe_audio(file, output_dir_trimmed) for file in audio_files]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Concatenate the transcriptions\n",
|
||||
"full_transcript = ' '.join(transcriptions)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Good afternoon, everyone. And welcome to FinTech Plus Sync's second quarter 2023 earnings call. I'm John Doe, CEO of FinTech Plus. We've had a stellar Q2 with a revenue of 125 million, a 25% increase year over year. Our gross profit margin stands at a solid 58%, due in part to cost efficiencies gained from our scalable business model. Our EBITDA has surged to 37.5 million, translating to a remarkable 30% EBITDA margin. Our net income for the quarter rose to 16 million, which is a noteworthy increase from 10 million in Q2 2022. Our total addressable market has grown substantially thanks to the expansion of our high yield savings product line and the new RoboAdvisor platform. We've been diversifying our asset-backed securities portfolio, investing heavily in collateralized. debt obligations, and residential mortgage-backed securities. We've also invested $25 million in AAA rated corporate bonds, enhancing our risk adjusted returns. As for our balance sheet, total assets reached $1.5 billion with total liabilities at $900 million, leaving us with a solid equity base of $600 million. Our debt-to-equity ratio stands at 1.5, a healthy figure considering our expansionary phase. We continue to see substantial organic user growth, with customer acquisition cost dropping by 15% and lifetime value growing by 25%. Our LTVCAC ratio is at an impressive 3.5%. In terms of risk management, we have a value-at-risk model in place with a 99%... confidence level indicating that our maximum loss will not exceed 5 million in the next trading day. We've adopted a conservative approach to managing our leverage and have a healthy tier one capital ratio of 12.5%. Our forecast for the coming quarter is positive. We expect revenue to be around 135 million and 8% quarter over quarter growth driven primarily by our cutting edge blockchain solutions and AI driven predictive analytics. We're also excited about the upcoming IPO of our FinTech subsidiary, Pay Plus, which we expect to raise 200 million. Significantly bolstering our liquidity and paving the way for aggressive growth strategies. We thank our shareholders for their continued faith in us and we look forward to an even more successful Q3. Thank you so much.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(full_transcript)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Remove non-ascii characters from the transcript\n",
|
||||
"ascii_transcript = remove_non_ascii(full_transcript)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 17,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Good afternoon, everyone. And welcome to FinTech Plus Sync's second quarter 2023 earnings call. I'm John Doe, CEO of FinTech Plus. We've had a stellar Q2 with a revenue of 125 million, a 25% increase year over year. Our gross profit margin stands at a solid 58%, due in part to cost efficiencies gained from our scalable business model. Our EBITDA has surged to 37.5 million, translating to a remarkable 30% EBITDA margin. Our net income for the quarter rose to 16 million, which is a noteworthy increase from 10 million in Q2 2022. Our total addressable market has grown substantially thanks to the expansion of our high yield savings product line and the new RoboAdvisor platform. We've been diversifying our asset-backed securities portfolio, investing heavily in collateralized. debt obligations, and residential mortgage-backed securities. We've also invested $25 million in AAA rated corporate bonds, enhancing our risk adjusted returns. As for our balance sheet, total assets reached $1.5 billion with total liabilities at $900 million, leaving us with a solid equity base of $600 million. Our debt-to-equity ratio stands at 1.5, a healthy figure considering our expansionary phase. We continue to see substantial organic user growth, with customer acquisition cost dropping by 15% and lifetime value growing by 25%. Our LTVCAC ratio is at an impressive 3.5%. In terms of risk management, we have a value-at-risk model in place with a 99%... confidence level indicating that our maximum loss will not exceed 5 million in the next trading day. We've adopted a conservative approach to managing our leverage and have a healthy tier one capital ratio of 12.5%. Our forecast for the coming quarter is positive. We expect revenue to be around 135 million and 8% quarter over quarter growth driven primarily by our cutting edge blockchain solutions and AI driven predictive analytics. We're also excited about the upcoming IPO of our FinTech subsidiary, Pay Plus, which we expect to raise 200 million. Significantly bolstering our liquidity and paving the way for aggressive growth strategies. We thank our shareholders for their continued faith in us and we look forward to an even more successful Q3. Thank you so much.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(ascii_transcript)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Use punctuation assistant function\n",
|
||||
"response = punctuation_assistant(ascii_transcript)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 19,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Extract the punctuated transcript from the model's response\n",
|
||||
"punctuated_transcript = response['choices'][0]['message']['content']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 20,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Good afternoon, everyone. And welcome to FinTech Plus Sync's second quarter 2023 earnings call. I'm John Doe, CEO of FinTech Plus. We've had a stellar Q2 with a revenue of $125 million, a 25% increase year over year. Our gross profit margin stands at a solid 58%, due in part to cost efficiencies gained from our scalable business model. Our EBITDA has surged to $37.5 million, translating to a remarkable 30% EBITDA margin. Our net income for the quarter rose to $16 million, which is a noteworthy increase from $10 million in Q2 2022. Our total addressable market has grown substantially thanks to the expansion of our high yield savings product line and the new RoboAdvisor platform. We've been diversifying our asset-backed securities portfolio, investing heavily in collateralized debt obligations, and residential mortgage-backed securities. We've also invested $25 million in AAA rated corporate bonds, enhancing our risk-adjusted returns. As for our balance sheet, total assets reached $1.5 billion with total liabilities at $900 million, leaving us with a solid equity base of $600 million. Our debt-to-equity ratio stands at 1.5, a healthy figure considering our expansionary phase. We continue to see substantial organic user growth, with customer acquisition cost dropping by 15% and lifetime value growing by 25%. Our LTVCAC ratio is at an impressive 3.5%. In terms of risk management, we have a value-at-risk model in place with a 99% confidence level indicating that our maximum loss will not exceed $5 million in the next trading day. We've adopted a conservative approach to managing our leverage and have a healthy tier one capital ratio of 12.5%. Our forecast for the coming quarter is positive. We expect revenue to be around $135 million and 8% quarter over quarter growth driven primarily by our cutting-edge blockchain solutions and AI-driven predictive analytics. We're also excited about the upcoming IPO of our FinTech subsidiary, Pay Plus, which we expect to raise $200 million, significantly bolstering our liquidity and paving the way for aggressive growth strategies. We thank our shareholders for their continued faith in us and we look forward to an even more successful Q3. Thank you so much.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(punctuated_transcript)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 21,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Use product assistant function\n",
|
||||
"response = product_assistant(punctuated_transcript)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 22,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Extract the final transcript from the model's response\n",
|
||||
"final_transcript = response['choices'][0]['message']['content']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 23,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Good afternoon, everyone. And welcome to FinTech Plus Sync's second quarter 2023 earnings call. I'm John Doe, CEO of FinTech Plus. We've had a stellar second quarter (Q2) with a revenue of $125 million, a 25% increase year over year. Our gross profit margin stands at a solid 58%, due in part to cost efficiencies gained from our scalable business model. Our Earnings Before Interest, Taxes, Depreciation, and Amortization (EBITDA) has surged to $37.5 million, translating to a remarkable 30% EBITDA margin. Our net income for the quarter rose to $16 million, which is a noteworthy increase from $10 million in Q2 2022. Our total addressable market has grown substantially thanks to the expansion of our high yield savings product line and the new RoboAdvisor platform. We've been diversifying our asset-backed securities portfolio, investing heavily in Collateralized Debt Obligations (CDOs), and Residential Mortgage-Backed Securities (RMBS). We've also invested $25 million in AAA rated corporate bonds, enhancing our risk-adjusted returns. As for our balance sheet, total assets reached $1.5 billion with total liabilities at $900 million, leaving us with a solid equity base of $600 million. Our debt-to-equity ratio stands at 1.5, a healthy figure considering our expansionary phase. We continue to see substantial organic user growth, with Customer Acquisition Cost (CAC) dropping by 15% and Lifetime Value (LTV) growing by 25%. Our LTV to CAC (LTVCAC) ratio is at an impressive 3.5%. In terms of risk management, we have a Value at Risk (VaR) model in place with a 99% confidence level indicating that our maximum loss will not exceed $5 million in the next trading day. We've adopted a conservative approach to managing our leverage and have a healthy Tier 1 Capital Ratio of 12.5%. Our forecast for the coming quarter is positive. We expect revenue to be around $135 million and 8% quarter over quarter growth driven primarily by our cutting-edge blockchain solutions and AI-driven predictive analytics. We're also excited about the upcoming Initial Public Offering (IPO) of our FinTech subsidiary, Pay Plus, which we expect to raise $200 million, significantly bolstering our liquidity and paving the way for aggressive growth strategies. We thank our shareholders for their continued faith in us and we look forward to an even more successful third quarter (Q3). Thank you so much.\n",
|
||||
"\n",
|
||||
"Words changed: \n",
|
||||
"Q2 to second quarter (Q2)\n",
|
||||
"EBITDA to Earnings Before Interest, Taxes, Depreciation, and Amortization (EBITDA)\n",
|
||||
"CDOs to Collateralized Debt Obligations (CDOs)\n",
|
||||
"RMBS to Residential Mortgage-Backed Securities (RMBS)\n",
|
||||
"CAC to Customer Acquisition Cost (CAC)\n",
|
||||
"LTV to Lifetime Value (LTV)\n",
|
||||
"LTVCAC to LTV to CAC (LTVCAC)\n",
|
||||
"VaR to Value at Risk (VaR)\n",
|
||||
"IPO to Initial Public Offering (IPO)\n",
|
||||
"Q3 to third quarter (Q3)\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(final_transcript)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.9"
|
||||
},
|
||||
"orig_nbformat": 4
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
Loading…
Reference in New Issue