mirror of
https://github.com/hwchase17/langchain
synced 2024-11-10 01:10:59 +00:00
4092fd21dc
This introduces the `YoutubeAudioLoader`, which will load blobs from a YouTube url and write them. Blobs are then parsed by `OpenAIWhisperParser()`, as show in this [PR](https://github.com/hwchase17/langchain/pull/5580), but we extend the parser to split audio such that each chuck meets the 25MB OpenAI size limit. As shown in the notebook, this enables a very simple UX: ``` # Transcribe the video to text loader = GenericLoader(YoutubeAudioLoader([url],save_dir),OpenAIWhisperParser()) docs = loader.load() ``` Tested on full set of Karpathy lecture videos: ``` # Karpathy lecture videos urls = ["https://youtu.be/VMj-3S1tku0" "https://youtu.be/PaCmpygFfXo", "https://youtu.be/TCH_1BHY58I", "https://youtu.be/P6sfmUTpUmc", "https://youtu.be/q8SA3rM6ckI", "https://youtu.be/t3YJ5hKiMQ0", "https://youtu.be/kCc8FmEb1nY"] # Directory to save audio files save_dir = "~/Downloads/YouTube" # Transcribe the videos to text loader = GenericLoader(YoutubeAudioLoader(urls,save_dir),OpenAIWhisperParser()) docs = loader.load() ``` |
||
|---|---|---|
| .. | ||
| agents | ||
| callbacks | ||
| chains | ||
| chat_models | ||
| client | ||
| data | ||
| docstore | ||
| document_loaders | ||
| evaluation | ||
| examples | ||
| llms | ||
| memory | ||
| output_parsers | ||
| prompts | ||
| retrievers | ||
| tools | ||
| utilities | ||
| vectorstores | ||
| __init__.py | ||
| conftest.py | ||
| test_bash.py | ||
| test_dependencies.py | ||
| test_document_transformers.py | ||
| test_formatting.py | ||
| test_math_utils.py | ||
| test_pytest_config.py | ||
| test_python.py | ||
| test_schema.py | ||
| test_sql_database_schema.py | ||
| test_sql_database.py | ||
| test_text_splitter.py | ||