mirror of
https://github.com/hwchase17/langchain
synced 2024-11-20 03:25:56 +00:00
4092fd21dc
This introduces the `YoutubeAudioLoader`, which will load blobs from a YouTube url and write them. Blobs are then parsed by `OpenAIWhisperParser()`, as show in this [PR](https://github.com/hwchase17/langchain/pull/5580), but we extend the parser to split audio such that each chuck meets the 25MB OpenAI size limit. As shown in the notebook, this enables a very simple UX: ``` # Transcribe the video to text loader = GenericLoader(YoutubeAudioLoader([url],save_dir),OpenAIWhisperParser()) docs = loader.load() ``` Tested on full set of Karpathy lecture videos: ``` # Karpathy lecture videos urls = ["https://youtu.be/VMj-3S1tku0" "https://youtu.be/PaCmpygFfXo", "https://youtu.be/TCH_1BHY58I", "https://youtu.be/P6sfmUTpUmc", "https://youtu.be/q8SA3rM6ckI", "https://youtu.be/t3YJ5hKiMQ0", "https://youtu.be/kCc8FmEb1nY"] # Directory to save audio files save_dir = "~/Downloads/YouTube" # Transcribe the videos to text loader = GenericLoader(YoutubeAudioLoader(urls,save_dir),OpenAIWhisperParser()) docs = loader.load() ``` |
||
|---|---|---|
| .. | ||
| blob_loaders | ||
| loaders | ||
| parsers | ||
| sample_documents | ||
| test_docs | ||
| __init__.py | ||
| test_base.py | ||
| test_bibtex.py | ||
| test_bshtml.py | ||
| test_confluence.py | ||
| test_csv_loader.py | ||
| test_detect_encoding.py | ||
| test_directory.py | ||
| test_evernote_loader.py | ||
| test_generic_loader.py | ||
| test_github.py | ||
| test_json_loader.py | ||
| test_psychic.py | ||
| test_readthedoc.py | ||
| test_telegram.py | ||
| test_trello.py | ||
| test_web_base.py | ||
| test_youtube.py | ||