Archives/langchain
2
0
Fork 1
mirror of https://github.com/hwchase17/langchain synced 2024-11-04 06:00:26 +00:00
Code Issues Packages Projects Releases Wiki Activity
7824291252
langchain/libs/community/tests/unit_tests/document_loaders/test_youtube.py

28 lines
1.3 KiB
Python
Raw Normal View History

Improve video_id extraction in YoutubeLoader (#4452) # Improve video_id extraction in `YoutubeLoader` `YoutubeLoader.from_youtube_url` can only deal with one specific url format. I've introduced `YoutubeLoader.extract_video_id` which can extract video id from common YT urls. Fixes #4451 @eyurtsev --------- Co-authored-by: Kamil Niski <kamil.niski@gmail.com>
2023-05-15 14:45:19 +00:00
import pytest
community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) Moved the following modules to new package langchain-community in a backwards compatible fashion: ``` mv langchain/langchain/adapters community/langchain_community mv langchain/langchain/callbacks community/langchain_community/callbacks mv langchain/langchain/chat_loaders community/langchain_community mv langchain/langchain/chat_models community/langchain_community mv langchain/langchain/document_loaders community/langchain_community mv langchain/langchain/docstore community/langchain_community mv langchain/langchain/document_transformers community/langchain_community mv langchain/langchain/embeddings community/langchain_community mv langchain/langchain/graphs community/langchain_community mv langchain/langchain/llms community/langchain_community mv langchain/langchain/memory/chat_message_histories community/langchain_community mv langchain/langchain/retrievers community/langchain_community mv langchain/langchain/storage community/langchain_community mv langchain/langchain/tools community/langchain_community mv langchain/langchain/utilities community/langchain_community mv langchain/langchain/vectorstores community/langchain_community mv langchain/langchain/agents/agent_toolkits community/langchain_community mv langchain/langchain/cache.py community/langchain_community mv langchain/langchain/adapters community/langchain_community mv langchain/langchain/callbacks community/langchain_community/callbacks mv langchain/langchain/chat_loaders community/langchain_community mv langchain/langchain/chat_models community/langchain_community mv langchain/langchain/document_loaders community/langchain_community mv langchain/langchain/docstore community/langchain_community mv langchain/langchain/document_transformers community/langchain_community mv langchain/langchain/embeddings community/langchain_community mv langchain/langchain/graphs community/langchain_community mv langchain/langchain/llms community/langchain_community mv langchain/langchain/memory/chat_message_histories community/langchain_community mv langchain/langchain/retrievers community/langchain_community mv langchain/langchain/storage community/langchain_community mv langchain/langchain/tools community/langchain_community mv langchain/langchain/utilities community/langchain_community mv langchain/langchain/vectorstores community/langchain_community mv langchain/langchain/agents/agent_toolkits community/langchain_community mv langchain/langchain/cache.py community/langchain_community ``` Moved the following to core ``` mv langchain/langchain/utils/json_schema.py core/langchain_core/utils mv langchain/langchain/utils/html.py core/langchain_core/utils mv langchain/langchain/utils/strings.py core/langchain_core/utils cat langchain/langchain/utils/env.py >> core/langchain_core/utils/env.py rm langchain/langchain/utils/env.py ``` See .scripts/community_split/script_integrations.sh for all changes
2023-12-11 21:53:30 +00:00
from langchain_community.document_loaders import YoutubeLoader
YouTube Loader: Replace regexp with built-in parsing (#4729)
2023-05-15 15:34:41 +00:00
Improve video_id extraction in YoutubeLoader (#4452) # Improve video_id extraction in `YoutubeLoader` `YoutubeLoader.from_youtube_url` can only deal with one specific url format. I've introduced `YoutubeLoader.extract_video_id` which can extract video id from common YT urls. Fixes #4451 @eyurtsev --------- Co-authored-by: Kamil Niski <kamil.niski@gmail.com>
2023-05-15 14:45:19 +00:00
@pytest.mark.parametrize(
"youtube_url, expected_video_id",
[
("http://www.youtube.com/watch?v=-wtIMTCHWuI", "-wtIMTCHWuI"),
("http://youtube.com/watch?v=-wtIMTCHWuI", "-wtIMTCHWuI"),
("http://m.youtube.com/watch?v=-wtIMTCHWuI", "-wtIMTCHWuI"),
("http://youtu.be/-wtIMTCHWuI", "-wtIMTCHWuI"),
("https://youtu.be/-wtIMTCHWuI", "-wtIMTCHWuI"),
("https://www.youtube.com/watch?v=lalOy8Mbfdc", "lalOy8Mbfdc"),
("https://m.youtube.com/watch?v=lalOy8Mbfdc", "lalOy8Mbfdc"),
("https://youtube.com/watch?v=lalOy8Mbfdc", "lalOy8Mbfdc"),
("http://youtu.be/lalOy8Mbfdc?t=1", "lalOy8Mbfdc"),
("http://youtu.be/lalOy8Mbfdc?t=1s", "lalOy8Mbfdc"),
("https://youtu.be/lalOy8Mbfdc?t=1", "lalOy8Mbfdc"),
("http://www.youtube-nocookie.com/embed/lalOy8Mbfdc?rel=0", "lalOy8Mbfdc"),
("https://youtu.be/lalOy8Mbfdc?t=1s", "lalOy8Mbfdc"),
YouTube Loader: Replace regexp with built-in parsing (#4729)
2023-05-15 15:34:41 +00:00
("https://www.youtube.com/shorts/cd0Fy92_w_s", "cd0Fy92_w_s"),
Improve video_id extraction in YoutubeLoader (#4452) # Improve video_id extraction in `YoutubeLoader` `YoutubeLoader.from_youtube_url` can only deal with one specific url format. I've introduced `YoutubeLoader.extract_video_id` which can extract video id from common YT urls. Fixes #4451 @eyurtsev --------- Co-authored-by: Kamil Niski <kamil.niski@gmail.com>
2023-05-15 14:45:19 +00:00
],
)
YouTube Loader: Replace regexp with built-in parsing (#4729)
2023-05-15 15:34:41 +00:00
def test_video_id_extraction(youtube_url: str, expected_video_id: str) -> None:
"""Test that the video id is extracted from a youtube url"""
Improve video_id extraction in YoutubeLoader (#4452) # Improve video_id extraction in `YoutubeLoader` `YoutubeLoader.from_youtube_url` can only deal with one specific url format. I've introduced `YoutubeLoader.extract_video_id` which can extract video id from common YT urls. Fixes #4451 @eyurtsev --------- Co-authored-by: Kamil Niski <kamil.niski@gmail.com>
2023-05-15 14:45:19 +00:00
assert YoutubeLoader.extract_video_id(youtube_url) == expected_video_id
Reference in New Issue Copy Permalink