[cleanup] Misc (#5044)

Authored by: gamer191, pukkandan
pull/4687/merge
gamer191 2 years ago committed by GitHub
parent 878eac3e2e
commit 304ad45a9b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

5
.gitignore vendored

@ -33,13 +33,14 @@ cookies
*.jpeg *.jpeg
*.jpg *.jpg
*.m4a *.m4a
*.mpga
*.m4v *.m4v
*.mhtml *.mhtml
*.mkv *.mkv
*.mov *.mov
*.mp3 *.mp3
*.mp4 *.mp4
*.mpga
*.oga
*.ogg *.ogg
*.opus *.opus
*.png *.png
@ -47,6 +48,7 @@ cookies
*.srt *.srt
*.swf *.swf
*.swp *.swp
*.tt
*.ttml *.ttml
*.url *.url
*.vtt *.vtt
@ -85,6 +87,7 @@ updates_key.pem
.tox .tox
*.class *.class
*.isorted *.isorted
*.stackdump
# Generated # Generated
AUTHORS AUTHORS

@ -17,8 +17,8 @@ pypi-files: AUTHORS Changelog.md LICENSE README.md README.txt supportedsites \
clean-test: clean-test:
rm -rf test/testdata/sigs/player-*.js tmp/ *.annotations.xml *.aria2 *.description *.dump *.frag \ rm -rf test/testdata/sigs/player-*.js tmp/ *.annotations.xml *.aria2 *.description *.dump *.frag \
*.frag.aria2 *.frag.urls *.info.json *.live_chat.json *.meta *.part* *.tmp *.temp *.unknown_video *.ytdl \ *.frag.aria2 *.frag.urls *.info.json *.live_chat.json *.meta *.part* *.tmp *.temp *.unknown_video *.ytdl \
*.3gp *.ape *.ass *.avi *.desktop *.f4v *.flac *.flv *.jpeg *.jpg *.m4a *.mpga *.m4v *.mhtml *.mkv *.mov \ *.3gp *.ape *.ass *.avi *.desktop *.f4v *.flac *.flv *.jpeg *.jpg *.m4a *.m4v *.mhtml *.mkv *.mov *.mp3 *.mp4 \
*.mp3 *.mp4 *.ogg *.opus *.png *.sbv *.srt *.swf *.swp *.ttml *.url *.vtt *.wav *.webloc *.webm *.webp *.mpga *.oga *.ogg *.opus *.png *.sbv *.srt *.swf *.swp *.tt *.ttml *.url *.vtt *.wav *.webloc *.webm *.webp
clean-dist: clean-dist:
rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ \ rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ \
yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS .mailmap yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS .mailmap

@ -48,6 +48,7 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t
* [SponsorBlock Options](#sponsorblock-options) * [SponsorBlock Options](#sponsorblock-options)
* [Extractor Options](#extractor-options) * [Extractor Options](#extractor-options)
* [CONFIGURATION](#configuration) * [CONFIGURATION](#configuration)
* [Configuration file encoding](#configuration-file-encoding)
* [Authentication with .netrc file](#authentication-with-netrc-file) * [Authentication with .netrc file](#authentication-with-netrc-file)
* [Notes about environment variables](#notes-about-environment-variables) * [Notes about environment variables](#notes-about-environment-variables)
* [OUTPUT TEMPLATE](#output-template) * [OUTPUT TEMPLATE](#output-template)
@ -75,7 +76,7 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t
* Merged with **youtube-dl v2021.12.17+ [commit/ed5c44e](https://github.com/ytdl-org/youtube-dl/commit/ed5c44e7b74ac77f87ca5ed6cb5e964a0c6a0678)**<!--([exceptions](https://github.com/yt-dlp/yt-dlp/issues/21))--> and **youtube-dlc v2020.11.11-3+ [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl) * Merged with **youtube-dl v2021.12.17+ [commit/ed5c44e](https://github.com/ytdl-org/youtube-dl/commit/ed5c44e7b74ac77f87ca5ed6cb5e964a0c6a0678)**<!--([exceptions](https://github.com/yt-dlp/yt-dlp/issues/21))--> and **youtube-dlc v2020.11.11-3+ [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl)
* **[SponsorBlock Integration](#sponsorblock-options)**: You can mark/remove sponsor sections in youtube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API * **[SponsorBlock Integration](#sponsorblock-options)**: You can mark/remove sponsor sections in YouTube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API
* **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection than what is possible by simply using `--format` ([examples](#format-selection-examples)) * **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection than what is possible by simply using `--format` ([examples](#format-selection-examples))
@ -89,7 +90,7 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t
* `255kbps` audio is extracted (if available) from YouTube Music when premium cookies are given * `255kbps` audio is extracted (if available) from YouTube Music when premium cookies are given
* Redirect channel's home URL automatically to `/video` to preserve the old behaviour * Redirect channel's home URL automatically to `/video` to preserve the old behaviour
* **Cookies from browser**: Cookies can be automatically extracted from all major web browsers using `--cookies-from-browser BROWSER[+KEYRING][:PROFILE]` * **Cookies from browser**: Cookies can be automatically extracted from all major web browsers using `--cookies-from-browser BROWSER[+KEYRING][:PROFILE][::CONTAINER]`
* **Download time range**: Videos can be downloaded partially based on either timestamps or chapters using `--download-sections` * **Download time range**: Videos can be downloaded partially based on either timestamps or chapters using `--download-sections`
@ -141,8 +142,8 @@ Some of yt-dlp's default options are different from that of youtube-dl and youtu
* `playlist_index` behaves differently when used with options like `--playlist-reverse` and `--playlist-items`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details. You can use `--compat-options playlist-index` if you want to keep the earlier behavior * `playlist_index` behaves differently when used with options like `--playlist-reverse` and `--playlist-items`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details. You can use `--compat-options playlist-index` if you want to keep the earlier behavior
* The output of `-F` is listed in a new format. Use `--compat-options list-formats` to revert this * The output of `-F` is listed in a new format. Use `--compat-options list-formats` to revert this
* Live chats (if available) are considered as subtitles. Use `--sub-langs all,-live_chat` to download all subtitles except live chat. You can also use `--compat-options no-live-chat` to prevent any live chat/danmaku from downloading * Live chats (if available) are considered as subtitles. Use `--sub-langs all,-live_chat` to download all subtitles except live chat. You can also use `--compat-options no-live-chat` to prevent any live chat/danmaku from downloading
* Youtube channel URLs are automatically redirected to `/video`. Append a `/featured` to the URL to download only the videos in the home page. If the channel does not have a videos tab, we try to download the equivalent `UU` playlist instead. For all other tabs, if the channel does not show the requested tab, an error will be raised. Also, `/live` URLs raise an error if there are no live videos instead of silently downloading the entire channel. You may use `--compat-options no-youtube-channel-redirect` to revert all these redirections * YouTube channel URLs are automatically redirected to `/video`. Append a `/featured` to the URL to download only the videos in the home page. If the channel does not have a videos tab, we try to download the equivalent `UU` playlist instead. For all other tabs, if the channel does not show the requested tab, an error will be raised. Also, `/live` URLs raise an error if there are no live videos instead of silently downloading the entire channel. You may use `--compat-options no-youtube-channel-redirect` to revert all these redirections
* Unavailable videos are also listed for youtube playlists. Use `--compat-options no-youtube-unavailable-videos` to remove this * Unavailable videos are also listed for YouTube playlists. Use `--compat-options no-youtube-unavailable-videos` to remove this
* The upload dates extracted from YouTube are in UTC [when available](https://github.com/yt-dlp/yt-dlp/blob/89e4d86171c7b7c997c77d4714542e0383bf0db0/yt_dlp/extractor/youtube.py#L3898-L3900). Use `--compat-options no-youtube-prefer-utc-upload-date` to prefer the non-UTC upload date. * The upload dates extracted from YouTube are in UTC [when available](https://github.com/yt-dlp/yt-dlp/blob/89e4d86171c7b7c997c77d4714542e0383bf0db0/yt_dlp/extractor/youtube.py#L3898-L3900). Use `--compat-options no-youtube-prefer-utc-upload-date` to prefer the non-UTC upload date.
* If `ffmpeg` is used as the downloader, the downloading and merging of formats happen in a single step when possible. Use `--compat-options no-direct-merge` to revert this * If `ffmpeg` is used as the downloader, the downloading and merging of formats happen in a single step when possible. Use `--compat-options no-direct-merge` to revert this
* Thumbnail embedding in `mp4` is done with mutagen if possible. Use `--compat-options embed-thumbnail-atomicparsley` to force the use of AtomicParsley instead * Thumbnail embedding in `mp4` is done with mutagen if possible. Use `--compat-options embed-thumbnail-atomicparsley` to force the use of AtomicParsley instead
@ -303,7 +304,7 @@ If you wish to build it anyway, install Python and py2exe, and then simply run `
* **`devscripts/set-variant.py variant [-M update_message]`** - Set the build variant of the executable * **`devscripts/set-variant.py variant [-M update_message]`** - Set the build variant of the executable
* **`devscripts/make_lazy_extractors.py`** - Create lazy extractors. Running this before building the binaries (any variant) will improve their startup performance. Set the environment variable `YTDLP_NO_LAZY_EXTRACTORS=1` if you wish to forcefully disable lazy extractor loading. * **`devscripts/make_lazy_extractors.py`** - Create lazy extractors. Running this before building the binaries (any variant) will improve their startup performance. Set the environment variable `YTDLP_NO_LAZY_EXTRACTORS=1` if you wish to forcefully disable lazy extractor loading.
You can also fork the project on github and run your fork's [build workflow](.github/workflows/build.yml) to automatically build a full release You can also fork the project on GitHub and run your fork's [build workflow](.github/workflows/build.yml) to automatically build a full release
# USAGE AND OPTIONS # USAGE AND OPTIONS
@ -1129,15 +1130,15 @@ Note that options in configuration file are just the same options aka switches u
You can use `--ignore-config` if you want to disable all configuration files for a particular yt-dlp run. If `--ignore-config` is found inside any configuration file, no further configuration will be loaded. For example, having the option in the portable configuration file prevents loading of home, user, and system configurations. Additionally, (for backward compatibility) if `--ignore-config` is found inside the system configuration file, the user configuration is not loaded. You can use `--ignore-config` if you want to disable all configuration files for a particular yt-dlp run. If `--ignore-config` is found inside any configuration file, no further configuration will be loaded. For example, having the option in the portable configuration file prevents loading of home, user, and system configurations. Additionally, (for backward compatibility) if `--ignore-config` is found inside the system configuration file, the user configuration is not loaded.
### Config file encoding ### Configuration file encoding
The config files are decoded according to the UTF BOM if present, and in the encoding from system locale otherwise. The configuration files are decoded according to the UTF BOM if present, and in the encoding from system locale otherwise.
If you want your file to be decoded differently, add `# coding: ENCODING` to the beginning of the file (e.g. `# coding: shift-jis`). There must be no characters before that, even spaces or BOM. If you want your file to be decoded differently, add `# coding: ENCODING` to the beginning of the file (e.g. `# coding: shift-jis`). There must be no characters before that, even spaces or BOM.
### Authentication with `.netrc` file ### Authentication with `.netrc` file
You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with `--username` and `--password`) in order not to pass credentials as command line arguments on every yt-dlp execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a [`.netrc` file](https://stackoverflow.com/tags/.netrc/info) on a per extractor basis. For that you will need to create a `.netrc` file in `--netrc-location` and restrict permissions to read/write by only you: You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with `--username` and `--password`) in order not to pass credentials as command line arguments on every yt-dlp execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a [`.netrc` file](https://stackoverflow.com/tags/.netrc/info) on a per-extractor basis. For that you will need to create a `.netrc` file in `--netrc-location` and restrict permissions to read/write by only you:
``` ```
touch ${HOME}/.netrc touch ${HOME}/.netrc
chmod a-rwx,u+rw ${HOME}/.netrc chmod a-rwx,u+rw ${HOME}/.netrc
@ -1184,7 +1185,7 @@ The field names themselves (the part inside the parenthesis) can also have some
1. **Alternatives**: Alternate fields can be specified separated with a `,`. E.g. `%(release_date>%Y,upload_date>%Y|Unknown)s` 1. **Alternatives**: Alternate fields can be specified separated with a `,`. E.g. `%(release_date>%Y,upload_date>%Y|Unknown)s`
1. **Replacement**: A replacement value can specified using a `&` separator. If the field is *not* empty, this replacement value will be used instead of the actual field content. This is done after alternate fields are considered; thus the replacement is used if *any* of the alternative fields is *not* empty. 1. **Replacement**: A replacement value can be specified using a `&` separator. If the field is *not* empty, this replacement value will be used instead of the actual field content. This is done after alternate fields are considered; thus the replacement is used if *any* of the alternative fields is *not* empty.
1. **Default**: A literal default value can be specified for when the field is empty using a `|` separator. This overrides `--output-na-placeholder`. E.g. `%(uploader|Unknown)s` 1. **Default**: A literal default value can be specified for when the field is empty using a `|` separator. This overrides `--output-na-placeholder`. E.g. `%(uploader|Unknown)s`
@ -1411,7 +1412,7 @@ For example, to download the worst quality video-only format you can use `-f wor
You can select the n'th best format of a type by using `best<type>.<n>`. For example, `best.2` will select the 2nd best combined format. Similarly, `bv*.3` will select the 3rd best format that contains a video stream. You can select the n'th best format of a type by using `best<type>.<n>`. For example, `best.2` will select the 2nd best combined format. Similarly, `bv*.3` will select the 3rd best format that contains a video stream.
If you want to download multiple videos and they don't have the same formats available, you can specify the order of preference using slashes. Note that formats on the left hand side are preferred; e.g. `-f 22/17/18` will download format 22 if it's available, otherwise it will download format 17 if it's available, otherwise it will download format 18 if it's available, otherwise it will complain that no suitable formats are available for download. If you want to download multiple videos, and they don't have the same formats available, you can specify the order of preference using slashes. Note that formats on the left hand side are preferred; e.g. `-f 22/17/18` will download format 22 if it's available, otherwise it will download format 17 if it's available, otherwise it will download format 18 if it's available, otherwise it will complain that no suitable formats are available for download.
If you want to download several formats of the same video use a comma as a separator, e.g. `-f 22,17,18` will download all these three formats, of course if they are available. Or a more sophisticated example combined with the precedence feature: `-f 136/137/mp4/bestvideo,140/m4a/bestaudio`. If you want to download several formats of the same video use a comma as a separator, e.g. `-f 22,17,18` will download all these three formats, of course if they are available. Or a more sophisticated example combined with the precedence feature: `-f 136/137/mp4/bestvideo,140/m4a/bestaudio`.
@ -1419,7 +1420,7 @@ You can merge the video and audio of multiple formats into a single file using `
**Deprecation warning**: Since the *below* described behavior is complex and counter-intuitive, this will be removed and multistreams will be enabled by default in the future. A new operator will be instead added to limit formats to single audio/video **Deprecation warning**: Since the *below* described behavior is complex and counter-intuitive, this will be removed and multistreams will be enabled by default in the future. A new operator will be instead added to limit formats to single audio/video
Unless `--video-multistreams` is used, all formats with a video stream except the first one are ignored. Similarly, unless `--audio-multistreams` is used, all formats with an audio stream except the first one are ignored. E.g. `-f bestvideo+best+bestaudio --video-multistreams --audio-multistreams` will download and merge all 3 given formats. The resulting file will have 2 video streams and 2 audio streams. But `-f bestvideo+best+bestaudio --no-video-multistreams` will download and merge only `bestvideo` and `bestaudio`. `best` is ignored since another format containing a video stream (`bestvideo`) has already been selected. The order of the formats is therefore important. `-f best+bestaudio --no-audio-multistreams` will download and merge both formats while `-f bestaudio+best --no-audio-multistreams` will ignore `best` and download only `bestaudio`. Unless `--video-multistreams` is used, all formats with a video stream except the first one are ignored. Similarly, unless `--audio-multistreams` is used, all formats with an audio stream except the first one are ignored. E.g. `-f bestvideo+best+bestaudio --video-multistreams --audio-multistreams` will download and merge all 3 given formats. The resulting file will have 2 video streams and 2 audio streams. But `-f bestvideo+best+bestaudio --no-video-multistreams` will download and merge only `bestvideo` and `bestaudio`. `best` is ignored since another format containing a video stream (`bestvideo`) has already been selected. The order of the formats is therefore important. `-f best+bestaudio --no-audio-multistreams` will download only `best` while `-f bestaudio+best --no-audio-multistreams` will ignore `best` and download only `bestaudio`.
## Filtering Formats ## Filtering Formats
@ -1468,8 +1469,8 @@ You can change the criteria for being considered the `best` by using `-S` (`--fo
The available fields are: The available fields are:
- `hasvid`: Gives priority to formats that has a video stream - `hasvid`: Gives priority to formats that have a video stream
- `hasaud`: Gives priority to formats that has a audio stream - `hasaud`: Gives priority to formats that have an audio stream
- `ie_pref`: The format preference - `ie_pref`: The format preference
- `lang`: The language preference - `lang`: The language preference
- `quality`: The quality of the format - `quality`: The quality of the format
@ -1711,7 +1712,7 @@ The following extractors use this feature:
#### youtube #### youtube
* `lang`: Language code to prefer translated metadata of this language (case-sensitive). By default, the video primary language metadata is preferred, with a fallback to `en` translated. See [youtube.py](https://github.com/yt-dlp/yt-dlp/blob/c26f9b991a0681fd3ea548d535919cec1fbbd430/yt_dlp/extractor/youtube.py#L381-L390) for list of supported content language codes * `lang`: Language code to prefer translated metadata of this language (case-sensitive). By default, the video primary language metadata is preferred, with a fallback to `en` translated. See [youtube.py](https://github.com/yt-dlp/yt-dlp/blob/c26f9b991a0681fd3ea548d535919cec1fbbd430/yt_dlp/extractor/youtube.py#L381-L390) for list of supported content language codes
* `skip`: One or more of `hls`, `dash` or `translated_subs` to skip extraction of the m3u8 manifests, dash manifests and [auto-translated subtitles](https://github.com/yt-dlp/yt-dlp/issues/4090#issuecomment-1158102032) respectively * `skip`: One or more of `hls`, `dash` or `translated_subs` to skip extraction of the m3u8 manifests, dash manifests and [auto-translated subtitles](https://github.com/yt-dlp/yt-dlp/issues/4090#issuecomment-1158102032) respectively
* `player_client`: Clients to extract video data from. The main clients are `web`, `android` and `ios` with variants `_music`, `_embedded`, `_embedscreen`, `_creator` (e.g. `web_embedded`); and `mweb` and `tv_embedded` (agegate bypass) with no variants. By default, `android,web` is used, but `tv_embedded` and `creator` variants are added as required for age-gated videos. Similarly the music variants are added for `music.youtube.com` urls. You can use `all` to use all the clients, and `default` for the default clients. * `player_client`: Clients to extract video data from. The main clients are `web`, `android` and `ios` with variants `_music`, `_embedded`, `_embedscreen`, `_creator` (e.g. `web_embedded`); and `mweb` and `tv_embedded` (agegate bypass) with no variants. By default, `android,web` is used, but `tv_embedded` and `creator` variants are added as required for age-gated videos. Similarly, the music variants are added for `music.youtube.com` urls. You can use `all` to use all the clients, and `default` for the default clients.
* `player_skip`: Skip some network requests that are generally needed for robust extraction. One or more of `configs` (skip client configs), `webpage` (skip initial webpage), `js` (skip js player). While these options can help reduce the number of requests needed or avoid some rate-limiting, they could cause some issues. See [#860](https://github.com/yt-dlp/yt-dlp/pull/860) for more details * `player_skip`: Skip some network requests that are generally needed for robust extraction. One or more of `configs` (skip client configs), `webpage` (skip initial webpage), `js` (skip js player). While these options can help reduce the number of requests needed or avoid some rate-limiting, they could cause some issues. See [#860](https://github.com/yt-dlp/yt-dlp/pull/860) for more details
* `comment_sort`: `top` or `new` (default) - choose comment sorting mode (on YouTube's side) * `comment_sort`: `top` or `new` (default) - choose comment sorting mode (on YouTube's side)
* `max_comments`: Limit the amount of comments to gather. Comma-separated list of integers representing `max-comments,max-parents,max-replies,max-replies-per-thread`. Default is `all,all,all,all` * `max_comments`: Limit the amount of comments to gather. Comma-separated list of integers representing `max-comments,max-parents,max-replies,max-replies-per-thread`. Default is `all,all,all,all`
@ -1725,11 +1726,11 @@ The following extractors use this feature:
* `approximate_date`: Extract approximate `upload_date` in flat-playlist. This may cause date-based filters to be slightly off * `approximate_date`: Extract approximate `upload_date` in flat-playlist. This may cause date-based filters to be slightly off
#### funimation #### funimation
* `language`: Languages to extract, e.g. `funimation:language=english,japanese` * `language`: Audio languages to extract, e.g. `funimation:language=english,japanese`
* `version`: The video version to extract - `uncut` or `simulcast` * `version`: The video version to extract - `uncut` or `simulcast`
#### crunchyroll #### crunchyroll
* `language`: Languages to extract, e.g. `crunchyroll:language=jaJp` * `language`: Audio languages to extract, e.g. `crunchyroll:language=jaJp`
* `hardsub`: Which hard-sub versions to extract, e.g. `crunchyroll:hardsub=None,enUS` * `hardsub`: Which hard-sub versions to extract, e.g. `crunchyroll:hardsub=None,enUS`
#### crunchyrollbeta #### crunchyrollbeta

@ -84,7 +84,7 @@ class AcFunVideoIE(AcFunVideoBaseIE):
video_id = self._match_id(url) video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id) webpage = self._download_webpage(url, video_id)
json_all = self._search_json(r'window.videoInfo\s*=\s*', webpage, 'videoInfo', video_id) json_all = self._search_json(r'window.videoInfo\s*=', webpage, 'videoInfo', video_id)
title = json_all.get('title') title = json_all.get('title')
video_list = json_all.get('videoList') or [] video_list = json_all.get('videoList') or []
@ -164,7 +164,7 @@ class AcFunBangumiIE(AcFunVideoBaseIE):
video_id = f'{video_id}{format_field(ac_idx, template="__%s")}' video_id = f'{video_id}{format_field(ac_idx, template="__%s")}'
webpage = self._download_webpage(url, video_id) webpage = self._download_webpage(url, video_id)
json_bangumi_data = self._search_json(r'window.bangumiData\s*=\s*', webpage, 'bangumiData', video_id) json_bangumi_data = self._search_json(r'window.bangumiData\s*=', webpage, 'bangumiData', video_id)
if ac_idx: if ac_idx:
video_info = json_bangumi_data['hlVideoInfo'] video_info = json_bangumi_data['hlVideoInfo']
@ -181,7 +181,7 @@ class AcFunBangumiIE(AcFunVideoBaseIE):
if v.get('id') == season_id), 1) if v.get('id') == season_id), 1)
json_bangumi_list = self._search_json( json_bangumi_list = self._search_json(
r'window\.bangumiList\s*=\s*', webpage, 'bangumiList', video_id, fatal=False) r'window\.bangumiList\s*=', webpage, 'bangumiList', video_id, fatal=False)
video_internal_id = int_or_none(traverse_obj(json_bangumi_data, ('currentVideoInfo', 'id'))) video_internal_id = int_or_none(traverse_obj(json_bangumi_data, ('currentVideoInfo', 'id')))
episode_number = video_internal_id and next(( episode_number = video_internal_id and next((
idx for idx, v in enumerate(json_bangumi_list.get('items') or [], 1) idx for idx, v in enumerate(json_bangumi_list.get('items') or [], 1)

@ -10,11 +10,11 @@ from ..aes import aes_encrypt
from ..utils import ( from ..utils import (
bytes_to_intlist, bytes_to_intlist,
determine_ext, determine_ext,
intlist_to_bytes,
int_or_none, int_or_none,
intlist_to_bytes,
join_nonempty, join_nonempty,
strip_jsonp,
smuggle_url, smuggle_url,
strip_jsonp,
traverse_obj, traverse_obj,
unescapeHTML, unescapeHTML,
unsmuggle_url, unsmuggle_url,

@ -1,10 +1,5 @@
from .common import InfoExtractor from .common import InfoExtractor
from ..utils import ( from ..utils import clean_html, float_or_none, traverse_obj, unescapeHTML
clean_html,
float_or_none,
unescapeHTML,
traverse_obj,
)
class AudioBoomIE(InfoExtractor): class AudioBoomIE(InfoExtractor):

@ -5,16 +5,16 @@ import time
from .common import InfoExtractor from .common import InfoExtractor
from ..compat import compat_str from ..compat import compat_str
from ..utils import ( from ..utils import (
KNOWN_EXTENSIONS,
ExtractorError, ExtractorError,
float_or_none, float_or_none,
int_or_none, int_or_none,
KNOWN_EXTENSIONS,
parse_filesize, parse_filesize,
str_or_none, str_or_none,
try_get, try_get,
update_url_query,
unified_strdate, unified_strdate,
unified_timestamp, unified_timestamp,
update_url_query,
url_or_none, url_or_none,
urljoin, urljoin,
) )

@ -1,6 +1,7 @@
import json import json
import re import re
from .common import InfoExtractor
from ..utils import ( from ..utils import (
int_or_none, int_or_none,
traverse_obj, traverse_obj,
@ -8,7 +9,6 @@ from ..utils import (
unescapeHTML, unescapeHTML,
unified_timestamp, unified_timestamp,
) )
from .common import InfoExtractor
class HRFernsehenIE(InfoExtractor): class HRFernsehenIE(InfoExtractor):

@ -54,7 +54,7 @@ class HuyaLiveIE(InfoExtractor):
def _real_extract(self, url): def _real_extract(self, url):
video_id = self._match_id(url) video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id=video_id) webpage = self._download_webpage(url, video_id=video_id)
stream_data = self._search_json(r'stream:\s+', webpage, 'stream', video_id=video_id, default=None) stream_data = self._search_json(r'stream:\s', webpage, 'stream', video_id=video_id, default=None)
room_info = try_get(stream_data, lambda x: x['data'][0]['gameLiveInfo']) room_info = try_get(stream_data, lambda x: x['data'][0]['gameLiveInfo'])
if not room_info: if not room_info:
raise ExtractorError('Can not extract the room info', expected=True) raise ExtractorError('Can not extract the room info', expected=True)

@ -41,7 +41,7 @@ class IltalehtiIE(InfoExtractor):
article_id = self._match_id(url) article_id = self._match_id(url)
webpage = self._download_webpage(url, article_id) webpage = self._download_webpage(url, article_id)
info = self._search_json( info = self._search_json(
r'<script>\s*window.App\s*=\s*', webpage, 'json', article_id, r'<script>\s*window.App\s*=', webpage, 'json', article_id,
transform_source=js_to_json) transform_source=js_to_json)
props = traverse_obj(info, ( props = traverse_obj(info, (
'state', 'articles', ..., 'items', (('main_media', 'properties'), ('body', ..., 'properties')))) 'state', 'articles', ..., 'items', (('main_media', 'properties'), ('body', ..., 'properties'))))

@ -452,7 +452,7 @@ class InstagramIE(InstagramBaseIE):
webpage = self._download_webpage( webpage = self._download_webpage(
f'{url}/embed/', video_id, note='Downloading embed webpage', fatal=False) f'{url}/embed/', video_id, note='Downloading embed webpage', fatal=False)
additional_data = self._search_json( additional_data = self._search_json(
r'window\.__additionalDataLoaded\s*\(\s*[^,]+,\s*', webpage, 'additional data', video_id, fatal=False) r'window\.__additionalDataLoaded\s*\(\s*[^,]+,', webpage, 'additional data', video_id, fatal=False)
if not additional_data and not media: if not additional_data and not media:
self.raise_login_required('Requested content is not available, rate-limit reached or login required') self.raise_login_required('Requested content is not available, rate-limit reached or login required')

@ -57,7 +57,7 @@ class Liputan6IE(InfoExtractor):
webpage = self._download_webpage(url, display_id) webpage = self._download_webpage(url, display_id)
json_data = self._search_json( json_data = self._search_json(
r'window.kmklabs.gtm\s*=\s*', webpage, 'json_data', display_id) r'window.kmklabs.gtm\s*=', webpage, 'json_data', display_id)
video_id = json_data['videos']['video_1']['video_id'] video_id = json_data['videos']['video_1']['video_id']
return self.url_result( return self.url_result(

@ -1,9 +1,5 @@
from .common import InfoExtractor from .common import InfoExtractor
from ..utils import ( from ..utils import int_or_none, traverse_obj, unified_timestamp
int_or_none,
traverse_obj,
unified_timestamp,
)
class MicrosoftEmbedIE(InfoExtractor): class MicrosoftEmbedIE(InfoExtractor):

@ -643,7 +643,7 @@ class NBCStationsIE(InfoExtractor):
webpage = self._download_webpage(url, video_id) webpage = self._download_webpage(url, video_id)
nbc_data = self._search_json( nbc_data = self._search_json(
r'<script>var\s*nbc\s*=\s*', webpage, 'NBC JSON data', video_id) r'<script>var\s*nbc\s*=', webpage, 'NBC JSON data', video_id)
pdk_acct = nbc_data.get('pdkAcct') or 'Yh1nAC' pdk_acct = nbc_data.get('pdkAcct') or 'Yh1nAC'
fw_ssid = traverse_obj(nbc_data, ('video', 'fwSSID')) fw_ssid = traverse_obj(nbc_data, ('video', 'fwSSID'))
fw_network_id = traverse_obj(nbc_data, ('video', 'fwNetworkID'), default='382114') fw_network_id = traverse_obj(nbc_data, ('video', 'fwNetworkID'), default='382114')

@ -2,10 +2,10 @@ import re
from .common import InfoExtractor from .common import InfoExtractor
from ..utils import ( from ..utils import (
clean_html,
ExtractorError, ExtractorError,
js_to_json,
base_url, base_url,
clean_html,
js_to_json,
url_basename, url_basename,
urljoin, urljoin,
) )

@ -7,9 +7,9 @@ from .common import InfoExtractor
from ..utils import ( from ..utils import (
ExtractorError, ExtractorError,
format_field, format_field,
traverse_obj,
int_or_none, int_or_none,
str_or_none, str_or_none,
traverse_obj,
try_get, try_get,
) )

@ -62,7 +62,7 @@ class TVIPlayerIE(InfoExtractor):
webpage = self._download_webpage(url, video_id) webpage = self._download_webpage(url, video_id)
json_data = self._search_json( json_data = self._search_json(
r'<script>\s*jsonData\s*=\s*', webpage, 'json_data', video_id) r'<script>\s*jsonData\s*=', webpage, 'json_data', video_id)
formats, subtitles = self._extract_m3u8_formats_and_subtitles( formats, subtitles = self._extract_m3u8_formats_and_subtitles(
f'{json_data["videoUrl"]}?wmsAuthSign={self.wms_auth_sign_token}', f'{json_data["videoUrl"]}?wmsAuthSign={self.wms_auth_sign_token}',

@ -6,9 +6,9 @@ from ..utils import (
determine_ext, determine_ext,
extract_attributes, extract_attributes,
int_or_none, int_or_none,
lowercase_escape,
try_get, try_get,
url_or_none, url_or_none,
lowercase_escape,
) )

@ -1417,7 +1417,9 @@ def create_parser():
help='Do not load cookies from browser (default)') help='Do not load cookies from browser (default)')
filesystem.add_option( filesystem.add_option(
'--cache-dir', dest='cachedir', default=None, metavar='DIR', '--cache-dir', dest='cachedir', default=None, metavar='DIR',
help='Location in the filesystem where yt-dlp can store some downloaded information (such as client ids and signatures) permanently. By default $XDG_CACHE_HOME/yt-dlp or ~/.cache/yt-dlp') help=(
'Location in the filesystem where yt-dlp can store some downloaded information '
'(such as client ids and signatures) permanently. By default ${XDG_CACHE_HOME}/yt-dlp'))
filesystem.add_option( filesystem.add_option(
'--no-cache-dir', action='store_false', dest='cachedir', '--no-cache-dir', action='store_false', dest='cachedir',
help='Disable filesystem caching') help='Disable filesystem caching')

@ -3180,6 +3180,10 @@ def multipart_encode(data, boundary=None):
return out, content_type return out, content_type
def variadic(x, allowed_types=(str, bytes, dict)):
return x if isinstance(x, collections.abc.Iterable) and not isinstance(x, allowed_types) else (x,)
def dict_get(d, key_or_keys, default=None, skip_false_values=True): def dict_get(d, key_or_keys, default=None, skip_false_values=True):
for val in map(d.get, variadic(key_or_keys)): for val in map(d.get, variadic(key_or_keys)):
if val is not None and (val or not skip_false_values): if val is not None and (val or not skip_false_values):
@ -5446,10 +5450,6 @@ def get_first(obj, keys, **kwargs):
return traverse_obj(obj, (..., *variadic(keys)), **kwargs, get_all=False) return traverse_obj(obj, (..., *variadic(keys)), **kwargs, get_all=False)
def variadic(x, allowed_types=(str, bytes, dict)):
return x if isinstance(x, collections.abc.Iterable) and not isinstance(x, allowed_types) else (x,)
def time_seconds(**kwargs): def time_seconds(**kwargs):
t = datetime.datetime.now(datetime.timezone(datetime.timedelta(**kwargs))) t = datetime.datetime.now(datetime.timezone(datetime.timedelta(**kwargs)))
return t.timestamp() return t.timestamp()

Loading…
Cancel
Save