aboutsummaryrefslogtreecommitdiff
path: root/youtube_dl/utils.py
AgeCommit message (Collapse)Author
12 days[utils] Save `orig_msg` in `ExtractorError`dirkf
2024-05-30[utils] Update traverse_obj() from yt-dlpdirkf
* remove `is_user_input` option per https://github.com/yt-dlp/yt-dlp/pull/8673 * support traversal of compat_xml_etree_ElementTree_Element per https://github.com/yt-dlp/yt-dlp/pull/8911 * allow un/branching using all and any per https://github.com/yt-dlp/yt-dlp/pull/9571 * support traversal of compat_cookies.Morsel and multiple types in `set()` keys per https://github.com/yt-dlp/yt-dlp/pull/9577 thx Grub4k for these * also, move traversal tests to a separate class * allow for unordered dicts in tests for Py<3.7
2024-04-05[utils] Fix crash in _report_ignoring_subs from c58b655 (#32762)Ori Avtalion
Align `utils.bug_reports_message()` with yt-dlp https://github.com/yt-dlp/yt-dlp/commit/5873d4ccdd, thanks fstirlitz --------- Co-authored-by: dirkf <fieldhouse@gmx.net>
2024-03-27[utils] Apply compat_contextlib_suppressdirkf
2024-03-08[utils] Let int_or_none() accept a base, like int()dirkf
2024-03-04[utils] Handle user:pass in URLs (#28801)Hubert Hirtz
* Handle user:pass in URLs Fixes "nonnumeric port" errors when youtube-dl is given URLs with usernames and passwords such as: http://username:password@example.com/myvideo.mp4 Refs: - https://en.wikipedia.org/wiki/Basic_access_authentication - https://tools.ietf.org/html/rfc1738#section-3.1 - https://docs.python.org/3.8/library/urllib.parse.html#urllib.parse.urlsplit Fixes #18276 (point 4) Fixes #20258 Fixes #26211 (see comment) * Align code with yt-dlp --------- Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-11-29[utils] Make restricted filenames ignore characters in Unicode categories ↵dirkf
Mark, Other Resolves #32629
2023-09-03[utils] Revert bbd3e7e, updating docstring, test insteaddirkf
2023-09-03[utils] Properly handle list values in update_url()dirkf
An actual list value in a query update could have been treated as a list of values because of the key:list parse_qs format.
2023-08-01[utils] Advertise optional supported `Content-Encoding`sdirkf
2023-08-01[utils] Add and use `filter_dict()` from yt-dlpdirkf
2023-08-01[utils] Revise `isinstance()` tests (especially for str/unicode/bytes) to ↵dirkf
complete Linter fix
2023-08-01[utils] Avoid comparing `type(var)`, etc, to pass new Linter rulesdirkf
2023-07-29[utils] Rework URL path munging for ., .. componentsdirkf
* move processing to YoutubeDLHandler * also process `Location` header for redirect * use tests from https://github.com/yt-dlp/yt-dlp/pull/7662
2023-07-29[utils] Rework decoding of `Content-Encoding`sdirkf
* support nested encodings * support optional `br` encoding, if brotli package is installed * support optional 'compress' encoding, if ncompress package is installed * response `Content-Encoding` has only unprocessed encodings, or removed * response `Content-Length` is decoded length (usable for filesize metadata) * use zlib for both deflate and gzip decompression * some elements taken from yt-dlp: thx especially coletdjnz
2023-07-25[utils] Fix update_Request() with empty data (not None)dirkf
2023-07-20[utils] Remove stray undocumented Host header in redirect (fix 46fde7c)dirkf
2023-07-19[utils] Fix broken Py 3.11+ compat in `traverse_obj()`dirkf
* inspect.getargspec is missing despite doc claiming backward compat * replace with emulation of `Signature.bind()`
2023-07-19[utils] Minor updates (merge_dicts, T)dirkf
A couple of mods to ease yt-dlp back-ports: * add kwargs to merge_dicts: `unblank=True` (disallow empty string), `rev=False` (reverse the merge list) * add `T(x)` shortcut for `{x}`, unsupported in Py2.6
2023-07-19[utils] Improve js_to_json, align with yt-dlpdirkf
* support variable substitution, from https://github.com/yt-dlp/yt-dlp/pull/#521 etc, thanks ChillingPepper, Grub4k, pukkandan * improve escape handling, from https://github.com/yt-dlp/yt-dlp/pull/#521 thanks Grub4k * support template strings from https://github.com/yt-dlp/yt-dlp/pull/6623 thanks Grub4k * add limited `!` evaluation (eg, !!0 -> false, see tests)
2023-07-19[utils] Align traverse_obj() with yt-dlpdirkf
Thanks Grub4k for these: * traverse `Iterable`s, from https://github.com/yt-dlp/yt-dlp/pull/6902, etc * traverse `set` key for transformations/filters, `re.Match` group names, from https://github.com/yt-dlp/yt-dlp/commit/776995bc109c5cd1aa56b684fada2ce718a386ec, etc * traverse `re.Match`es, from https://github.com/yt-dlp/yt-dlp/pull/5174 * always return list when branching, from https://github.com/yt-dlp/yt-dlp/pull/5170
2023-07-18[test] Fixes for old Pythonsdirkf
2023-07-18[utils] `YoutubeDLCookieJar`: Add `get_cookie_header` and ↵bashonly
`get_cookies_for_url` methods
2023-07-18[core] Remove `Cookie` header on redirect to prevent leaksdirkf
Adated from yt-dlp/yt-dlp-ghsa-v8mc-9377-rwjj/pull/1/commits/101caac Thx coletdjnz
2023-07-18[core] Update redirect handling from yt-dlpdirkf
* Thx coletdjnz: https://github.com/yt-dlp/yt-dlp/pull/7094 * add test that redirected `POST` loses its `Content-Type`
2023-07-18[utils] Add {expected_type} and Iterable support to traverse_obj()dirkf
2023-07-05[Misc] Fixes for 2.6 compatibilitydirkf
2023-05-11[utils] Fix `compiled_regex_type` in 249f2b6dirkf
2023-04-23[YouTube] Support Releases tabdirkf
2023-04-05[devscripts] Improve hack to convert command-line options to API optionsdirkf
* define equality for DateRange * don't show default DateRange
2023-03-19[utils] Ensure `allow_types` for `variadic()` is a tupledirkf
2023-02-20Escape URLs in `sanitized_Request`, not `sanitize_url` ↵pukkandan
d2558234cf5dd12d6896eed5427b7dcdb3ab7b5a added escaping of URLs while sanitizing. However, `sanitize_url` may not always receive an actual URL. Eg: When using `youtube-dl "search query" --default-search ytsearch`, `search query` gets escaped to `search%20query` before being prefixed with `ytsearch:` which is not the intended behavior. So the escaping is moved to `sanitized_Request` instead.
2023-02-13[utils] Add parse_qs, update_urldirkf
[skip ci]
2023-02-13[YouTube] Bypass age-gating for certain restricted videosdirkf
* Use TVHTML5_SIMPLY_EMBEDDED_PLAYER client * Also add and fix tests * Introduce and use new utility function `update_url()`
2022-11-03[utils] Backport traverse_obj (etc) from yt-dlp (#31156)Andrei Lebedev
* Backport traverse_obj and closely related function from yt-dlp (code by pukkandan) * Backport LazyList, variadic(), try_call (code by pukkandan) * Recast using yt-dlp's newer traverse_obj() implementation and tests (code by grub4k) * Add tests for Unicode case folding support matching Py3.5+ (requires f102e3d) * Improve/add tests for variadic, try_call, join_nonempty Co-authored-by: dirkf <fieldhouse@gmx.net>
2022-10-11[utils] Sanitize look-alike Unicode glyphs in non-ID filename fields when ↵dirkf
--restrict-filenames Implements https://github.com/ytdl-org/youtube-dl/issues/31216#issuecomment-1236102822, which has a test.
2022-08-21[utils] Ensure RFC3986 encoding result is unicodedirkf
2022-08-14[jsinterp] Overhaul JSInterp to handle new YT players 4c3f79c5, 324f67b9 ↵dirkf
(#31170) * back-port from yt-dlp 8f53dc44a0cc1c2d98c35740b9293462c080f5d0, thanks pukkandan * also support void, improve <</>> precedence, improve expressions in comma-list * add more tests
2022-06-10[utils, etc] Kill child processes when yt-dl is killedpukkandan
* derived from PR #26592, closes #26592 Authored by: Unrud
2022-06-06[utils] Escape URL while sanitizingpukkandan
Closes #31008, #yt-dlp/263 While this fixes the issue in question, it does not try to address the root-cause of the problem Refer: 915f911e365736227e134ad654601443dbfd7ccb, f5fa042c82300218a2d07b95dd6b9c0756745db3
2022-05-28[utils] Enable ALPN in HTTPS to satisfy broken serversdirkf
See https://github.com/yt-dlp/yt-dlp/issues/3878
2021-04-17[utils] PEP 8Sergey M․
2021-04-17[utils] Add support for support for experimental HTTP response status code ↵Sergey M․
308 Permanent Redirect (refs #27877, refs #28768)
2021-01-04[utils] add a function to clean podcast URLsRemita Amine
2020-12-30[utils] accept only supported protocols in url_or_noneRemita Amine
2020-11-21Fix typos (#27084)Josh Soref
* spelling: authorization Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: brightcove Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: creation Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: exceeded Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: exception Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: extension Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: extracting Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: extraction Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: frontline Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: improve Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: length Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: listsubtitles Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: multimedia Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: obfuscated Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: partitioning Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: playlist Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: playlists Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: restriction Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: services Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: split Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: srmediathek Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: support Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: thumbnail Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: verification Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: whitespaces Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-11-17[utils] Skip ! prefixed code in js_to_jsonSergey M․
2020-10-18[utils] Don't attempt to coerce JS strings to numbers in js_to_json (#26851)Kevin O'Connor
The current logic in `js_to_json` tries to rewrite octal/hex numbers to decimal. However, when the logic actually happens the `"` or `'` have already been trimmed off. This causes what were originally strings, that happen to look like octal/hex numbers, to get rewritten to decimal and returned as a number rather than a string. In practive something like: ```js { "0x40": "foo", "040": "bar", } ``` would get rewritten as: ```json { 64: "foo", 32: "bar } ``` This is problematic since this isn't valid JSON as you cannot have non-string keys.
2020-09-06[utils] Recognize wav mimetype (closes #26463)Sergey M․
2020-05-20[utils] Fix file permissions in write_json_file (closes #12471) (#25122)Rob
generated by cgit v1.2.3 (git 2.26.2) at 2024-07-02 20:11:14 +0000