Commit Graph

7610 Commits

Author SHA1 Message Date
Mike Fährmann
239de3f092 [civitai] update 'quality-videos' default (#8787)
from "quality=100" to "original=true,quality=100"
to prevent a server timeout / error
2025-12-31 09:56:39 +01:00
Mike Fährmann
c91052f257 [tapas] unescape HTML entities in image URLs (#8790) 2025-12-31 08:47:37 +01:00
Mike Fährmann
51c155df94 [yourlesbians] add 'album' extractor (#8713) 2025-12-30 20:33:50 +01:00
Mike Fährmann
a97c320a38 [sankaku] fix re-authentication (#8779)
Unset the `Authorization` header before performing a re-login
2025-12-30 17:25:24 +01:00
CasualYouTuber31
a6c845bdc8 [tiktok] remove yt-dlp dependency & add support for more post types (#8715)
#7246 #8035 #8466 #8730

* [tiktok] support extracting videos directly without yt-dlp
* [tiktok] support extracting users directly without yt-dlp
* [tiktok] fixing logic, tests, linting errors
* [tiktok] implement tiktok-range support for non-yt-dlp user extractor
* [tiktok] Skip range filter if no ranges are given
* [tiktok] Remove debug code
* [tiktok] only check for faulty device IDs during the first couple of passes
    I think the original yt-dlp solution assumes that if a device ID works once, it will always work.
    Plus, my approach would cause needless retries in certain cases if hasMorePrevious does end up being wrong like the original algorithm accounts for. So let's copy the original algorithm here, too.
* [tiktok] support stories
* [tiktok] you can now extract audio without extracting photos
* [tiktok] add TiktokFollowingExtractor
* [tiktok] update supportedsites to include stories
* [tiktok] Keep tiktok-range option for no content user account test
    It acts as a nice guard against that account suddenly having lots of posts to extract
* [tiktok] TiktokUserExtractor and TiktokFollowingExtractor rewrite
* [tiktok] Fix avatar naming convention to match that of posts
* [tiktok] remove type hints for compatibility with older Python versions
* [tiktok] Improve performance of TiktokFollowingExtractor
    This was largely achieved using the story/batch/item_list endpoint
* [tiktok] Forgot to run flake8
* [tiktok] remove old constant
* [tiktok] Support order-posts config item
* [tiktok] flake8
* [tiktok] Older Python versions don't support match
* [tiktok] always ask for posts in chronological order when in "desc" mode
    We should aim to avoid having pinned posts returned before non-pinned ones
* [tiktok] Add liked posts extraction
* [tiktok] Add reposts extraction
* [tiktok] Add saved posts extraction

* cleanup imports
* remove '# MARK:' comments
* remove & simplify 'except' statements
    KeyboardInterrupt & SystemExit inherit from BaseException (not Exception)
    and therefore don't need special handling
* split 'user' extractor
* move PATTERNs into their respective functions
* use dict comprehensions
* add only-matching test URLs for split user extractors
* update config docs
    rename 'tiktok-user-extractor' to 'ytdl'
* document '"popular"' 'order-posts' value
* inline and remove 'util.chunk()'
2025-12-30 17:17:57 +01:00
Mike Fährmann
c8c4575c7f [dl:http] add MIME type and signature for .aac files 2025-12-29 19:05:34 +01:00
Mike Fährmann
c2917ad5c5 [ahottie] add aupport (#8710) 2025-12-29 18:47:10 +01:00
Mike Fährmann
ec2267244f [tumblr:search] prevent KeyError when using 'offset' pagination (#8720) 2025-12-29 16:57:04 +01:00
Mike Fährmann
ebfd87bbb0 [instagram] detect homepage redirects (#8714) 2025-12-29 16:28:37 +01:00
Mike Fährmann
e12b17649a [dl:ytdl] support '_ytdl_params' 2025-12-28 23:35:21 +01:00
Mike Fährmann
5427e337d4 [dl:ytdl] unset '_mtime_http' (#8767)
prevent an mtime value from a previous HTTP download
from being reused for a file downloaded by 'ytdl'
2025-12-28 22:46:52 +01:00
Mike Fährmann
4f535ec606 [imgpv] add 'image' extractor (#8773) 2025-12-28 21:36:06 +01:00
Oleguer Llopart
bea0e16970 [webtoons] extract 'num_play' & 'num_stop' for background music (#8733 #8755)
* [webtoons] play_image and stop_image in background music
* [webtoons] add play_image and stop_image fields to test results
* [webtoons]: flake8
* [webtoons]: flake8 (2)

* update & improve
    - remove global 'images_urls'
    - move enumerate loop into images() extraction
* simplify 'play_image' etc extraction
* rename 'play_image' etc
    - num_play
    - num_stop
    - filename_play
    - filename_stop

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2025-12-27 09:09:10 +01:00
Mike Fährmann
f2efccaadf [instagram:stories] extract correct 'expires' dates (#8764) 2025-12-26 20:27:43 +01:00
Mike Fährmann
05e6340867 [ytdl] expect all exception types when extracting info_dict (#8343)
not only 'YoutubeDLError's
2025-12-26 20:27:04 +01:00
Mike Fährmann
daf6cc2c78 [twitter] add 'notifications' extractor (#7974) 2025-12-26 18:06:00 +01:00
Mike Fährmann
fd6f19347a [actions] fix 'abort' (#8753)
fixes regression introduced in 2eb5e52055

forward the correct arguments to StopExtraction.__init__
instead of logging message and 'extras'
2025-12-26 14:07:48 +01:00
Mike Fährmann
c384cd06c2 [fansly] add 'previews' option (#8686) 2025-12-25 20:25:47 +01:00
Mike Fährmann
01fe6870a3 [fansly] disable 'formats' check by default (#8757)
should allow downloading and selecting the highest resolution video format
regardless of its 'type' / format code
2025-12-25 19:15:21 +01:00
Mike Fährmann
6845661123 [discord] update asset collection & handling (#8589) 2025-12-25 18:22:55 +01:00
Mike Fährmann
57c2327ae3 [tests/results] implement more specific '#auth' checks 2025-12-23 22:17:33 +01:00
Mike Fährmann
9c50c10f1a [deviantart:stash] fix JSONDecodeEerror for folders (#8750)
remove extra ',"submit":{}' after end of JSON object
2025-12-23 20:54:46 +01:00
Mike Fährmann
cc8cd78103 [webtoons] download episode background music (#8733)
add 'bgm' option
2025-12-23 19:08:30 +01:00
Mike Fährmann
2be54be692 [subscribestar] merge 'user-tag' into regular 'user' extractor (#8737) 2025-12-23 18:58:25 +01:00
Mike Fährmann
c3ef760a9f [discord] add 'server-assets' extractor (#8589) 2025-12-22 21:08:33 +01:00
Mike Fährmann
eda6957d4c [imagehosts] remove 'splitext' import 2025-12-22 17:29:29 +01:00
Mike Fährmann
568faee726 [imagehosts] remove '_https' 2025-12-22 11:48:08 +01:00
Mike Fährmann
7669a1f13a [subscribestar:user-tag] update 'pattern' 2025-12-22 11:43:30 +01:00
Mike Fährmann
b5a7540619 Merge branch 'op+': use '+' for 2-element string concatenations 2025-12-22 11:34:21 +01:00
Mike Fährmann
b69e5a7a58 [scripts/init] use 'operator +' when building 'pattern' 2025-12-22 11:31:48 +01:00
Mike Fährmann
00c6821a3f replace 2-element f-strings with simple '+' concatenations
Python's 'ast' module and its 'NodeVisitor' class
were incredibly helpful in identifying these
2025-12-22 11:26:04 +01:00
Mike Fährmann
457c8b2e76 [exhentai] fix possible exception in 'finalize()' (#8741) 2025-12-22 11:22:15 +01:00
Mike Fährmann
609e19273d [subscribestar] add 'user-tag' extractor (#8737) 2025-12-21 22:14:17 +01:00
Mike Fährmann
375ab4084b [facebook] add 'loop' option (#8696) 2025-12-21 21:27:33 +01:00
Mike Fährmann
d0f06be0d2 use 'operator +' when building 'pattern' 2025-12-20 22:07:44 +01:00
Mike Fährmann
e006d26c8e Revert "use f-strings when building 'pattern'"
revert d7c97d5a97.
2025-12-20 22:07:37 +01:00
Mike Fährmann
19a27a47b3 [options] add '--sleep-skip' option (#6582)
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3677460485
2025-12-20 19:06:11 +01:00
Mike Fährmann
575bf7a9f9 release version 1.31.1 2025-12-20 10:56:40 +01:00
Mike Fährmann
af475a311b [scripts/release] reduce expected executable sizes 2025-12-20 10:47:20 +01:00
Mike Fährmann
4dba40703d [tests:pp] prevent failures due to config dependencies
'filename' & 'directory' set in 'path' tests cause
'ClassifyTest' to fail
2025-12-20 10:37:28 +01:00
Mike Fährmann
afe867361f [weebdex] add support (#8722) 2025-12-19 20:06:40 +01:00
Mike Fährmann
e971b4c989 [tests] add 'path' tests 2025-12-19 17:10:53 +01:00
Mike Fährmann
8ec48a039f [aryion:favorite] ignore already seen folders (#8728) 2025-12-19 11:29:53 +01:00
Mike Fährmann
c3d8602418 [path] implement dynamic length directories (#1350)
append directory segments for each item of a list (or general non-string
iterable), which can be returned with the 'I' specifier
2025-12-18 09:53:26 +01:00
Mike Fährmann
73db1846e1 [furaffinity] fix 'tags' metadata (#8724)
Co-authored-by: d-koc <141529764+d-koc@users.noreply.github.com>
2025-12-18 08:55:27 +01:00
Mike Fährmann
982d9908c1 [aryion] fix "AttributeError: '_pagination'" (#8723)
fixes regression introduced in 62d6a2206d
2025-12-18 08:36:34 +01:00
Mike Fährmann
218dbdb3ca [path] combine 'build_*' & 'build_*_conditional' 2025-12-17 19:45:38 +01:00
Mike Fährmann
ff94002b44 [formatter] add 'I' format specifier - identity 2025-12-17 19:22:18 +01:00
Mike Fährmann
717d38f38c [reddit] use '"videos": "dash"' by default (#8657)
https://github.com/mikf/gallery-dl/issues/8657#issuecomment-3663560902
2025-12-17 18:45:06 +01:00
Peter Cunha
f9096370e6 [picazor] add 'user' extractor (#7083 #7504 #7795 #8717)
* [picazor] Add support for Picazor
* [picazor] Improve tests for picazor
* [picazor] run flake8
* [picazor] working on tests
* update extractor
* update tests
* update supportedsites

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2025-12-16 18:55:18 +01:00