github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,808 Commits 1 Branch 0 Tags
2cbd9740f26bbf3d6604439a626bebedcec330ed
Commit Graph

7808 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
2cbd9740f2 enable 'metadata-url/-path/-parent/-extractor]' by default 
assign values to "private" keys to not have them appear in the
standard JSON output, while still making it easier to access them
 2026-02-03 19:28:33 +01:00
Mike Fährmann
f0f9575406 [job] fix 'AttributeError' when enabling 'init' for non-DownloadJob 
fixes bug in 56dcd00391
 2026-02-03 19:00:45 +01:00
Mike Fährmann
0be3383110 [formatter] add 'q' & 'Q' conversions - URL-en/decode values 2026-02-03 17:35:05 +01:00
Mike Fährmann
5ddee423ed [exhentai][schalenetwork] enable 'tags' categories by default 2026-02-03 10:49:35 +01:00
Mike Fährmann
d232f0d252 [gofile][pixeldrain] enable 'recursive' by default 2026-02-03 09:11:47 +01:00
Mike Fährmann
25d45478d6 [docs/configuration] document postprocessor /MODE and @EVENT (#8980) 
4e9cb428d6
 2026-02-02 22:08:10 +01:00
Mike Fährmann
0c04090ff4 [util] set flags in range predicates 
use flags to stop extractors immediately when hitting the upper range
limit instead of raising an exception one predicate check later
 2026-02-02 21:48:45 +01:00
CasualYouTuber31
e44c5bd707 [tiktok] identify when user accounts do not exist (#8977) 2026-02-02 19:30:12 +01:00
CasualYouTuber31
7c8fbb6fa5 [tiktok] do not exit early when rolling back cursor (#8968) 
* [tiktok] do not exit account extraction early when we need to manually roll back the cursor
* [tiktok] fix rehydration data error string formatting
 2026-02-02 19:29:24 +01:00
CasualYouTuber31
a7cc447f51 [tiktok] fix outdated error message (#8979) 2026-02-02 19:26:46 +01:00
Mike Fährmann
cbb9264ddc use tempfile when updating input files (#8981) 
0d72789aa3
 2026-02-02 19:15:26 +01:00
Mike Fährmann
e37fe053d2 [docs/configuration] use anchors for alternative/deprecated names 
instead of doubled option titles
 2026-02-02 19:10:20 +01:00
Mike Fährmann
87323fd71e rename 'image-*' & 'chapter-*' options to 'file-*' & 'child-*' 
keep support for both names, e.g. file-filter & image-filter,
while prioritizing the new names
 2026-02-02 18:01:45 +01:00
Mike Fährmann
153e589dd0 [twitter] implement '"search-pagination": "date"' 2026-02-02 16:19:48 +01:00
Mike Fährmann
a95cf92d61 [twitter] add & use '_tweetid_to_datetime' method 
use 'dt' functions directly
 2026-02-02 16:11:59 +01:00
Mike Fährmann
39fb51dade [mangafire] export 'VRF' functions 2026-02-02 11:45:12 +01:00
Mike Fährmann
5c8bea3cc0 [8chan] fail downloads of 'POW' images (#8975) 2026-02-02 10:51:24 +01:00
Mike Fährmann
a2c79ba057 [8chan] skip 'TOS' cookie name lookup if already present 2026-02-02 10:51:20 +01:00
Mike Fährmann
17e1d25784 [scrolller] add 'user' extractor (#8961) 2026-02-02 09:09:50 +01:00
Mike Fährmann
f99c8f5ad9 [xhamster] fix user profile extraction (#8974) 2026-02-02 08:53:24 +01:00
Mike Fährmann
44e18f9b2f [tsumino] remove module 
" Tsumino - The End
  We're shutting Tsumino down. "
 2026-02-01 22:15:06 +01:00
Mike Fährmann
c42a5dce5c [pp] add 'child' & 'child-after' events 2026-02-01 22:14:14 +01:00
Mike Fährmann
56dcd00391 [job] add 'init' option 2026-02-01 20:48:37 +01:00
Mike Fährmann
efa301fcc2 Merge branch 'res' 2026-02-01 19:52:20 +01:00
Mike Fährmann
451ed23737 [discord:server-search] use 'max_id' for pagination 
'offset' is limited to 10_000
'max_id' is hopefully not
 2026-02-01 19:48:15 +01:00
Mike Fährmann
eed46f8dcf [build] update PyInstaller hiddenimports and py2exe modules 2026-02-01 19:29:30 +01:00
Mike Fährmann
73bf99612a [scrolller] move GraphQL queries 2026-02-01 19:18:52 +01:00
Mike Fährmann
cc645984a4 [luscious] export GraphQL queries 2026-02-01 19:18:14 +01:00
Mike Fährmann
0c24955507 [mangapark] export GraphQL queries 2026-02-01 19:18:10 +01:00
Mike Fährmann
40a4ff935a [500px] export GraphQL queries 2026-02-01 19:16:14 +01:00
Mike Fährmann
51d9fd2f4d [behance] export GraphQL queries 2026-02-01 19:13:38 +01:00
Mike Fährmann
1c2e2d5d08 [deviantart] export journal templates 2026-02-01 18:59:31 +01:00
Mike Fährmann
3d114dbc67 [deviantart] export 'tiptap' functions 2026-02-01 18:53:22 +01:00
Mike Fährmann
20ef39be45 [tsumino] export 'jsurl' code 2026-02-01 18:50:09 +01:00
Mike Fährmann
7692d31a57 [twitter] move transaction_id.py 2026-02-01 18:48:30 +01:00
Mike Fährmann
343981ac1c [common] add 'utils()' method 2026-02-01 18:48:17 +01:00
Mike Fährmann
b37acd1e28 [artstation] fix embedded videos (#8972) 2026-02-01 13:00:48 +01:00
Mike Fährmann
1a8ccbe720 [common] use f-string in BaseExtractor.update() 2026-02-01 08:54:15 +01:00
Mike Fährmann
9fd732afe8 [config] support setting default file type via GDL_CONFIG_TYPE 2026-01-31 17:55:17 +01:00
Mike Fährmann
fd6bc3961c release version 1.31.5 2026-01-31 10:49:00 +01:00
Mike Fährmann
1286839037 [socialmediagirlsforum] add tests 2026-01-31 09:55:45 +01:00
Mike Fährmann
5b8ad403dd [xenforo] decode '/goto/link-confirmation' links (#8964) 2026-01-31 09:55:45 +01:00
Mike Fährmann
4e6e2c27d5 [xenforo] support 'forums.socialmediagirls.com' (#8964) 2026-01-31 09:55:41 +01:00
CasualYouTuber31
01657caa15 [tiktok] do not fail entire extraction if one post fails (#8962) 2026-01-30 23:03:59 +01:00
bassberry
fd5f5611f6 [tiktok] extract subtitles and all cover types (#8805) 
* Make sure that `img_id`, `audio_id` and `cover_id` fields are always available.
    The values are set '' where they are not applicable.
    Having `img_id` is necessary for the default `archive_fmt`, the other fields are handled for consistency.
* Allow downloading more than one cover.
    The previous behavior is kept as-is, but setting the "covers" option to "all" now grabs all available covers.
* Add support for downloading subtitles
    Allows filtering subtitles by source type (ASR, MT) and language.
* Ensure archive uniqueness for covers and subtitles.
* Update the URL test pattern to include the `image` extension.
    Although Tiktok may serve the covers with jpeg content, the file ending can be `.image`.
    The test before 0c14b164 failed because the asserted URL did not match all cover types, but the now used pattern needs the mentioned file ending.
* Add support for "creator_caption" subtitles in "LC" format.
    These subtitles have the keys "Format" set to "creator_caption" and "Source" to "LC".
* Add "LC" (Local Captions) as a subtitle source type in the documentation
* Code deduplication and renaming subtitle metadata
    Changed the item type from singular `subtitle` to `subtitles`.
    Removed the wrong descriptor `cover` from the subtitles fallback title.
* Refactor subtitle filtering
    The filter is now prepared in `_init` to prevent parsing the same config parameter for every item.
    The `_extract_subtitles` function will still extract if either filter (source or language) matches.
* Generate a `file_id` for subtitles
    Subtitles have multiple fields that determine the unique file, so these are simply concatenated.
    This is similar to the cover types, only with more variations.
* Added tests for subtitles
* fix docs entries
* fix '"covers": "all"'
* simplify some code
* Fix fallback title for subtitles
    Added the missing "f" to the f-string and added "subtitle" to the title.
    The resulting title will look like "TikTok video subtitle #1234567"
 2026-01-30 21:01:06 +01:00
CasualYouTuber31
2d01fef300 [tiktok] Restructure to allow user extractors to provide their own rehydration data (#8848) 2026-01-30 15:18:56 +01:00
Mike Fährmann
3445c51ca4 [job] add 'output.jsonl' option (#8953) 2026-01-30 09:36:28 +01:00
Mike Fährmann
532ab7112e [discord] add 'server-search' extractor 
requested on Discord

https://discord.com/channels/SERVER_ID/search?from=USER_ID
 2026-01-30 07:58:14 +01:00
Mike Fährmann
690b3ba200 [civitai:user-posts] fix pagination (#8955) 
fix '400 Bad Request' errors when retrieving
more than the first batch of posts.
 2026-01-29 18:53:08 +01:00
Mike Fährmann
56168fbc87 [weebdex] add 'lang' option, support query params (#8957) 
for example '?order=asc&group=j0fsj3oem3&tlang=en'
 2026-01-29 17:01:02 +01:00