github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,806 Commits 1 Branch 0 Tags
0be3383110d2005c643e708726af1da4c55145aa
Commit Graph

1805 Commits

Author SHA1 Message Date
Mike Fährmann
0be3383110 [formatter] add 'q' & 'Q' conversions - URL-en/decode values 2026-02-03 17:35:05 +01:00
Mike Fährmann
5ddee423ed [exhentai][schalenetwork] enable 'tags' categories by default 2026-02-03 10:49:35 +01:00
Mike Fährmann
d232f0d252 [gofile][pixeldrain] enable 'recursive' by default 2026-02-03 09:11:47 +01:00
Mike Fährmann
25d45478d6 [docs/configuration] document postprocessor /MODE and @EVENT (#8980) 
4e9cb428d6
 2026-02-02 22:08:10 +01:00
Mike Fährmann
e37fe053d2 [docs/configuration] use anchors for alternative/deprecated names 
instead of doubled option titles
 2026-02-02 19:10:20 +01:00
Mike Fährmann
87323fd71e rename 'image-*' & 'chapter-*' options to 'file-*' & 'child-*' 
keep support for both names, e.g. file-filter & image-filter,
while prioritizing the new names
 2026-02-02 18:01:45 +01:00
Mike Fährmann
153e589dd0 [twitter] implement '"search-pagination": "date"' 2026-02-02 16:19:48 +01:00
Mike Fährmann
17e1d25784 [scrolller] add 'user' extractor (#8961) 2026-02-02 09:09:50 +01:00
Mike Fährmann
44e18f9b2f [tsumino] remove module 
" Tsumino - The End
  We're shutting Tsumino down. "
 2026-02-01 22:15:06 +01:00
Mike Fährmann
c42a5dce5c [pp] add 'child' & 'child-after' events 2026-02-01 22:14:14 +01:00
Mike Fährmann
56dcd00391 [job] add 'init' option 2026-02-01 20:48:37 +01:00
Mike Fährmann
4e6e2c27d5 [xenforo] support 'forums.socialmediagirls.com' (#8964) 2026-01-31 09:55:41 +01:00
bassberry
fd5f5611f6 [tiktok] extract subtitles and all cover types (#8805) 
* Make sure that `img_id`, `audio_id` and `cover_id` fields are always available.
    The values are set '' where they are not applicable.
    Having `img_id` is necessary for the default `archive_fmt`, the other fields are handled for consistency.
* Allow downloading more than one cover.
    The previous behavior is kept as-is, but setting the "covers" option to "all" now grabs all available covers.
* Add support for downloading subtitles
    Allows filtering subtitles by source type (ASR, MT) and language.
* Ensure archive uniqueness for covers and subtitles.
* Update the URL test pattern to include the `image` extension.
    Although Tiktok may serve the covers with jpeg content, the file ending can be `.image`.
    The test before 0c14b164 failed because the asserted URL did not match all cover types, but the now used pattern needs the mentioned file ending.
* Add support for "creator_caption" subtitles in "LC" format.
    These subtitles have the keys "Format" set to "creator_caption" and "Source" to "LC".
* Add "LC" (Local Captions) as a subtitle source type in the documentation
* Code deduplication and renaming subtitle metadata
    Changed the item type from singular `subtitle` to `subtitles`.
    Removed the wrong descriptor `cover` from the subtitles fallback title.
* Refactor subtitle filtering
    The filter is now prepared in `_init` to prevent parsing the same config parameter for every item.
    The `_extract_subtitles` function will still extract if either filter (source or language) matches.
* Generate a `file_id` for subtitles
    Subtitles have multiple fields that determine the unique file, so these are simply concatenated.
    This is similar to the cover types, only with more variations.
* Added tests for subtitles
* fix docs entries
* fix '"covers": "all"'
* simplify some code
* Fix fallback title for subtitles
    Added the missing "f" to the f-string and added "subtitle" to the title.
    The resulting title will look like "TikTok video subtitle #1234567"
 2026-01-30 21:01:06 +01:00
Mike Fährmann
3445c51ca4 [job] add 'output.jsonl' option (#8953) 2026-01-30 09:36:28 +01:00
Mike Fährmann
532ab7112e [discord] add 'server-search' extractor 
requested on Discord

https://discord.com/channels/SERVER_ID/search?from=USER_ID
 2026-01-30 07:58:14 +01:00
Mike Fährmann
56168fbc87 [weebdex] add 'lang' option, support query params (#8957) 
for example '?order=asc&group=j0fsj3oem3&tlang=en'
 2026-01-29 17:01:02 +01:00
Mike Fährmann
6c9dff1e29 [docs/options] add Table of Contents 2026-01-27 19:27:40 +01:00
SubmarineScurvy
ef8f2869e7 [listal] add 'image' & 'people' extractors (#1589 #8921) 
* listal extractor
* add listal to init
* fix flake8 & formatting & extractor names/subcategories

* remove 're' import
* remove 'datetime' import
* update & simplify extractors
* update supportedsites
* add tests

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2026-01-27 18:26:41 +01:00
Mike Fährmann
eaaa25b6e4 [job] enable all 'parent-…' options for parent extractors by default 
- parent-directory
- parent-metadata
- parent-session
- parent-skip

- add general 'parent' option
 2026-01-27 12:05:19 +01:00
Mike Fährmann
b67e3c15ff [xenforo] support 'titsintops.com' (#8945) 2026-01-27 10:31:26 +01:00
Mike Fährmann
93bf4ccc18 merge #8928: [mangafreak] add support 2026-01-25 19:52:34 +01:00
Mike Fährmann
7026611f31 merge #8925: [mangatown] add support 2026-01-25 18:35:39 +01:00
Duy Nguyen
8b0e8c656d feat(mangafreak): add support for MangaFreak 
Add chapter and manga extractors for ww2.mangafreak.me with support
for bonus chapters (e.g., 167e suffix).
 2026-01-25 15:56:52 +01:00
Mike Fährmann
adca123646 [weibo:user] add 'subalbums' include (#8792) 2026-01-25 11:16:41 +01:00
Duy Nguyen
9f2d5cbd5d docs: add mangatown to supported sites 2026-01-25 00:04:23 +01:00
Mike Fährmann
1530778bfb merge #8917: [kaliscan] add support 2026-01-23 21:04:48 +01:00
Duy Nguyen
5c71993e0b docs: add kaliscan to supported sites 2026-01-23 20:29:24 +01:00
Mike Fährmann
e93cfa3348 [twitter] implement '"ratelimit": "abort:N"' (#5251 #8864) 2026-01-23 19:54:28 +01:00
Mike Fährmann
f869085476 [weebdex] add 'data-saver' option (#8914) 2026-01-23 09:22:41 +01:00
Mike Fährmann
fb0d639f68 [xenforo] add 'media-album' extractor (#8902) 2026-01-22 09:10:31 +01:00
Mike Fährmann
18fabb9605 [batoto] remove module (#8908) 
"Bato.to has shut down."

There are mirror sites, but they are unscrapeable
due to heavily obfuscated HTML and JS
 2026-01-21 20:33:08 +01:00
Mike Fährmann
4798ac4836 [common] implement 'parent-session' 2026-01-21 20:33:08 +01:00
Mike Fährmann
9ca45aae73 [nitter] re-add instances 2026-01-21 20:32:58 +01:00
Mike Fährmann
63df6423bf [nitter] use 'gallery-dl/<version>' User-Agent (#7045 #8130 #8409) 2026-01-21 18:07:47 +01:00
Mike Fährmann
78da7edde8 [common] add 'googlebot' User-Agent preset 2026-01-21 17:57:26 +01:00
brerk
e00c717b15 [turbo] update 'saint' extractors (#8893 #8896) 
* Implements turbo.py & remove from domain pattern from saints.py
* Remove leftover commented pattern from saints.py
* Make turbo.py comply with flake8
* Add album support
* Improved metadata extracion for albums and single files & created turbo.py tests using saints.py test
* Align turbo.py extractor with flake8 rules
* Fix #class name on turbo.py tests
* Fix #category test
* Fix #category test x2
* Fix #category tests
* Fix #category tests
* Fix TurboMediaExtractor self.groups unpacking

* update basic module formatting
* replace 'saint' with 'turbo' in modules list
* remove saint extractors and tests
* update & simplify 'media' extractor
* update & simplify 'album' extractor
* update tests
* update supportedsites
* update 'category-map' & 'config-map'

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2026-01-19 22:20:13 +01:00
Mike Fährmann
cc5bfa6eb0 [xenforo] support 'celebforum.to' (#8902) 2026-01-19 16:04:33 +01:00
Mike Fährmann
254bb4505c [chevereto] implement password support 2026-01-16 09:33:55 +01:00
Stephon Parker
43387c535d [thefap] add support (#8821 #8822) 
* adding site support for thefap.com
* fixing typo in url tld
* improve & simplify 'model' extractor
* update 'post' extractor
* update docs/supportedsites
* add tests

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2026-01-14 21:11:56 +01:00
Mike Fährmann
fea4f4c30b [pixeldrain:album] add 'zip' option 2026-01-14 12:03:52 +01:00
Mike Fährmann
29e8440ec4 [formatter] add 'format-operator-dot' option 
to allow disabling the behavior introduced in commit
a79a945494
 2026-01-13 22:25:37 +01:00
Mike Fährmann
d7c1c30c62 [booth] add 'category' extractor (#8867) 2026-01-11 22:17:01 +01:00
camellia2077
084a6d73e0 [bilibili] add support for Live Photo (video) downloads (#8860) 
* bilibili: add support for live photo downloads
* fix: resolve flake8 linting errors (whitespace and line length)
* fix: resolve flake8 E302 and W293 linting errors
* fix: resolve flake8 W293 and E302 linting errors

* simplify syntax
* add 'livephoto' option
* add tests
 2026-01-10 19:27:34 +01:00
Mike Fährmann
7e513f35f2 [wikimedia] use 'gallery-dl/<version>' User-Agent (#8770 #8861) 2026-01-10 18:08:13 +01:00
Mike Fährmann
76930ad44c [saint] support 'turbovid.cr' URLs (#8851) 2026-01-10 18:08:13 +01:00
Mike Fährmann
bb7d540c7a [xenforo] add 'media-category' extractor (#8785) 2026-01-10 18:08:05 +01:00
Mike Fährmann
60104f20af [xenforo] implement 'media' support (#8785) 
add 'media-user' and 'media-item' extractors
TODO: 'media-category' extractor (?)
 2026-01-09 21:21:19 +01:00
Mike Fährmann
b1a708beb9 [weibo:album] implement 'subalbum' support (#8792) 
https://weibo.com/u/USER?tabtype=album-1
https://weibo.com/u/USER?tabtype=album-NAME
https://weibo.com/u/USER?tabtype=album-all
https://weibo.com/u/USER?tabtype=album-only
 2026-01-07 19:23:54 +01:00
Mike Fährmann
6cc7153b6c [webtoons] extend 'bgm' option (#8733) 
allow remuxing bgm audio into a different format/container
 2026-01-06 21:53:16 +01:00
Mike Fährmann
cf96fc6ebe [koofr] refactor (#8803) 
https://github.com/mikf/gallery-dl/issues/8803#issuecomment-3708215475
https://github.com/mikf/gallery-dl/issues/8803#issuecomment-3708358606

- add 'recursive' option, remove 'zip'
- recurse into subdirectories
- add 'path' metadata
- remove 'count' & 'num' metadata
- update default directory & archive format
 2026-01-06 21:52:58 +01:00