Commit Graph

6930 Commits

Author SHA1 Message Date
Mike Fährmann
3eb0b28d6d [facebook] implement 'include' option & add 'avatar' extractor (#7848)
rename 'profile' extractor to 'photos'
2025-07-25 18:20:05 +02:00
Mike Fährmann
ef4f09fab1 [patreon] support 'date-max' for '/home' URLs (#7856) 2025-07-25 00:22:55 +02:00
Mike Fährmann
da44d83438 [patreon] implement 'cursor' support (#7856) 2025-07-24 23:08:58 +02:00
Mike Fährmann
2c85f4a32d [civitai] change default includes to 'user-images,user-videos' (#7874)
less metadata by default, but grabs *all* user files
2025-07-24 18:10:45 +02:00
Mike Fährmann
0440d4128a [bilibili] simplify user articles extractors 2025-07-24 17:46:37 +02:00
Mike Fährmann
20af1e2508 [bilibili] warn about blocked articles (#7880) 2025-07-24 16:24:04 +02:00
Mike Fährmann
cefdde65ba [readcomiconline] use 'text.re()' 2025-07-24 15:58:21 +02:00
Mike Fährmann
aae85fef61 [readcomiconline] force 'One page' Reading mode (#7890) 2025-07-24 15:53:39 +02:00
Mike Fährmann
aa6c2dcbac [behance] provide 'creator[name]' metadata (#7885) 2025-07-24 15:30:51 +02:00
Mike Fährmann
849a5b191f [behance] use '"browser": "firefox"' by default (#7803 #7877) 2025-07-24 09:31:12 +02:00
Mike Fährmann
28205f47b0 [tests/results] fix '#log' checks for URLs raising exceptions 2025-07-24 09:18:58 +02:00
Mike Fährmann
f39c8b125f [4chan] detect files containing only null bytes (#7883) 2025-07-24 09:11:52 +02:00
Mike Fährmann
b2a354f0d2 [urlgalleries] do not strip query parameters (#7888)
fixes regression introduced in fc06243b89
2025-07-24 08:46:59 +02:00
Farahat
cf2e5a1619 [leakgallery] add support (#7872)
* add new extractor for leakgallery.com

    Added support for downloading photo and video posts from leakgallery.com.

    Supports:
    * Individual post URLs
    * User profile URLs with pagination via AJAX
    * Optional type/sort filters (e.g. /Photos/MostRecent)
    * Proper file extension handling
    * Creator-based folder structure
    * Compatibility with --download-archive

    Tested locally and functional, but may still need review or improvement.
    
* [leakgallery] add support
    Added leakgallery to extractor module imports so it's recognized and used.
* [leakgallery] update extractor structure
    - Refactored using LeakGalleryExtractorBase to remove duplication
    - Moved init logic into items() using self.groups
    - Replaced re with text.re as per upstream guidance
    - Added creator fallback and media deduplication
    - Aligned structure with gallery-dl maintainer review tips
* [leakgallery] add support
    - Added leakgallery entry to supportedsites.md
    - Includes post, user, trending, and most-liked subcategories
* add exported extractor results
* [leakgallery] fix flake8 style issues
    Cleaned up code to comply with flake8 rules, especially:
    - removed unused imports
    - split long lines >79 chars
    - ensured newline at EOF
    No functional changes made; purely formatting to satisfy CI checks.
* [tests] update extractor results
* [leakgallery] fix flake8 style issues (part 2)
    Fix remaining flake8 issues in leakgallery.py:
    - Reformat line breaks to avoid W503 (line break before binary operator)
    - Wrap long lines to respect E501 (line too long > 79 characters)
    - Cleaned up exception logging for better clarity
    - Confirmed all flake8 checks now pass successfully
    This superseedes the previous commit which partially fixed formatting violations.
* [leakgallery] fix flake8 style issues (part 3)
* [leakgallery] rename extractor classes
* [tests] update extractor results
* [tests] rename extractor results
* [leakgallery] rename extractor classes (part 2)
* [leakgallery] rename example
* update docs/supportedsites
* update test results
    and convert line endings to '\n'
* update
    - convert line endings to '\n'
    - use _pagination method
    - fix logging calls
* return more metadata for _pagination() results
2025-07-22 22:50:25 +02:00
Mike Fährmann
158ba1b95a [docs/configuration] update
- add Jinja options (#1390)
- add and use custom 'Module' type
- improve 'Path' description
2025-07-22 20:57:54 +02:00
Mike Fährmann
a097a373a9 simplify if statements by using walrus operators (#7671) 2025-07-22 20:57:54 +02:00
Mike Fährmann
e8b2a496ba [scripts] ensure files use 'utf-8' encoding and '\n' newlines (#7872) 2025-07-22 20:57:54 +02:00
Mike Fährmann
afc5b9c9c6 [tests/extractor] improve example URL mismatch error message (#7872) 2025-07-22 20:57:54 +02:00
Mike Fährmann
05cba49062 [options] add '--sleep-429' command-line option (#7871) 2025-07-22 20:57:54 +02:00
Mike Fährmann
a08f4aa1c0 [nhentai] provide 'gallery_id' for pagination results (#7868) 2025-07-22 20:57:54 +02:00
Mike Fährmann
ea61f432f2 [formatter/jinja] add support for options (#1390)
- custom filters
- custom tests
- policies
- Environment.__init__() arguments
2025-07-22 20:57:54 +02:00
Mike Fährmann
f0de034889 [exception] rename 'LoginRequired' to 'AuthRequired' 2025-07-22 20:57:54 +02:00
林博仁 Buo-ren Lin
668556251c [snap] fix deprecated CRAFT_ARCH_TRIPLET usage (#7866)
This fixes the following warning during the pull
phase:

```
CRAFT_ARCH_TRIPLET is deprecated, use CRAFT_ARCH_TRIPLET_BUILD_{ON|FOR}
```
2025-07-22 06:56:50 +02:00
林博仁 Buo-ren Lin
28988ce0c3 [snap] switch to yt-dlp (#7865)
https://github.com/mikf/gallery-dl/pull/7841#pullrequestreview-3030959915
2025-07-22 06:56:38 +02:00
Mike Fährmann
e0599d35dc [civitai] fix 'AttributeError' when a file's post was deleted (#7860) 2025-07-20 20:02:09 +02:00
Mike Fährmann
9bbfb09952 [pp:exec] add 'session' option (#6582)
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3067297033
2025-07-20 19:55:33 +02:00
Mike Fährmann
fc06243b89 [urlgalleries] fix extraction (#7858) 2025-07-20 19:55:33 +02:00
Mike Fährmann
008a08ca43 [ytdl] add 'deprecations' option
https://github.com/gdl-org/builds/issues/3
2025-07-20 19:55:24 +02:00
Mike Fährmann
240f958707 [formatter] add 'W' conversion - sanitize whitespace (#6582)
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3090777073
2025-07-19 21:44:55 +02:00
Mike Fährmann
7bb4053396 [text] add 'sanitize_whitespace()' 2025-07-19 20:49:48 +02:00
Mike Fährmann
923e27f4fd [civitai] fix 'extension' for videos without 'name' and 'mimeType'
use _video_ext instead of _image_ext
2025-07-19 18:30:29 +02:00
Mike Fährmann
0a245e8d80 [civitai] improve '_extract_version_id()'
also check for 'modelVersionIdsManual'
2025-07-19 18:30:29 +02:00
Mike Fährmann
272df323f3 [civitai] ensure 'file' & 'post' data has a 'date' value (#7548)
https://github.com/mikf/gallery-dl/issues/7548#issuecomment-3091492518
2025-07-19 18:30:29 +02:00
Mike Fährmann
0b991148a1 [civitai] rename 'generate' to 'generated' (#7796) 2025-07-19 18:30:29 +02:00
林博仁 Buo-ren Lin
acc89ed59f [snap] migrate base to core22 (#7841)
This fixes build failure due to incompatible Python runtime.

Signed-off-by: 林博仁(Buo-ren Lin) <buo.ren.lin@gmail.com>
2025-07-19 15:15:29 +02:00
Luca Russo
869a644e07 [facebook] add retries to profile page request (#7725 #7834 #7852) 2025-07-19 10:18:39 +02:00
turaiel
7464a774f6 [inkbunny] enable pool metadata (#7850) 2025-07-19 09:36:26 +02:00
Mike Fährmann
b0a33d402d [sexcom] update 'search' extractor (#7807) 2025-07-18 21:02:32 +02:00
Mike Fährmann
22ec1aad9a [pornhub] fix 'gallery' extractor (#7842)
- update API endpoint
- extract and send 'token' parameter
2025-07-18 19:08:55 +02:00
Mike Fährmann
67a4472bc2 [civitai] add 'generate' extractor (#7796) 2025-07-18 18:34:17 +02:00
Mike Fährmann
fee14e0055 [madokami] improve 'chapter_string' extraction (#7828) 2025-07-17 22:12:30 +02:00
Mike Fährmann
1561284815 [madokami] add 'manga' extractor (#7828) 2025-07-17 20:40:26 +02:00
Mike Fährmann
493fc483c6 [scripts/init] handle subdomains when building BASE_PATTERN 2025-07-17 18:38:54 +02:00
Mike Fährmann
629db56b12 [common] detect CF & DG challenges in request_json & _xml (#7833)
even when called when fatal=False
2025-07-17 17:17:01 +02:00
Mike Fährmann
dd09937d69 fix exit code for requests' JSONDecodeError (#4380) 2025-07-17 16:37:40 +02:00
Mike Fährmann
64d6e8c8ff [azurelanewiki] prevent Anubis challenge 2025-07-17 16:37:30 +02:00
Mike Fährmann
87cf635f6e [wikimedia] add missing 'self' argument when calling 'prepare()' (#7835)
fixes regression introduced in 811b665e33
2025-07-17 16:30:36 +02:00
Mike Fährmann
232e30f64e [actions] fix 'parse_logging' import (#7837)
fixes regressions introduced in bccf467d19
2025-07-17 16:30:10 +02:00
Mike Fährmann
22fd866b73 [formatter] update format string type lookup
use a dict instead of if/elif/else
2025-07-16 23:24:21 +02:00
Mike Fährmann
f25c5f4b0c [formatter] add basic 'Jinja2' template support (#1390) 2025-07-16 23:23:58 +02:00