Commit Graph

6908 Commits

Author SHA1 Message Date
林博仁 Buo-ren Lin
668556251c [snap] fix deprecated CRAFT_ARCH_TRIPLET usage (#7866)
This fixes the following warning during the pull
phase:

```
CRAFT_ARCH_TRIPLET is deprecated, use CRAFT_ARCH_TRIPLET_BUILD_{ON|FOR}
```
2025-07-22 06:56:50 +02:00
林博仁 Buo-ren Lin
28988ce0c3 [snap] switch to yt-dlp (#7865)
https://github.com/mikf/gallery-dl/pull/7841#pullrequestreview-3030959915
2025-07-22 06:56:38 +02:00
Mike Fährmann
e0599d35dc [civitai] fix 'AttributeError' when a file's post was deleted (#7860) 2025-07-20 20:02:09 +02:00
Mike Fährmann
9bbfb09952 [pp:exec] add 'session' option (#6582)
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3067297033
2025-07-20 19:55:33 +02:00
Mike Fährmann
fc06243b89 [urlgalleries] fix extraction (#7858) 2025-07-20 19:55:33 +02:00
Mike Fährmann
008a08ca43 [ytdl] add 'deprecations' option
https://github.com/gdl-org/builds/issues/3
2025-07-20 19:55:24 +02:00
Mike Fährmann
240f958707 [formatter] add 'W' conversion - sanitize whitespace (#6582)
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3090777073
2025-07-19 21:44:55 +02:00
Mike Fährmann
7bb4053396 [text] add 'sanitize_whitespace()' 2025-07-19 20:49:48 +02:00
Mike Fährmann
923e27f4fd [civitai] fix 'extension' for videos without 'name' and 'mimeType'
use _video_ext instead of _image_ext
2025-07-19 18:30:29 +02:00
Mike Fährmann
0a245e8d80 [civitai] improve '_extract_version_id()'
also check for 'modelVersionIdsManual'
2025-07-19 18:30:29 +02:00
Mike Fährmann
272df323f3 [civitai] ensure 'file' & 'post' data has a 'date' value (#7548)
https://github.com/mikf/gallery-dl/issues/7548#issuecomment-3091492518
2025-07-19 18:30:29 +02:00
Mike Fährmann
0b991148a1 [civitai] rename 'generate' to 'generated' (#7796) 2025-07-19 18:30:29 +02:00
林博仁 Buo-ren Lin
acc89ed59f [snap] migrate base to core22 (#7841)
This fixes build failure due to incompatible Python runtime.

Signed-off-by: 林博仁(Buo-ren Lin) <buo.ren.lin@gmail.com>
2025-07-19 15:15:29 +02:00
Luca Russo
869a644e07 [facebook] add retries to profile page request (#7725 #7834 #7852) 2025-07-19 10:18:39 +02:00
turaiel
7464a774f6 [inkbunny] enable pool metadata (#7850) 2025-07-19 09:36:26 +02:00
Mike Fährmann
b0a33d402d [sexcom] update 'search' extractor (#7807) 2025-07-18 21:02:32 +02:00
Mike Fährmann
22ec1aad9a [pornhub] fix 'gallery' extractor (#7842)
- update API endpoint
- extract and send 'token' parameter
2025-07-18 19:08:55 +02:00
Mike Fährmann
67a4472bc2 [civitai] add 'generate' extractor (#7796) 2025-07-18 18:34:17 +02:00
Mike Fährmann
fee14e0055 [madokami] improve 'chapter_string' extraction (#7828) 2025-07-17 22:12:30 +02:00
Mike Fährmann
1561284815 [madokami] add 'manga' extractor (#7828) 2025-07-17 20:40:26 +02:00
Mike Fährmann
493fc483c6 [scripts/init] handle subdomains when building BASE_PATTERN 2025-07-17 18:38:54 +02:00
Mike Fährmann
629db56b12 [common] detect CF & DG challenges in request_json & _xml (#7833)
even when called when fatal=False
2025-07-17 17:17:01 +02:00
Mike Fährmann
dd09937d69 fix exit code for requests' JSONDecodeError (#4380) 2025-07-17 16:37:40 +02:00
Mike Fährmann
64d6e8c8ff [azurelanewiki] prevent Anubis challenge 2025-07-17 16:37:30 +02:00
Mike Fährmann
87cf635f6e [wikimedia] add missing 'self' argument when calling 'prepare()' (#7835)
fixes regression introduced in 811b665e33
2025-07-17 16:30:36 +02:00
Mike Fährmann
232e30f64e [actions] fix 'parse_logging' import (#7837)
fixes regressions introduced in bccf467d19
2025-07-17 16:30:10 +02:00
Mike Fährmann
22fd866b73 [formatter] update format string type lookup
use a dict instead of if/elif/else
2025-07-16 23:24:21 +02:00
Mike Fährmann
f25c5f4b0c [formatter] add basic 'Jinja2' template support (#1390) 2025-07-16 23:23:58 +02:00
Mike Fährmann
cbf930c99c release version 1.30.0 2025-07-15 19:08:43 +02:00
Mike Fährmann
b66a476cab [komikcast] update domain to 'komikcast.li' 2025-07-15 08:12:55 +02:00
Mike Fährmann
bccf467d19 [actions] allow using actions as 'signals-actions' target 2025-07-14 22:58:02 +02:00
Mike Fährmann
f7e84f0a09 [actions] add 'raise' action 2025-07-14 21:32:56 +02:00
Mike Fährmann
0a6e58b8ba [actions] add 'flag' action 2025-07-14 21:29:40 +02:00
Mike Fährmann
50cec2fb0c [util] add 'error()' function to global scope (#7814) 2025-07-14 20:05:20 +02:00
Mike Fährmann
d9a650d547 [options] add '--compat' command-line option 2025-07-14 18:52:39 +02:00
Mike Fährmann
bbe7faed21 [iwara] update
- add 'include' docs
- simplify code for different result types
- provide 'count' and 'num' metadata
- remove 'search_type' metadata
2025-07-14 18:47:14 +02:00
Mike Fährmann
fca1cd51f5 [gelbooru] improve error for 401 responses (#7674) 2025-07-14 16:54:21 +02:00
Mike Fährmann
df946faf40 [scripts/init] fix extra blank line without copyright
1686f32a0d (commitcomment-162021403)
2025-07-14 16:54:21 +02:00
NecRaul
089f815560 [motherless] Add group support (#7774 #7787)
* [motherless] Add group support
* [motherless] Update old unit tests
* [motherless] Add new unit tests
* [motherless] Update docs
2025-07-14 16:50:07 +02:00
Appu
75582e38e9 [VSCO] Fix JSON returned by VSCO (#7821)
* [VSCO] Fix JSON returned by VSCO

The JSON returned contains multiple `undefined` values that aren't valid
JSON, changing these values to `null` fixes the issue.

* [VSCO] Make JSON fix more targeted

Change only occurrences that are values of a key-value
2025-07-14 15:29:22 +02:00
NecRaul
a7ebb835ea [iwara] Add support (#2652 #5840 #7785)
* [iwara] Add initial support
* [iwara] Add search support
* [iwara] Code cleanup
* [iwara] Small fixes and additions
* [iwara] Add tag support
* [iwara] Add mime-type to metadata
* [iwara] Refactor patterns/matching using urllib
* [iwara] Add unit tests
* [iwara] Update docs
* [iwara] Fix linting on older Python versions
* [iwara] update 'IwaraAPI' interface class
    - define endpoints inside methods
    - implement and use _call() and _pagination()
    - cache auth tokens
* [iwara] split and rename 'profile' extractor
    TODO:
    - update test results
    - simplify code
* [iwara] simplify '_user_params()' usage
* [iwara] update 'video' extractor
    and move user data extraction into 'yield_video'
* [iwara] update 'image' extractor
    and move user info extraction into 'yield_image()'
* [iwara] update 'playlist' extractor
* [iwara] update 'search' extractor
* [iwara] update 'tag' extractor
* [iwara] simplify 'yield_image' usage
    perform API calls to get full 'files' list inside the function
* [iwara] add video "image" test
* [iwara] provide 'date' metadata
* [iwara] simplify 'source()'
    remove urllib.parse usage
* [iwara] small optimizations
    * get("key", {}) -> get("key") or {}
    * split("…", 1) -> partition("…")
    * use f-strings for all patterns
* [iwara] add missing 'keyarg=1' to profile() memcache decorator
* [tests/iwara] update results
* [iwara] extract more 'user' metadata
* [iwara] update default format strings
    include 'date' in filenames to order them chronologically
* [iwara] restructure image/video handling
    - use less generators
    - make processing individual media items non-fatal
* [iwara] fix login and token handling
* [iwara] add 'favorite' extractor
* [iwara] add 'following' and 'followers' extractors

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2025-07-13 21:30:25 +02:00
Mike Fährmann
1686f32a0d [scripts/init] split into separate scripts
- init.py:
    - generate initial extractor module code and test result file
    - insert new entries into modules list and site names
- generate_test_result.py:
    - generate test result for a given URL
    - insert it into the test result file generated by init.py
      (or an already existing one)
2025-07-12 21:14:29 +02:00
Mike Fährmann
fc968ebf20 [artstation] support downloading '.mview' files (#7812) 2025-07-12 20:53:16 +02:00
Mike Fährmann
518c771e21 [instagram] provide 'post_url' for stories and highlights (#7810)
fixes regression introduced in 25c63deebf
2025-07-12 17:30:06 +02:00
Mike Fährmann
98895b732f [reddit] improve archive IDs of fallback files (#7760)
prevent 'DASH...' and 'HLS...' entries
2025-07-11 22:59:44 +02:00
Mike Fährmann
755434600e [signals] implement stopping the current download with Ctrl+C (#1861)
{
    "signals-actions": {
        "SIGINT": "download:fail"
    }
}
2025-07-11 22:29:08 +02:00
Mike Fährmann
d8a370da0b [signals] update FLAGS handling 2025-07-11 22:28:26 +02:00
Mike Fährmann
2d3b6155eb [erome] restructure extractor hierarchy (#7804) 2025-07-11 22:28:26 +02:00
Mike Fährmann
2ccb9acf1a [pinterest] support 'pin.it' board redirects (#7805) 2025-07-11 22:28:26 +02:00
Mike Fährmann
8e40ea2fe2 [pinterest] match board URLs with query strings (#7805) 2025-07-11 22:28:26 +02:00