github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,869 Commits 1 Branch 0 Tags
0f41f343f4a71550d362e0e898f52d33bbce9949
Commit Graph

319 Commits

Author SHA1 Message Date
Mike Fährmann
0f41f343f4 implement linear/exponential backoff for 'sleep-429' 2026-02-12 19:23:29 +01:00
Mike Fährmann
12f5e24ab5 use sets for ' in { ... }' checks 2026-02-11 22:55:01 +01:00
Mike Fährmann
7a98a93a8e [common] only call 'skip()' & 'finalize()' when defined 2026-02-07 21:47:17 +01:00
Mike Fährmann
40e4cc62c4 [common] pass job status to 'finalize()' 2026-02-07 21:47:17 +01:00
Mike Fährmann
947085c6e3 [common] add '--xff' / 'geo-bypass' option 2026-02-04 18:45:02 +01:00
Mike Fährmann
f688c59f8d [common] add __str__ method for Extractor class 2026-02-04 11:10:49 +01:00
Mike Fährmann
efa301fcc2 Merge branch 'res' 2026-02-01 19:52:20 +01:00
Mike Fährmann
343981ac1c [common] add 'utils()' method 2026-02-01 18:48:17 +01:00
Mike Fährmann
1a8ccbe720 [common] use f-string in BaseExtractor.update() 2026-02-01 08:54:15 +01:00
Mike Fährmann
cd83be41c5 [common] allow Dispatch 'alt' extractors to use custom URLs 2026-01-25 11:15:30 +01:00
Mike Fährmann
4798ac4836 [common] implement 'parent-session' 2026-01-21 20:33:08 +01:00
Mike Fährmann
78da7edde8 [common] add 'googlebot' User-Agent preset 2026-01-21 17:57:26 +01:00
Mike Fährmann
366b0750a8 [common] use extractor subcategory for 'notfound=True' 2026-01-19 11:19:35 +01:00
Mike Fährmann
2cfe2b3b04 [common] implement '"user-agent": "+PRESET"' 
support using a 'User-Agent' header preset, e.g. "+firefox"
 2026-01-03 21:07:45 +01:00
Mike Fährmann
00c6821a3f replace 2-element f-strings with simple '+' concatenations 
Python's 'ast' module and its 'NodeVisitor' class
were incredibly helpful in identifying these
 2025-12-22 11:26:04 +01:00
Mike Fährmann
b1b70528a6 [common] use 'parent' value as 'parent-metadata' default (#8525 #8604) 
enable 'parent-metadata' by default for
'chevereto' & 'imagehost' (album/gallery) extractors
 2025-12-08 20:07:56 +01:00
Mike Fährmann
d1de48e7b2 remove Message.Version 2025-12-07 17:13:25 +01:00
Mike Fährmann
968597a302 yield 3-tuples for Message.Directory 
adapt tuples to the same length and semantics as other messages
 2025-12-05 21:39:52 +01:00
Mike Fährmann
89ece8f900 [common] allow general ISO 8601 values for 'date-min' & '-max' 
and parse them as UTC times instead of local time
 2025-11-09 23:00:09 +01:00
Mike Fährmann
62e9d75611 [common] fix 'AttributeError' 
fixes regression introduced in 98d3354575
 2025-10-29 11:38:26 +01:00
Mike Fährmann
da4b8c1e83 [wikimedia] fix 'limit' config lookup 
rework config handling in general
 2025-10-23 20:16:46 +02:00
Mike Fährmann
98d3354575 [wikimedia] implement config lookup for fandom/wikigg sites (#7283) 
{
    "extractor": {
        "fandom": {
            "filename": "..."
        }
    }
}
 2025-10-23 20:14:56 +02:00
Mike Fährmann
085616e0a8 [dt] replace 'text.parse_datetime()' & 'text.parse_timestamp()' 2025-10-17 17:43:06 +02:00
Mike Fährmann
69f7cfdd0c [dt] replace 'datetime' imports 2025-10-16 11:42:42 +02:00
Mike Fährmann
0eb3c8a994 [dt] replace 'util' datetime functions 2025-10-15 21:30:38 +02:00
Mike Fährmann
99d5c521d1 use 'encoding="utf-8"' when opening files in text mode (#8376) 2025-10-09 09:54:18 +02:00
Mike Fährmann
c497d96c3f [common] disable 'interval' delay for 'request_loocation()' 2025-10-03 22:29:03 +02:00
Mike Fährmann
0dfc3b18d3 [common] support single quotes in '_extract_jsonld' & '…_nextdata' 2025-09-11 20:21:55 +02:00
Mike Fährmann
977e1bece3 [common] add optional 'password' argument to '_get_auth_info()' 2025-09-09 22:06:13 +02:00
Mike Fährmann
5a5857bafb [common] prevent exception when using empty 'user-agent' (#8116) 
use '"headers": {"User-Agent": ""}' to send an empty User-Agent header
 2025-08-26 09:08:49 +02:00
Mike Fährmann
a358d13062 [common] add 'expected' argument to 'request()' 2025-08-22 16:09:31 +02:00
Mike Fährmann
ea80fc80c8 [common] improve error message for non-Netscape cookie files (#8014) 2025-08-13 08:03:01 +02:00
Mike Fährmann
5bc198a7e6 [common] try subprocess.Popen when webbrowser.get fails (#7947) 2025-08-08 21:25:42 +02:00
Mike Fährmann
16acfbd1e7 [common] implement '"user-agent": "@BROWSER"' (#7947) 
support automatically fetching the 'User-Agent' header
of a specific installed browser, e.g. "@firefox"
 2025-08-07 20:58:29 +02:00
Mike Fährmann
7f414cdb4e [common] add 'interval' argument to 'request()' 
to disable 'request_interval' sleeps for "background" requests
 2025-08-01 20:40:20 +02:00
Mike Fährmann
a097a373a9 simplify if statements by using walrus operators (#7671) 2025-07-22 20:57:54 +02:00
Mike Fährmann
629db56b12 [common] detect CF & DG challenges in request_json & _xml (#7833) 
even when called when fatal=False
 2025-07-17 17:17:01 +02:00
Mike Fährmann
e68dc960c2 [pixiv] add workaround for 'novel-*' includes (#7746) 2025-07-11 22:27:45 +02:00
Mike Fährmann
267661680c add 'pixiv-novel' compat categories (#7746) 2025-07-11 11:00:19 +02:00
Mike Fährmann
096bc6f784 replace more '.format(…)' calls with f-strings (#7671) 2025-07-10 21:17:17 +02:00
Mike Fährmann
d8ef1d693f rename 'StopExtraction' to 'AbortExtraction' 
for cases where StopExtraction was used to report errors
 2025-07-09 21:07:28 +02:00
Mike Fährmann
f17ed0569a [common] raise ChallengeError for CF & DDG challenges (#1945) 
this sets bit 0x08 in the exit status bitmask
 2025-07-09 18:58:21 +02:00
Mike Fährmann
05b0a25120 [common] update 'browser' user-agents and headers 
support Firefox 140 ESR and Chrome/Chromium 138
 2025-07-06 19:41:37 +02:00
Mike Fährmann
1bbacba4ed [common] introduce 'status' attribute to Extractors 
allows reporting error codes for exceptions that are not handled
by the Job.run() try-except block

- fixes Job.status being 0 in certain situations even when errors occurred
- fixes some URLs not getting written to -e/--error-file (#7758)
 2025-07-05 21:33:01 +02:00
Mike Fährmann
95338ff0ec replace 'result' with 'results' for lists 
more consistent names
 2025-06-30 12:10:57 +02:00
Mike Fährmann
9dbe33b6de replace old %-formatted and .format(…) strings with f-strings (#7671) 
mostly using flynt
https://github.com/ikamensh/flynt
 2025-06-29 17:50:19 +02:00
Mike Fährmann
d969dbbab1 [common] fix bug in df6f4e53 when first group is empty 2025-06-28 09:02:33 +02:00
Mike Fährmann
df6f4e5307 [common] only auto-set page_url when first group starts with / 2025-06-26 23:35:53 +02:00
Mike Fährmann
26e81e4162 [common] rename 'gallery_url'/'manga_url' to 'page_url 2025-06-26 22:06:57 +02:00
Mike Fährmann
b0d7de3603 support using system certificates via 'truststore' (#6582) 
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-2989290495
 2025-06-20 19:55:01 +02:00