Commit Graph

6635 Commits

Author SHA1 Message Date
Mike Fährmann
931c2949ed [kemono] fix tagged creator posts
fixes regression introduced in 16b5d41ce5
2025-05-30 21:53:00 +02:00
Mike Fährmann
f9dd8318f1 [newgrounds] ignore '<script>' content in 'tags' (#7604) 2025-05-30 21:52:48 +02:00
Mike Fährmann
ec523c2c2c [mangasee] remove module 2025-05-30 18:04:55 +02:00
Mike Fährmann
783f1c1b11 add 'category-map' option
primarily to achieve some form of backwards compatibility for
922c296482
2025-05-30 17:55:49 +02:00
Mike Fährmann
922c296482 [kemono][coomer][schalenetwork] rename modules & extractors
category changes:

- kemonoparty -> kemono
- coomerparty -> coomer
- koharu      -> schalenetwork

also wanted to rename '2chan' -> 'sturdychan',
but the site's main page is still titled '2chen'
2025-05-30 17:51:49 +02:00
Mike Fährmann
2a5eec953e move cache/config/update handling before loading extractors 2025-05-30 10:15:58 +02:00
Mike Fährmann
8dc5794972 [common] return NullResponse for non-fatal requests (#7598)
Make Extractor.request(…, fatal=False) actually non-fatal
by returning an empty response instead of raising an exception
when a request fails due to connection issues.
2025-05-29 20:31:41 +02:00
Mike Fährmann
002c25d417 [util] add 'NullResponse' class 2025-05-29 20:27:40 +02:00
Mike Fährmann
539ed8fef4 [formatter] add 'D' conversion 2025-05-29 20:09:26 +02:00
Mike Fährmann
17b2910938 [util] implement 'to_datetime()' 2025-05-28 20:10:18 +02:00
Mike Fährmann
129fc00962 [pyinstaller] exclude 'pkg_resources' module (#7592) 2025-05-28 09:30:11 +02:00
Mike Fährmann
6e87ad2ccc [fanbox] extract 'archives' metadata (#7454) 2025-05-27 18:04:35 +02:00
Mike Fährmann
47af8459c2 [util] add collections of image/video/archive filename extensions 2025-05-26 23:08:27 +02:00
Mike Fährmann
45cd81639e [fanbox] return 'fileMap' files in order (#2718) 2025-05-26 22:05:51 +02:00
Mike Fährmann
204c94ccb4 [deviantart:tag] fix 'username' (#7587) 2025-05-26 21:34:06 +02:00
Mike Fährmann
a798b8971e [dl:ytdl] fix postprocessing/merge errors (#7581)
don't use a hardcoded '.part' filename extension

fixes regression introduced in 29d315e495
2025-05-26 18:56:40 +02:00
Mike Fährmann
5ebea85bd1 [patreon] send 'Referer' when downloading m3u8 videos (#7571) 2025-05-26 17:35:49 +02:00
Mike Fährmann
e05b82253a [dl:ytdl] support custom headers when fetching HLS/DASH manifests 2025-05-26 17:17:37 +02:00
Mike Fährmann
456e93d489 [tests:results] enable 'extension' and ':?' in format tests 2025-05-26 13:50:42 +02:00
Mike Fährmann
e4d139c3ce [tests:text] add cases for query params with empty values 2025-05-26 13:47:46 +02:00
Mike Fährmann
f1ab8c39fb [kemonoparty] add 'artists' extractor (#7582) 2025-05-26 11:35:55 +02:00
Mike Fährmann
e469dc0da8 [imagevenue] detect 404 image files (#7570) 2025-05-24 22:12:48 +02:00
Mike Fährmann
5e07d30d91 [arcalive] replace 'ac-p.namu' subdomains with 'ac-o.namu' (#7556)
to prevent "403 Forbidden" errors when downloading
2025-05-24 22:12:15 +02:00
Mike Fährmann
3e7a5fcb00 [dl:ytdl] fix crash when using legacy 'youtube_dl' (#6949)
https://github.com/mikf/gallery-dl/issues/6949#issuecomment-2906920039

fixes regression introduced in 29d315e495
2025-05-24 19:58:09 +02:00
Mike Fährmann
1f869c2786 [kemonoparty:discord] update server & channel metadata (#7569)
to match the Discord Extractors' names

- rename
    server       -> server_id
    channel      -> channel_id
    channel_name -> channel
- add
    server
    channel_nsfw
    channel_type
    channel_topic
    parent_id
2025-05-24 18:05:03 +02:00
Mike Fährmann
ed9c960bb9 [kemonoparty] remove '_prepare_ddosguard_cookies()' 2025-05-24 18:05:03 +02:00
Mike Fährmann
e199396872 [common] simplify 'user' extractors by using 'Dispatch' mixin 2025-05-24 18:04:53 +02:00
Mike Fährmann
c3e8af945d [sankaku] fix passing cookies (#7333)
to allow '"tags": "extended"' to work properly
2025-05-23 19:21:56 +02:00
Mike Fährmann
b81fc5c124 replace text.rextract() with rextr() 2025-05-23 18:28:58 +02:00
Mike Fährmann
fe39b7d8c8 [text] slightly improve performance of 'extract' functions
by using 'None' instead of '0' as default 'pos' value
this only saves a few nanoseconds per call, but still
2025-05-23 17:53:28 +02:00
Mike Fährmann
f3ed15573a [text] add 'rextr()' 2025-05-23 17:28:58 +02:00
Mike Fährmann
e8b7d93b43 release version 1.29.7 2025-05-23 11:18:15 +02:00
Mike Fährmann
83054325e0 [arcalive] allow overriding default User-Agent header (#7556) 2025-05-23 08:05:47 +02:00
Mike Fährmann
9c06acb385 [sankaku] compile extended 'tags' pattern only once
per extractor run
2025-05-22 22:30:41 +02:00
Mike Fährmann
b9f077a1c5 [civitai] make metadata extraction non-fatal (#7562) 2025-05-22 19:18:53 +02:00
Mike Fährmann
f5b8c25559 [pinterest] ignore 'story_pin_product_sticker_block' blocks (#7563) 2025-05-22 18:42:39 +02:00
Mike Fährmann
7b5dd61e17 [sankaku] implement support for new 'tags' categories (#7333 #7553) 2025-05-22 12:41:03 +02:00
prowlguru
b8f3ca6c4e [pixiv] Implement sanity handling for ugoira works (#4327 #6297 #7285 #7434)
* [pixiv] Implement sanity handling for ugoira works

* [pixiv] fix PEP formatting for sanity ugoira handling

* [pixiv] update Ugoira AJAX error message

* [pixiv] respect disabled 'ugoira' option

* [tests] add 'limit_sanity_level' ugoira test

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2025-05-21 22:01:09 +02:00
bradenhilton
3ba4404d21 [pixeldrain] add support for filesystem URLs (#7473) 2025-05-21 17:28:09 +02:00
Mike Fährmann
b76e7de1a7 [dl:http] fix setting 'mtime' per file (#7529)
introduce '_http_lastmodified' meta field
2025-05-21 13:50:51 +02:00
Mike Fährmann
29d315e495 [dl:ytdl] improve temp/part file handling (#6949 #7494)
- fixes files not being moved from 'part-directory' (#6949)
- fixes 'skip: enumerate' overwriting files (#7494)
2025-05-21 13:50:07 +02:00
Mike Fährmann
b13f464c9a [idolcomplex:pool] add 'Login required' warning 2025-05-21 07:50:22 +02:00
Mike Fährmann
ee3fdb60e9 [idolcomplex] fix 429 error during login by adding a 10s delay 2025-05-21 07:37:11 +02:00
Mike Fährmann
96f7f626d5 [idolcomplex] fix/update pagination logic (#7549) 2025-05-21 07:35:43 +02:00
Mike Fährmann
c929a3d3cc merge #7546: [cookies] add support for Zen Browser 2025-05-19 16:52:03 +02:00
Mike Fährmann
ac1c61f64a [fanbox] use '"browser": "firefox"' by default (#7490) 2025-05-19 16:47:43 +02:00
Mike Fährmann
0d07dec19c [instagram:stories] fix 'post_date' metadata (#7521) 2025-05-19 16:43:12 +02:00
Mike Fährmann
c28df68f68 [instagram:stories] add 'split' option (#7521) 2025-05-19 16:42:35 +02:00
Braden Hilton
27d3bc08cc [cookies] add support for Zen Browser 2025-05-19 04:42:46 +01:00
Mike Fährmann
a6de87d00e [civitai] fix exception for images without modelVersionId (#7432) 2025-05-17 20:17:57 +02:00