github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,793 Commits 1 Branch 0 Tags
52ca441dc2452c05104c4437ecd58eceba04c2fa
Commit Graph

5023 Commits

Author SHA1 Message Date
Mike Fährmann
52ca441dc2 [patreon] yield metadata for paywalled posts as well (#6582) 
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-2998535431
 2025-06-24 09:09:52 +02:00
Mike Fährmann
92ac35c932 [imgbox] fix 'TypeError' when calling 'get_image_metadata()' 
fixes regression introduced in 811b665e33
 2025-06-23 18:16:46 +02:00
Mike Fährmann
97cac544ed [facebook] fix 'TypeError' when calling 'decode_all()' 
fixes regression introduced in 811b665e33
 2025-06-23 17:55:32 +02:00
Luca Russo
1055773301 [discord] support forwarded messages + handle missing threads (#7706 #7722) 
* support forwarded messages + handle missing threads
* flake
 2025-06-23 17:14:31 +02:00
Mike Fährmann
fc1ed03484 [dankefuerslesen] fix several issues (#7669) 
- return chapters in first-to-last order
- provide 'group' as list of names
- remove hardcoded 'en' and 'English' language results
- replace 'popitem()' with non-destructive 'next(iter(…))'
- 'chapter-minor' -> 'chapter_minor'
 2025-06-22 19:52:40 +02:00
Mike Fährmann
9c9eb7a3bb [tiktok] detect login page redirects (#7716) 2025-06-22 13:14:59 +02:00
Mike Fährmann
68960e29a1 [dankefuerslesen] add support (#7669) 2025-06-22 12:13:12 +02:00
Mike Fährmann
b0d7de3603 support using system certificates via 'truststore' (#6582) 
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-2989290495
 2025-06-20 19:55:01 +02:00
Mike Fährmann
74c9356442 [rule34] fix file downloads (#7697) 
replace 'api-cdn' subdomain of image files with 'wimg'
 2025-06-20 15:07:10 +02:00
Mike Fährmann
dba45923c8 [dl:http] fail downloads with HTML content (#7697) 
add 'validate-html' option
 2025-06-20 14:34:40 +02:00
Mike Fährmann
e8d28d3d0d merge #7699: [warosu] handle missing images by checking hostname (#7698) 2025-06-20 13:17:44 +02:00
NecRaul
5ba7c98bc2 [warosu] Simpler/less costly hostname check 2025-06-20 14:27:21 +04:00
NecRaul
3c85032b9b [warosu] Handle missing images in the extractor by checking hostname 2025-06-20 09:34:49 +04:00
Mike Fährmann
4279928d0b [gelbooru_v02] extract 'total' / 'search_count' metadata (#7689) 2025-06-19 19:15:27 +02:00
Mike Fährmann
fcd1b8a155 [common] add a 'kwdict' member to extractor instances 
to allow setting general metadata at any point and without having to
rely on a manually implemented 'metadata()' method
 2025-06-19 19:08:35 +02:00
Mike Fährmann
a80d55d974 [common] improve 'user-agent' override logic 
Prevent a general 'user-agent' setting, i.e. extractor.user-agent, from
overriding the User-Agent header set by an extractor's 'browser'
option, default or otherwise, meaning only extractor-level or top-level
'user-agent' settings will override the 'browser' User-Agent.

https://github.com/mikf/gallery-dl/issues/7382#issuecomment-2985296321
 2025-06-19 18:58:39 +02:00
Mike Fährmann
4f7a0e5d02 [arcalive] fix download URLs (#7678) 
https://github.com/danbooru/danbooru/pull/6041
 2025-06-18 21:12:18 +02:00
Mike Fährmann
b0580aba86 update 'match.lastindex' usage 2025-06-18 20:24:13 +02:00
Mike Fährmann
41191bb60a 'match.group(N)' -> 'match[N]' (#7671) 
2.5x faster
 2025-06-18 13:05:58 +02:00
pocketinternet
3ea244eebb [webtoons] add 'thumbnails' option (#6468 #7441) 
* Update webtoons.py
    Added thumbnail download capability which defaults to false
* Update configuration.rst
    Added documentation for webtoon thumbnail option
* extract thumbnails in GalleryExtractor.assets()
* simplify & fix flake8
* include 'type' in default filenames
* add test
* update docs

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-06-17 19:34:58 +02:00
Mike Fährmann
d0b1da0fa7 [tiktok] handle exceptions when extracting avatars (#7682) 2025-06-17 12:21:20 +02:00
Mike Fährmann
efa2ab5903 [common] allow GalleryExtractors to return additional asset files 
like thumbnails or covers
 2025-06-16 22:45:52 +02:00
SpiffyChatterbox
e0f65be36b [nudostar] add support (#5735 #6556) 
* Drafting initial basic extractor layout
* Better debug logging
* Update nudostar.py
    Still tinkering
* Update nudostar.py
    Basic extractor is working. Now starting on Gallery
* Update nudostar.py
    Still a work in progress.
    Got individual posts working, galleries are not.
* Update nudostar.py
* Site now appears working. Added Tests.
* PEP Updates
* PEP - Line Length Updates
* Update nudostar.py
    Resolving PEP8 issues.
* update 'gallery' extractor, rename to 'model'
* update 'image' extractor
* expand tests
* update docs/supportedsites

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-06-16 19:21:49 +02:00
missionfloyd
13cb031fe2 [girlsreleased] add support (#6200) 
* [girlsreleased] add support
* Lint
* Change "galleries" to "sets"
    As it is on the site
* Add pagination
    Add tests
* Fix tests
* Remove leftover print()
* Don't remove first set
* Yield pages
* Add filename metadata
* [girlsreleased] Refactor
* Return models as array
* Add filename numbering
    Add date metadata
* Add URL metadata
* Spawn set extractor the right way
* Adjust model/site regex
* update
    - restructure some code
    - remove constructors
    - use f-strings
* expand tests
* update docs/supportedsites
 2025-06-16 19:18:19 +02:00
SpiffyChatterbox
48ac41605d [redbust] add support (#6759 #6918 #7043) 
* init - Redbust.com Support
* Added Test
    Could use a second set of eyes on this
* update 'gallery' extractor
    - extract more metadata
    - simplify image extraction
    - support legacy galleries
* add tests
* update 'image' extractor
* add 'tag' extractor
* add 'archive' extractor
* restrict 'image' extractor pattern
* update docs/supportedsites
* replace quotes inside f-string

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-06-16 12:10:42 +02:00
Mike Fährmann
d9432ee297 [warosu] restore correct 'now' values 2025-06-16 12:00:59 +02:00
NecRaul
3bc6bc7c77 [warosu] Single quotes when string has a quotation mark 2025-06-16 12:50:20 +04:00
NecRaul
f56e810f42 [warosu] Attribute fix 2025-06-16 12:33:16 +04:00
Mike Fährmann
0a3fac2dfe merge #7664: [archivedmoe] redirect URL fixes (#7652) 2025-06-15 10:03:34 +02:00
Mike Fährmann
b245218c1d [archivedmoe] reword some comments and variable names 2025-06-15 10:00:45 +02:00
hunter-gatherer8
96f5cfb305 [girlswithmuscle] add support (#4493 #6016) 
* [girlswithmuscle] init
* [girlswithmuscle]: fix metadata extraction (site layout change)
* [girlswithmuscle]: fix tags extraction (site layout change)
* update login code
* update 'post' extractor
* update 'gallery' extractor, rename to 'search' extractor
* update docs
* add test cases

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-06-14 23:05:49 +02:00
Mike Fährmann
b583891df6 [pixiv] remove '/jump.php' from *all* AJAX caption links (#4327) 
https://github.com/mikf/gallery-dl/issues/4327#issuecomment-2969765775
 2025-06-13 21:10:51 +02:00
NecRaul
6668acf91e [archivedmoe] Sort boards alphabetically 2025-06-13 19:29:47 +04:00
NecRaul
3a4e19d284 [archivedmoe] Simplify board extraction from url 2025-06-13 18:44:02 +04:00
NecRaul
a7aa18a8c1 [archivedmoe] remove unnecessary logging 2025-06-13 18:28:21 +04:00
NecRaul
8b2adeb41e [archivedmoe] simplify board URL redirection logic 2025-06-13 18:26:39 +04:00
NecRaul
05081dea2e Lint with flake8 2025-06-13 17:56:43 +04:00
NecRaul
223fe960a0 [archivedmoe] redirect URL changes (again) 
Redirects to warosu.org instead of 4chan's cdn for certain boards
Redirects to archive.4plebs.org instead of 4chan's cdn for /tg/
Slices the filename only if it's redirecting to certain archives
 2025-06-13 17:43:16 +04:00
Mike Fährmann
9d3cf67f3e [pinterest] remove excess whitespace from 'description' fields (#4335) 
and 'closeup_unified_description' & 'closeup_description'
 2025-06-13 13:11:18 +02:00
Mike Fährmann
e08ec7e083 update copyright notices 2025-06-13 00:03:41 +02:00
Mike Fährmann
811b665e33 remove @staticmethod decorators 
There might have been a time when calling a static method was faster
than a regular method, but that is no longer the case. According to
micro-benchmarks, it is 70% slower in CPython 3.13 and it also makes
executing the code of a class definition slower.
 2025-06-12 22:50:52 +02:00
Mike Fährmann
e41c317039 [fantia] prevent '.html' file downloads 2025-06-12 20:41:47 +02:00
Mike Fährmann
e2d104a110 [twitter] extract 'source_id' and 'source_user' metadata (#7470 #7640) 2025-06-12 18:59:22 +02:00
Mike Fährmann
06e2f2cd91 [twitter] restructure media data extraction 2025-06-12 18:53:15 +02:00
Mike Fährmann
56ea27c474 [blogger] move original/s0 URL code into a separate function 2025-06-12 17:07:56 +02:00
Mike Fährmann
16fc5e0d68 [batoto] fix downloading manga with alerts/notices (#7657) 
and improve alert message extraction
 2025-06-12 08:26:26 +02:00
Mike Fährmann
a14671992c [sexcom] prevent '.css' file downloads (#7632) 
by detecting homepage redirects
and improve redirect handling in general
 2025-06-11 22:32:08 +02:00
Mike Fährmann
0df083b208 [vk] prevent '404 Not Found' errors for file downloads 
only strip query parameters when regex substitution applies
 2025-06-11 22:32:08 +02:00
Mike Fährmann
d065452ba3 merge #7653: [archivedmoe] fix redirection issue (#7652) 2025-06-11 20:04:42 +02:00
Mike Fährmann
80599fa610 [vk] fix 'user' metadata extraction 
add boolean 'group' field
 2025-06-11 20:01:27 +02:00