github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,484 Commits 1 Branch 0 Tags
cff04f1181685dce7d8191374eeaba7d6b1a82be
Commit Graph

7484 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
cff04f1181 [dl:http] implement '_http_segmented' (#8602) 2025-11-30 19:23:10 +01:00
Mike Fährmann
5f38d8ab25 [twitter] increase 'retries-api' default to '9' (#8317) 2025-11-30 16:52:26 +01:00
Mike Fährmann
76765340eb [2ch] support 'org' TLD (#8629) 2025-11-30 16:49:23 +01:00
Mike Fährmann
f51544618e [audiochan] add 'search' extractor (#8602) 2025-11-29 20:01:15 +01:00
Mike Fährmann
9bf52ab8c5 [civitai] implement extracting 'tags' metadata (#8626) 2025-11-29 18:37:57 +01:00
Mike Fährmann
5c7a27e003 [schalenetwork] fix 'tags' categories (#8625) 2025-11-29 09:29:21 +01:00
Mike Fährmann
994cc34f75 [audiochan] add initial support (#8602) 2025-11-28 21:46:22 +01:00
Mike Fährmann
e589d6f6ee [pp:exec] add 'verbose' option (#7743) 2025-11-28 19:55:04 +01:00
Mike Fährmann
bb16e036e4 [twitter] reset 'api_tries' after successful API call (#8317) 2025-11-28 16:48:59 +01:00
Mike Fährmann
5e3284f0e6 [actions] add 'LoggerAdapter.traceback()' method (#8317) 
8c62be343e
 2025-11-28 14:15:20 +01:00
spicybiguy
11151033f6 [motherless] fix 'gallery_title' extraction (#8605) 
* Update motherless.py for title selector

Updated the selector for the title property for Motherless galleries to be an h2 instead of an h1 to reflect changes on the site

* fix 'gallery_title' extraction

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-11-28 10:53:16 +01:00
Mike Fährmann
acf281a46a [twitter] implement 'retries-api' option (#8317) 
retry API requests when encountering server-related errors
 2025-11-27 19:23:27 +01:00
Mike Fährmann
1e7f4ee178 [output] add 'defer' option for logging files (#8523) 
- defer file creation of error files by default
- implement custom FileHandler class
  to support deferred directory creation
 2025-11-27 19:23:04 +01:00
Mike Fährmann
0b05254055 [bunkr] detect when album is deleted mid-download (#8619) 2025-11-27 11:39:32 +01:00
Mike Fährmann
b3aefd4a99 [twitter] restore better user NotFoundError messages (#8621) 
fixes regression introduced in 5747dbf00c
 2025-11-27 11:39:32 +01:00
Mike Fährmann
36fa543715 [twitter] add 'search-results' option (#8613) 2025-11-27 11:31:07 +01:00
Mike Fährmann
1dc7955ba2 [simpcity] extract "click to load media" URLs (#8609) 2025-11-26 18:35:58 +01:00
Mike Fährmann
25ac5099cf [simpcity] extract attachment/inline files (#8560) 2025-11-26 18:18:41 +01:00
Mike Fährmann
aac1d8f36d [simpcity] fix 'content' for first post of a thread 2025-11-24 20:59:15 +01:00
Mike Fährmann
60b3f66950 [belazon] fix starting from a specific page 
with the default '"order-posts": "desc"'
 2025-11-23 18:12:10 +01:00
Mike Fährmann
8ae4c553d2 [simpcity] fix starting from a specific page (#8599) 
with the default '"order-posts": "desc"'
 2025-11-23 18:08:32 +01:00
Mike Fährmann
f06772e56e [tiktok] ignore empty 'music' entries (#8571) 2025-11-21 20:06:55 +01:00
Octavia Togami
685777e43e [patreon] include full metadata with each URL (#4286 #8498) 
* include full metadata object with each url
    Fixes #4286
* rename 'url_metadata' to 'file'
 2025-11-20 19:30:13 +01:00
MyFinalBellyache
d74809c538 [eporner] add support (#8581) 
* [eporner] Add support
* Line-length <80 chars
* Fix flake8 indentation
* update
    - generalize example URL
    - strip "Photo Gallery" from 'title'
    - extract 'id'
* update supportedsites
* update test results
 2025-11-20 18:03:54 +01:00
MyFinalBellyache
2bff6828e1 [scripts/init] Fix error running with default values (#8583) 
Fixes:
```sh
python scripts/init.py newsite https://newsite.com
# [INFO] gallery_dl/extractor/newsite.py
# [ERROR] TypeError: 'NoneType' object is not iterable
# Traceback (most recent call last):
#   File "/gallery-dl/scripts/init.py", line 58, in init_extractor
#     fp.write(func(args))
#              ^^^^^^^^^^
#   File "/gallery-dl/scripts/init.py", line 88, in generate_module
#     {generate_extractors(args)}\
#      ^^^^^^^^^^^^^^^^^^^^^^^^^
#   File "/gallery-dl/scripts/init.py", line 108, in generate_extractors_basic
#     for subcat in args.subcategories:
# TypeError: 'NoneType' object is not iterable
# [INFO] gallery_dl/extractor/__init__.py
# [INFO] test/results/newsite.py
```
 2025-11-20 11:00:35 +01:00
Mike Fährmann
7b059958c2 [fansly] handle posts without accountId/contentId/attachments (#8572) 2025-11-19 20:46:34 +01:00
Mike Fährmann
e91337d5c3 [Danbooru] add 'media-asset' extractor (#8580) 2025-11-19 20:46:22 +01:00
Mike Fährmann
89be2f45e4 [mangadex] add 'data-saver' option (#8573) 2025-11-19 19:54:53 +01:00
Mike Fährmann
aa39770783 [aryion:search] simplify further 
- skip 'build_query()' step
- add underscores to prefixes
 2025-11-19 19:54:53 +01:00
MyFinalBellyache
a9687d2928 [pornstarstube] add support (#8576) 
* Add support for pornstars.tube
* update
    - adjust pattern/example/page_url
    - extract '' & '' metadata
    - prevent unnecessary request
    - fix result URLs
    - return list comprehension result
* update supportedsites
* update test results
* fix extractor name

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-11-19 16:25:01 +01:00
MyFinalBellyache
cc7003a14c [okporn] add support (#8575) 
* Add support for ok.porn
* update
    - adjust pattern/example/page_url
    - prevent unnecessary request
    - extract 'tags'
    - text.extract -> text.extr
    - return list comprehension result
* update supportedsites name
* update test results

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-11-19 15:49:13 +01:00
MyFinalBellyache
157a37b053 [sxypix] add support (#4507 #8391 #8574) 
* Add support for sxypix.com
* update & simplify
    - text.extract -> text.extr
    - self.request().json() -> self.request_json()
    - use self.groups
    - remove __init__
    - list comprehension
* update test results

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-11-19 10:30:29 +01:00
vorsatile
991fe0f2a7 [aryion] add 'search' extractor (#8567) 
* [aryion] Implement search extractor.
* [aryion] Update capabilities.
* [aryion] Adjust example.
* fix flake8 errors

* update & simplify
    - use existing '_pagination_next()'
    - remove '_pagination_search()'
    - update 'search[…]' metadata
* add tests

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-11-18 20:57:08 +01:00
Mike Fährmann
e60007add6 [xasiat] fix 'IndexError' for albums without category (#8569) 2025-11-18 19:38:30 +01:00
Mike Fährmann
0e0f446368 [rawkuma] update extractors to new site layout (#8568) 2025-11-18 19:38:30 +01:00
Mike Fährmann
9e6b87f850 [fanbox:redirect] disable cookie usage (#8565) 2025-11-18 19:38:14 +01:00
Mike Fährmann
3b33c25378 [reddit] use REST API by default (#8559) 2025-11-17 16:24:54 +01:00
Mike Fährmann
6ee7096558 [instagram] fix "AttributeError: 'videos_dash'" (#8561) 
fixes regression introduced in ab3e34e9c3
 2025-11-15 23:23:31 +01:00
SpiffyChatterbox
f6e67116f2 [cfake] add support (#707 #6021 #8549 #8430) 
* Create cfake.py
* Update __init__.py
* Create cfake.py tests
* update
    - simplify & combine code
    - adjust 'pattern': use '[^/?#]', match lines and groups
    - generalize example URLs
    - update default filenames
* update docs/supportedsites
* update test results

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-11-15 20:55:00 +01:00
Mike Fährmann
2578f7b5c1 [flickr] extract public API key from website (#7564 #7649 #7700 #8553) 
this breaks 'oauth:flickr' with the default key,
but it allows downloading without custom key / Flickr Pro
 2025-11-14 11:38:28 +01:00
Mike Fährmann
5da4fb38c6 [reddit] fix "KeyError: 'media_metadata'" for embeds (#8551) 2025-11-14 08:52:57 +01:00
Mike Fährmann
e6930ebb23 [instagram] improve error for non-existent profiles (#8550) 2025-11-13 20:34:46 +01:00
Mike Fährmann
5cda64c8d8 [job] apply 'keywords-eval' to 'keywords-global' values (#6582) 
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3527246051
 2025-11-13 20:29:31 +01:00
Mike Fährmann
e840e3c680 [docs/configuration] update 'twitter.users' example (#8548) 2025-11-12 22:09:00 +01:00
Mike Fährmann
1b6353064d [rule34us:tag] support URLs with empty 'q' query parameter (#8546) 2025-11-12 21:04:11 +01:00
Mike Fährmann
65c4f00939 [imagehosts] provide 'post_url' metadata (#6582) 
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3520249958
 2025-11-12 19:10:57 +01:00
Mike Fährmann
359681a8e4 [pixhost] extract 'directory' metadata (#6582) 
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3520249958
 2025-11-12 19:09:46 +01:00
Mike Fährmann
691775d0fb [job] add 'keywords-global' option (#6582) 
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3505732704
 2025-11-12 18:48:01 +01:00
Mike Fährmann
d7e1351987 [bellazon] improve 'filename' & 'extension' (#8544) 
https://github.com/mikf/gallery-dl/issues/8544#issuecomment-3522617104

- use 'extension' from file URL (except attachments)
- strip everything after the first '.' from non-URL filenames
 2025-11-12 18:38:44 +01:00
Mike Fährmann
da47214ca0 [bellazon] remove query parameters from attachment IDs (#8544) 2025-11-12 17:46:16 +01:00