Commit Graph

7856 Commits

Author SHA1 Message Date
Mike Fährmann
a8376f2804 [instagram] add 'user-cache' option (#8978 #9025) 2026-02-10 12:01:37 +01:00
Mike Fährmann
ace8c50278 [imagefap] handle '/galleries?folderid=0' URLs (#9034) 2026-02-10 10:56:30 +01:00
Mike Fährmann
ce8d61df66 [imagefap] don't return anything for empty profiles (#9034) 2026-02-10 10:28:49 +01:00
Mike Fährmann
640d5f1621 [fikfap] improve URL patterns
use '[^/?#]+' for names
2026-02-10 07:56:39 +01:00
Mike Fährmann
52a5e39fc6 [reddit:user] fix user lookup when using sub view (#8228 #9032)
e.g. USER/submitted or USER/comments
fixes regression introduced in c16892a150
2026-02-09 18:57:00 +01:00
Mike Fährmann
b769dc76f4 [pornpics] fix 'search' extractor pagination (#9022)
make stop condition more lenient
2026-02-09 18:57:00 +01:00
wise-immersion
d77078d853 [fikfap] support main page post URLs (#9026)
* Update fikfap.py to allow for extracting a single post from the main page
    Current post extractor only works on links to posts
    on user pages but not on direct links to posts
* include 'singlepost' logic into existing 'post' extractor

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2026-02-09 18:54:33 +01:00
Mike Fährmann
3cf8813298 [instagram] fix errors for missing user profiles 2026-02-08 18:46:57 +01:00
Mike Fährmann
e628f4c098 [cookies] support Firefox 147+ profile paths (#8803)
https://github.com/mikf/gallery-dl/issues/8803#issuecomment-3866682047
2026-02-08 16:56:07 +01:00
Mike Fährmann
53747c63ba [instagram] fix 'avatar' & 'info' extractors (#8978)
export user lookup logic into 'user_by_screen_name' method
2026-02-08 15:54:20 +01:00
Mike Fährmann
d3c4328078 [imagefap:user] support multiple pages (#9016) 2026-02-08 11:49:11 +01:00
wise-immersion
a8636e75a1 [fikfap] add 'hashtag' extractor (#9018)
Added functionality to extract by hashtag and save to directory named after the hashtag.
2026-02-08 11:42:48 +01:00
wise-immersion
5d9b607158 [fikfap] allow for dash in usernames (#9019) 2026-02-08 11:07:00 +01:00
Mike Fährmann
56f2790626 [options] add '--config-type' command-line option
can also be set via 'GDL_CONFIG_TYPE' environment variable
9fd732afe8
2026-02-07 21:47:17 +01:00
Mike Fährmann
8eafa1564a [reddit] try to improve comment metadata (#8228)
* provide toplevel 'date'
* preserve 'submission' data
2026-02-07 21:47:17 +01:00
Mike Fährmann
935bdb6229 [reddit:user] implement 'only' option (#8228) 2026-02-07 21:47:17 +01:00
Mike Fährmann
c16892a150 [reddit:user] provide 'user' metadata field (#8228) 2026-02-07 21:47:17 +01:00
Mike Fährmann
b174bb412a [job] improve 'metadata_url' handling 2026-02-07 21:47:17 +01:00
Mike Fährmann
2d64e76223 [job] implement 'follow' option (#8752)
Follow and process URLs found in the given format string result.
2026-02-07 21:47:17 +01:00
Mike Fährmann
c978fe18d4 [text] add 'extract_urls()' helper 2026-02-07 21:47:17 +01:00
Mike Fährmann
98ef34a9be [twitter] support 'article' media (#8995) 2026-02-07 21:47:17 +01:00
Mike Fährmann
7a98a93a8e [common] only call 'skip()' & 'finalize()' when defined 2026-02-07 21:47:17 +01:00
Mike Fährmann
40e4cc62c4 [common] pass job status to 'finalize()' 2026-02-07 21:47:17 +01:00
Mike Fährmann
da887721c9 [instagram] use '/topsearch/' to fetch user information (#8978) 2026-02-07 21:47:16 +01:00
dimomarg
06da53f2c4 [cookies] add support for Floorp (#9005) 2026-02-06 14:30:19 +01:00
Mike Fährmann
da2a6a8ffa [imhentai] use alternate strategy for galleries without image data (#8951) 2026-02-06 11:51:39 +01:00
Mike Fährmann
71680feab9 [actions] implement 'flag … = skip' (#8960) 2026-02-06 10:10:38 +01:00
Mike Fährmann
22b12a1798 [tests:job] test 'parent-metadata' / '_extractor' handling 2026-02-05 22:37:30 +01:00
Mike Fährmann
f046529f28 [tests:job] add tests for DataJob 'resolve' 2026-02-05 22:37:30 +01:00
Mike Fährmann
cf6dc2e83d [config] improve --config-status
- remove 'Not Present' output
- add debug logs for exceptions
- use correct type when reporting invalid data
2026-02-05 22:37:30 +01:00
Mike Fährmann
d3adfd603b [artstation] fix & update 'challenge' extractor 2026-02-05 22:37:10 +01:00
Mike Fährmann
04442e262e [artstation] download '/8k/' images (#9003) 2026-02-05 17:32:55 +01:00
Mike Fährmann
fdc59efdda [pixiv] fix errors when using metadata options for avatar/background
(#9002)
2026-02-05 12:07:42 +01:00
Mike Fährmann
2ac55f4870 [instagram] cache '/users/web_profile_info' results on disk (#8978)
In the rare case this endpoint returns results and not a 429 error,
store them locally so they can be re-used the next time this user
is downloaded from.
2026-02-05 11:21:15 +01:00
Mike Fährmann
df8aaa7e22 [options] add '--restrict-filenames' & '--windows-filenames' 2026-02-05 09:32:24 +01:00
Mike Fährmann
09fbb3a594 [imagefap] use self.groups, remove __init__ 2026-02-05 09:04:55 +01:00
Mike Fährmann
42407afb6d [xenforo] implement '"order-posts": "reaction"' (#8997) 2026-02-04 21:57:30 +01:00
Mike Fährmann
3d36ee0e53 [common] support multiple codes/blocks for '--xff'
for example '--xff JP,CN,105.48.0.0/12'
2026-02-04 20:58:59 +01:00
Mike Fährmann
947085c6e3 [common] add '--xff' / 'geo-bypass' option 2026-02-04 18:45:02 +01:00
Mike Fährmann
b329e6e472 [xenforo] add 'attachments' & 'embeds' options 2026-02-04 16:16:55 +01:00
Mike Fährmann
9958678af1 [simpcity] extract 'reddit' media embeds (#8994) 2026-02-04 11:50:07 +01:00
Mike Fährmann
9379397eec [simpcity] extract 'tiktok' media embeds (#8994) 2026-02-04 11:20:52 +01:00
Mike Fährmann
b4351b8193 [job] more 'is None' checks 2026-02-04 11:10:49 +01:00
Mike Fährmann
49b2fa4b8e [job] fix overwriting '_extractor' (#8958) 2026-02-04 11:10:49 +01:00
Mike Fährmann
f688c59f8d [common] add __str__ method for Extractor class 2026-02-04 11:10:49 +01:00
CasualYouTuber31
cbf8ed1be9 [tiktok] always try to resolve JS challenges even if retries is set to 0 (#8993)
* [tiktok] always try to resolve JS challenges even if retries is set to 0

* add 1 to tries counter when logging to retain existing logging behavior

* clear html data in the case where resolving the challenge worked but extracting the rehydration data afterward did not
2026-02-04 08:26:28 +01:00
CasualYouTuber31
0b49712b62 [tiktok] use time cursor for story requests (#8991) 2026-02-04 07:24:21 +01:00
Mike Fährmann
614e1a514a [twitter] implement 'metadata-user' option (#8984) 2026-02-03 20:14:34 +01:00
Mike Fährmann
2cbd9740f2 enable 'metadata-url/-path/-parent/-extractor]' by default
assign values to "private" keys to not have them appear in the
standard JSON output, while still making it easier to access them
2026-02-03 19:28:33 +01:00
Mike Fährmann
f0f9575406 [job] fix 'AttributeError' when enabling 'init' for non-DownloadJob
fixes bug in 56dcd00391
2026-02-03 19:00:45 +01:00