Commit Graph

7117 Commits

Author SHA1 Message Date
Mike Fährmann
04442e262e [artstation] download '/8k/' images (#9003) 2026-02-05 17:32:55 +01:00
Mike Fährmann
fdc59efdda [pixiv] fix errors when using metadata options for avatar/background
(#9002)
2026-02-05 12:07:42 +01:00
Mike Fährmann
2ac55f4870 [instagram] cache '/users/web_profile_info' results on disk (#8978)
In the rare case this endpoint returns results and not a 429 error,
store them locally so they can be re-used the next time this user
is downloaded from.
2026-02-05 11:21:15 +01:00
Mike Fährmann
df8aaa7e22 [options] add '--restrict-filenames' & '--windows-filenames' 2026-02-05 09:32:24 +01:00
Mike Fährmann
09fbb3a594 [imagefap] use self.groups, remove __init__ 2026-02-05 09:04:55 +01:00
Mike Fährmann
42407afb6d [xenforo] implement '"order-posts": "reaction"' (#8997) 2026-02-04 21:57:30 +01:00
Mike Fährmann
3d36ee0e53 [common] support multiple codes/blocks for '--xff'
for example '--xff JP,CN,105.48.0.0/12'
2026-02-04 20:58:59 +01:00
Mike Fährmann
947085c6e3 [common] add '--xff' / 'geo-bypass' option 2026-02-04 18:45:02 +01:00
Mike Fährmann
b329e6e472 [xenforo] add 'attachments' & 'embeds' options 2026-02-04 16:16:55 +01:00
Mike Fährmann
9958678af1 [simpcity] extract 'reddit' media embeds (#8994) 2026-02-04 11:50:07 +01:00
Mike Fährmann
9379397eec [simpcity] extract 'tiktok' media embeds (#8994) 2026-02-04 11:20:52 +01:00
Mike Fährmann
b4351b8193 [job] more 'is None' checks 2026-02-04 11:10:49 +01:00
Mike Fährmann
49b2fa4b8e [job] fix overwriting '_extractor' (#8958) 2026-02-04 11:10:49 +01:00
Mike Fährmann
f688c59f8d [common] add __str__ method for Extractor class 2026-02-04 11:10:49 +01:00
CasualYouTuber31
cbf8ed1be9 [tiktok] always try to resolve JS challenges even if retries is set to 0 (#8993)
* [tiktok] always try to resolve JS challenges even if retries is set to 0

* add 1 to tries counter when logging to retain existing logging behavior

* clear html data in the case where resolving the challenge worked but extracting the rehydration data afterward did not
2026-02-04 08:26:28 +01:00
CasualYouTuber31
0b49712b62 [tiktok] use time cursor for story requests (#8991) 2026-02-04 07:24:21 +01:00
Mike Fährmann
614e1a514a [twitter] implement 'metadata-user' option (#8984) 2026-02-03 20:14:34 +01:00
Mike Fährmann
2cbd9740f2 enable 'metadata-url/-path/-parent/-extractor]' by default
assign values to "private" keys to not have them appear in the
standard JSON output, while still making it easier to access them
2026-02-03 19:28:33 +01:00
Mike Fährmann
f0f9575406 [job] fix 'AttributeError' when enabling 'init' for non-DownloadJob
fixes bug in 56dcd00391
2026-02-03 19:00:45 +01:00
Mike Fährmann
0be3383110 [formatter] add 'q' & 'Q' conversions - URL-en/decode values 2026-02-03 17:35:05 +01:00
Mike Fährmann
5ddee423ed [exhentai][schalenetwork] enable 'tags' categories by default 2026-02-03 10:49:35 +01:00
Mike Fährmann
d232f0d252 [gofile][pixeldrain] enable 'recursive' by default 2026-02-03 09:11:47 +01:00
Mike Fährmann
0c04090ff4 [util] set flags in range predicates
use flags to stop extractors immediately when hitting the upper range
limit instead of raising an exception one predicate check later
2026-02-02 21:48:45 +01:00
CasualYouTuber31
e44c5bd707 [tiktok] identify when user accounts do not exist (#8977) 2026-02-02 19:30:12 +01:00
CasualYouTuber31
7c8fbb6fa5 [tiktok] do not exit early when rolling back cursor (#8968)
* [tiktok] do not exit account extraction early when we need to manually roll back the cursor
* [tiktok] fix rehydration data error string formatting
2026-02-02 19:29:24 +01:00
CasualYouTuber31
a7cc447f51 [tiktok] fix outdated error message (#8979) 2026-02-02 19:26:46 +01:00
Mike Fährmann
cbb9264ddc use tempfile when updating input files (#8981)
0d72789aa3
2026-02-02 19:15:26 +01:00
Mike Fährmann
87323fd71e rename 'image-*' & 'chapter-*' options to 'file-*' & 'child-*'
keep support for both names, e.g. file-filter & image-filter,
while prioritizing the new names
2026-02-02 18:01:45 +01:00
Mike Fährmann
153e589dd0 [twitter] implement '"search-pagination": "date"' 2026-02-02 16:19:48 +01:00
Mike Fährmann
a95cf92d61 [twitter] add & use '_tweetid_to_datetime' method
use 'dt' functions directly
2026-02-02 16:11:59 +01:00
Mike Fährmann
39fb51dade [mangafire] export 'VRF' functions 2026-02-02 11:45:12 +01:00
Mike Fährmann
5c8bea3cc0 [8chan] fail downloads of 'POW' images (#8975) 2026-02-02 10:51:24 +01:00
Mike Fährmann
a2c79ba057 [8chan] skip 'TOS' cookie name lookup if already present 2026-02-02 10:51:20 +01:00
Mike Fährmann
17e1d25784 [scrolller] add 'user' extractor (#8961) 2026-02-02 09:09:50 +01:00
Mike Fährmann
f99c8f5ad9 [xhamster] fix user profile extraction (#8974) 2026-02-02 08:53:24 +01:00
Mike Fährmann
44e18f9b2f [tsumino] remove module
" Tsumino - The End
  We're shutting Tsumino down. "
2026-02-01 22:15:06 +01:00
Mike Fährmann
c42a5dce5c [pp] add 'child' & 'child-after' events 2026-02-01 22:14:14 +01:00
Mike Fährmann
56dcd00391 [job] add 'init' option 2026-02-01 20:48:37 +01:00
Mike Fährmann
efa301fcc2 Merge branch 'res' 2026-02-01 19:52:20 +01:00
Mike Fährmann
451ed23737 [discord:server-search] use 'max_id' for pagination
'offset' is limited to 10_000
'max_id' is hopefully not
2026-02-01 19:48:15 +01:00
Mike Fährmann
73bf99612a [scrolller] move GraphQL queries 2026-02-01 19:18:52 +01:00
Mike Fährmann
cc645984a4 [luscious] export GraphQL queries 2026-02-01 19:18:14 +01:00
Mike Fährmann
0c24955507 [mangapark] export GraphQL queries 2026-02-01 19:18:10 +01:00
Mike Fährmann
40a4ff935a [500px] export GraphQL queries 2026-02-01 19:16:14 +01:00
Mike Fährmann
51d9fd2f4d [behance] export GraphQL queries 2026-02-01 19:13:38 +01:00
Mike Fährmann
1c2e2d5d08 [deviantart] export journal templates 2026-02-01 18:59:31 +01:00
Mike Fährmann
3d114dbc67 [deviantart] export 'tiptap' functions 2026-02-01 18:53:22 +01:00
Mike Fährmann
20ef39be45 [tsumino] export 'jsurl' code 2026-02-01 18:50:09 +01:00
Mike Fährmann
7692d31a57 [twitter] move transaction_id.py 2026-02-01 18:48:30 +01:00
Mike Fährmann
343981ac1c [common] add 'utils()' method 2026-02-01 18:48:17 +01:00