Commit Graph

3156 Commits

Author SHA1 Message Date
Mike Fährmann
26c3292538 [twitter] disable TLS 1.2 ciphers by default (#3522) 2023-01-13 16:05:43 +01:00
Mike Fährmann
18fe4b334d [twitter] remove 'tweet_search_mode' from search parameters (#3522)
and update API root and general query parameters
2023-01-13 15:50:46 +01:00
Mike Fährmann
85bd1cbc89 [kemonoparty] fix regression from 473bd380 (#3519)
- do not access 'response.content' unless necessary
- only validate responses if filename extensions differ
2023-01-11 15:25:01 +01:00
Mike Fährmann
473bd380c8 [kemonoparty] reject invalid/empty files (#3510) 2023-01-10 19:04:47 +01:00
Mike Fährmann
4833ec323e [imagefap] add 'folder' extractor (#3504) 2023-01-08 16:57:31 +01:00
Mike Fährmann
362cd6991b [pixiv] implement 'metadata-bookmark' option (#3417) 2023-01-07 23:19:43 +01:00
Mike Fährmann
2142b9c7ae merge #3503: [myhentaigallery] handle whitespace before title tag 2023-01-07 15:15:11 +01:00
Mike Fährmann
3a0450adbf [behance] use default delay between requests (#2507) 2023-01-07 14:49:26 +01:00
Mike Fährmann
2cae4567ba [telegraph] fix file URLs (#3506) 2023-01-07 14:35:06 +01:00
Mike Fährmann
cbaeee9533 [imagefap] warn about redirects to '/human-verification' (#1140) 2023-01-07 13:04:42 +01:00
Mike Fährmann
435de1329a [imagefap] use default delay between requests (#1140) 2023-01-07 12:59:09 +01:00
Erik Rimskog
a8a982359e [myhentaigallery] handle whitespace before the title tag 2023-01-06 16:33:04 +01:00
Mike Fährmann
d1dd52349a merge #3189: [tcbscans] add 'chapter' and 'manga' extractors 2023-01-06 13:42:27 +01:00
Mike Fährmann
2f31d21509 merge #3455: [twitter] apply tweet type checks before uniqueness check 2023-01-06 13:32:50 +01:00
enduser420
e8541a131d [tcbscans] add 'chapter' and 'manga' extractors 2023-01-06 16:16:31 +05:30
Mike Fährmann
9695c4e88d emit debug logging message when loading cookies from file
attempt nr. 2
no idea how I managed to remove 6514828d in a918ce29
2023-01-06 11:13:44 +01:00
Mike Fährmann
30a31836e7 merge #3449: [twitter] force HTTPS for TwitPic URLs 2023-01-05 14:57:03 +01:00
Mike Fährmann
e18482e9ae [twitter] improve 'http' -> 'https' replacement 2023-01-05 14:55:55 +01:00
Mike Fährmann
4fd6da474f merge #3473: [twitter] fix crash when using 'expand' and 'syndication' 2023-01-05 14:19:47 +01:00
Mike Fährmann
a918ce29b5 run tests on ubuntu-20.04
and remove Python 3.4, since that's no longer available
on this test runner
2023-01-05 13:33:27 +01:00
Mike Fährmann
6514828d4e emit debug logging message when loading cookies from file 2023-01-05 12:40:22 +01:00
Mike Fährmann
3a238fd490 [poipiku] warn about login requirements 2023-01-05 12:29:31 +01:00
Mike Fährmann
f29ba089ff merge #3474: [fanleaks] add 'post' and 'model' extractors 2023-01-04 17:39:17 +01:00
Mike Fährmann
6933727b45 merge #3483: [twitter] implement 'syndication=extended' 2023-01-04 17:36:17 +01:00
Mike Fährmann
07ed3a1fbf merge #3460: [poipiku] fix extraction for a different warning button style
(#3493, #3492)
2023-01-04 17:29:27 +01:00
Mike Fährmann
9116398c1c [pinterest] add 'domain' option (#3484)
use input URL domain by default
2023-01-04 17:20:14 +01:00
blankie
2f985bcddb [poipiku] fix extraction for a different warning button style 2023-01-04 08:28:31 +07:00
Mike Fährmann
294108c90a [pinterest] support 'All Pins' boards (#2855, #3484) 2023-01-03 19:11:20 +01:00
Mike Fährmann
77df8d3116 [deviantart] implement username&password login for scraps (#1029)
re-login when getting prematurely logged out by dA
is missing at the moment
2023-01-03 18:42:28 +01:00
Mike Fährmann
ed2d715019 fix 'keywords' in extractor tests (#3491) 2023-01-03 15:14:23 +01:00
ClosedPort22
6853b14be3 [twitter] apply suggestions from code review
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2023-01-02 21:03:01 +08:00
Mike Fährmann
4611237f8c merge #3457: [danbooru] extract uploader metadata (if option is set) 2023-01-01 21:20:39 +01:00
Mike Fährmann
e7522482bb merge #3463: [lynxchan] support 'bbw-chan.nl' 2023-01-01 15:50:09 +01:00
Mike Fährmann
7d6c846176 [fanbox] return 'imageMap' files in order (#2718) 2023-01-01 15:12:05 +01:00
Mike Fährmann
dc8e7ff54e [bunkr] fix URLs returned by API (#3481) 2023-01-01 14:07:15 +01:00
enduser420
5fedef3a1a [fanleaks] update 'model' URL pattern 2022-12-30 19:29:55 +05:30
enduser420
5a740ef78b [fanleaks] add 'post' and 'model' extractors 2022-12-30 19:24:05 +05:30
ClosedPort22
7c8eab8d52 [twitter] implement 'syndication=extended'
to be able to fetch extended user metadata
2022-12-30 20:48:41 +08:00
ClosedPort22
be3286206a [twitter] assume 'conversation_id' when using syndication
not possible to expand replies at the momemt
2022-12-30 13:57:37 +08:00
ClosedPort22
ce8dbb1ccc [twitter] fix crash when using 'expand' and 'syndication'
caused by KeyError: 'conversation_id_str'
2022-12-30 12:45:44 +08:00
ClosedPort22
38786a9593 [twitter] refactor extraction of TwitPic URLs
flattening
2022-12-27 12:23:12 +08:00
enduser420
527bb2c4ab [lynxchan/bbw-chan] add 'thread' and 'board' extractors 2022-12-26 20:59:42 +05:30
blankie
f82ee93676 [danbooru] extract uploader metadata (if metadata is set) 2022-12-25 11:00:59 +07:00
ClosedPort22
250d35107c [twitter] prioritize tweet type checks (#3439)
Do not consider a tweet seen before applying 'retweet', 'quote' and
'reply' checks. Otherwise the original tweets will also be skipped if
the "derivative" tweets and the original tweets are from the same user.
2022-12-24 12:50:52 +08:00
ClosedPort22
3eb352fcb0 [twitter] force HTTPS for TwitPic URLs 2022-12-23 18:16:34 +08:00
Mike Fährmann
bee354c264 Merge pull request #3415 from enduser420/extractor/fapello
[fapello] add 'post', 'user' and 'path' extractors
2022-12-17 20:05:46 +01:00
Mike Fährmann
8d7585534e Merge pull request #3367 from the-blank-x/deviantart-view
[deviantart] add /view URL support
2022-12-17 20:05:35 +01:00
blankie
6614d94b08 [deviantart] add /view URL support 2022-12-17 23:27:55 +07:00
Mike Fährmann
dd6eeb4336 Merge pull request #3366 from ClosedPort22/da-extra-stash
[deviantart] extract sta.sh URLs from `text_content`
2022-12-17 15:42:12 +01:00
Mike Fährmann
f36cbb3911 Merge pull request #3413 from ClosedPort22/e621-manual-pagination
[e621] implement manual pagination
2022-12-17 15:42:00 +01:00