Commit Graph

637 Commits

Author SHA1 Message Date
Mike Fährmann
5d69e437d0 [twitter] add option to download all media from a conversation
(fixes #1319)
2021-02-26 13:50:46 +01:00
Mike Fährmann
cf5fa75d4c add 'browser' option (#1117)
- change default user agent to Firefox ESR 78 on Windows 10
- remove 'ciphers' option
2021-02-26 13:41:27 +01:00
Mike Fährmann
e5735361ed [exhentai] add 'metadata' option (#1325)
to select between gallery metadata from 'api' or 'html'
2021-02-23 00:23:42 +01:00
Mike Fährmann
5ad2b9c82b [deviantart] extend 'extra' option
also download from embedded DeviantArt posts
2021-02-21 19:27:04 +01:00
Mike Fährmann
560277394e [downloader:http] add 'headers' option (#1322) 2021-02-21 19:13:39 +01:00
Mike Fährmann
6b0ecbf6bc [hentainexus] add 'orignal' option (#1322) 2021-02-21 19:06:12 +01:00
Mike Fährmann
8974f0361c [pixiv] update (#1304)
- remove login with username & password
- require a refresh token
- add 'oauth:pixiv' functionality

See also:
- https://github.com/upbit/pixivpy/issues/158
- https://gist.github.com/ZipFile/c9ebedb224406f4f11845ab700124362
2021-02-12 18:07:16 +01:00
Mike Fährmann
cdb0b02e30 [pillowfort] add 'reblogs' option (#846) 2021-01-25 15:03:59 +01:00
Mike Fährmann
7a096c443f [unsplash] add 'format' option (#1197) 2021-01-22 17:23:46 +01:00
Mike Fährmann
89a2bcbb2d [furaffinity] add 'descriptions' option (#1231) 2021-01-19 19:09:29 +01:00
Mike Fährmann
36f281330a [newgrounds] fix flash file extraction (closes #1257)
… and add a 'flash' option to choose between flash and video formats.
2021-01-19 17:48:14 +01:00
Mike Fährmann
c008cb5100 [pixiv] add 'related' option (#1237) 2021-01-17 22:48:32 +01:00
Mike Fährmann
01b9ccd4de [derpibooru] use the "Everything" filter by default (#1243)
when neither 'api-key' nor 'filter' are set
2021-01-14 18:29:09 +01:00
Mike Fährmann
5d4494b15f add "ascii" as a special 'path-restrict' value 2021-01-09 02:41:20 +01:00
Mike Fährmann
fa8ee6eac4 [derpibooru] add search and gallery extractors (#862) 2021-01-07 18:05:32 +01:00
Mike Fährmann
371e9ca6df [pinterest] implement video support (closes #1189) 2020-12-21 16:09:06 +01:00
Mike Fährmann
bf629a2818 [instagram] add 'include' option (closes #1180)
Split the functionality of the old 'user' extractor into separate
'posts' and 'highlights' extractors, which respond to virtual URLs
('/<user>/posts' and '/<user>/highlights')
2020-12-21 02:06:28 +01:00
Mike Fährmann
465015f75a [sankaku] reimplement login support (#1176, #1182) 2020-12-17 16:12:59 +01:00
Mike Fährmann
b2c55f0a72 [sankaku] remove login support
The old login method for 'https://chan.sankakucomplex.com/user/login'
and the cookies it produces have no effect on the results from
'beta.sankakucomplex.com'.
2020-12-08 21:05:47 +01:00
Mike Fährmann
0bc492c0fa add docs for 'event' and 'filename' options
from 9c3568c3 and ca59bd69
2020-11-25 12:12:41 +01:00
Mike Fährmann
9b1bd09454 change 'extension-map' default
Replace all JPEG filename extensions with 'jpg'.
2020-11-14 22:40:31 +01:00
Mike Fährmann
bc7b1d91bc fix rST markup in configuration.rst
[ci skip]
2020-11-02 15:32:29 +01:00
Mike Fährmann
e3480bc8de implement 'extension-map' option (#318) 2020-11-02 15:27:07 +01:00
Mike Fährmann
98a4d86a01 [sankakucomplex] extract videos and embeds (closes #308) 2020-10-30 01:21:11 +01:00
Mike Fährmann
d83b95fd28 [postprocessor:metadata] accept a string-list for 'content-format'
(closes #1080)
2020-10-27 20:09:58 +01:00
Mike Fährmann
5bcf28de93 add a 'extractor.modules' option 2020-10-25 03:05:10 +01:00
Mike Fährmann
1686dc1757 [twitter] support media from Cards (#1005, #937)
Can be enabled with 'extractor.twitter.cards', but for now disabled by
default because cards can redirect to rather large videos from YouTube
or Twitch.
2020-10-22 21:33:53 +02:00
Mike Fährmann
bac8af69e8 update configuration.rst
- add some lines to better explain post processor usage
- syntax highlighting for JSON blocks and other smaller stuff
2020-10-19 21:57:26 +02:00
Mike Fährmann
05d7009cc6 rename 'Authentication' entries in supportedsites.rst
- change 'Optional' to 'Supported'
- use 'OAuth' and 'Cookies' in their own
- add link to weasyl API key option
2020-10-19 20:16:17 +02:00
Mike Fährmann
d1c6d78477 fix rST markup in configuration.rst 2020-10-15 15:17:19 +02:00
Mike Fährmann
b8daabc3ca [pinterest] implement login support (closes #1055)
being logged allows access to secret/protected boards
2020-10-15 15:14:18 +02:00
Zanny
3ec60e894a [weasyl] api-key authentication (#1057)
* [weasyl] support api keys

* [weasyl] document api-key authentication

* [weasyl] usernames can contain ~
2020-10-15 15:12:09 +02:00
Mike Fährmann
35056a07d1 release version 1.15.1 2020-10-11 18:44:46 +02:00
Mike Fährmann
c874071f5a [kissmanga] remove module 2020-10-04 22:46:41 +02:00
Mike Fährmann
430b6d6e2e [twitter] extend 'retweets' option (closes #1026)
Setting 'retweets' to '"original"' will use metadata from the
original retweeted Tweets, and not from the Retweet entry.
2020-09-28 23:03:35 +02:00
Mike Fährmann
b9bdd2c564 [hentaifoundry] add support for stories (closes #734) 2020-09-27 02:27:40 +02:00
Mike Fährmann
117582d798 use definition lists instead of tables in docs/configuration.rst
and strip some clutter from generated man pages
2020-09-26 13:33:46 +02:00
Mike Fährmann
6514312126 [nijie] add 'include' option (closes #1018) 2020-09-25 18:18:35 +02:00
Mike Fährmann
0d43456323 [hentaifoundry] add 'include' option 2020-09-25 18:18:03 +02:00
Mike Fährmann
1e313d5b84 implement 'sleep-request' option 2020-09-20 20:28:17 +02:00
Mike Fährmann
6a5f6396d4 add clearing cache to dA client-id instructions (#996) 2020-09-16 01:24:18 +02:00
Mike Fährmann
3afd362e2e add 'sleep-extractor' option (closes #964)
(would have been nice if this were possible without code duplication)
2020-09-12 21:04:47 +02:00
Mike Fährmann
3918b69677 remove 'extractor.blacklist' context manager 2020-09-11 13:17:35 +02:00
Mike Fährmann
c78aa17506 add general 'blacklist' and 'whitelist' options (#492, #844) 2020-09-11 13:17:12 +02:00
Mike Fährmann
5912727b88 support format string replacement fields in archive paths
(closes #985)
2020-09-10 22:09:30 +02:00
Mike Fährmann
f6fd449b59 reduce wait time growth rate from exponential to linear
Waiting for 2**N seconds after each error grows too fast.
Simply waiting N seconds seems far more reasonable.
2020-09-06 22:38:25 +02:00
Mike Fährmann
ac3036ef56 add 'filesize-min' and 'filesize-max' options (closes #780) 2020-09-03 18:21:04 +02:00
ArtaxIsSleeping
0e941553ec [aryion] Add username/password support (#960)
* Add username/password support to aryion extractor

* Update docs to match

* Fix code style
2020-08-27 22:45:30 +02:00
Mike Fährmann
dbbbb21180 [exhentai] add ability to specify custom image limit (#940) 2020-08-17 22:29:20 +02:00
Mike Fährmann
901a897e6f [inkbunny] mention login support 2020-07-26 23:52:52 +02:00