Commit Graph

195 Commits

Author SHA1 Message Date
Mike Fährmann
19860655a3 [weibo] add 'user' and 'status' extractors 2019-02-17 18:18:31 +01:00
Mike Fährmann
ae353ed3b0 provide "extractor" and "job" keys for logging output
This allows for stuff like "{extractor.url}" and "{extractor.category}"
in logging format strings.
Accessing 'extractor' and 'job' in any way will return "None" if those
fields aren't defined, i.e. in general logging messages.
2019-02-14 11:09:58 +01:00
Mike Fährmann
53c2fd4664 add mastodon/foolslide/foolfuuka examples to example config 2019-02-05 16:17:25 +01:00
Mike Fährmann
d36ec51e5a [tsumino] add extractor for search results (#161) 2019-02-02 14:56:46 +01:00
Mike Fährmann
bfbbac4495 [tsumino] add login capabilities (#161) 2019-01-30 17:58:48 +01:00
Mike Fährmann
6126615698 update URLs for supportedsites.rst 2019-01-30 16:18:22 +01:00
Mike Fährmann
80a75a1ecf [tsumino] add gallery extractor (#161) 2019-01-29 17:28:48 +01:00
Mike Fährmann
0c32dc5858 [hentaifox] add extractor for search results (#160) 2019-01-28 22:38:32 +01:00
Mike Fährmann
0156189468 [hentaifox] add chapter extractor (#160) 2019-01-28 18:00:32 +01:00
Mike Fährmann
e4171d6baf [luscious] add login capabilities (closes #159) 2019-01-28 17:14:15 +01:00
Mike Fährmann
134487ffb0 [exhentai] stop extraction if image limit is exceeded (#141)
can be turned off with the `exhentai.limits' option
2019-01-26 18:40:39 +01:00
Mike Fährmann
b50b30f1c9 [photobucket] download subalbums (#117) 2019-01-22 14:05:18 +01:00
Mike Fährmann
277b52101a add 'category-transfer' option
[ci skip]
2019-01-19 20:28:19 +01:00
Mike Fährmann
4b441c162e release version 1.6.3 2019-01-18 12:31:04 +01:00
Mike Fährmann
8aba2bdebf [postprocessor:metadata] add 'tags' and 'custom' modes (#135) 2019-01-17 21:18:12 +01:00
Mike Fährmann
e53cdfd6a8 update build_supportedsites.py 2019-01-09 14:58:35 +01:00
Mike Fährmann
1e4d351ad3 [danbooru] add authentication support (closes #151)
... via HTTP Basic Auth with username and "password".

The password value in this case is not the account password itself,
but the"api_key" found in your user profile.
2019-01-09 14:19:07 +01:00
Mike Fährmann
06cbf5f9c4 implement 'chapter-reverse' option (#149)
Setting it to `true` will start with the latest chapter instead of the
first one.
2019-01-07 18:22:33 +01:00
Mike Fährmann
e95b24f056 [reactor] add wait-min & -max options (#148) 2019-01-07 18:04:47 +01:00
Mike Fährmann
a36f52a730 [joyreactor] add extractor for search results (#148) 2019-01-03 16:25:56 +01:00
Mike Fährmann
6c71e9cf5d [deviantart] add separate 'sta.sh' extractor (#113)
- supports multiple stashed deviations per page
- explicitly mentions sta.sh support on supportedsites.rst
2018-12-26 18:56:57 +01:00
Mike Fährmann
28f9539551 [tumblr] change default values for post types and inline media 2018-12-26 18:55:59 +01:00
Mike Fährmann
5be95034ba [tumblr] add option to download avatars (#137) 2018-12-26 14:29:30 +01:00
Mike Fährmann
be52069cbc update CHANGELOG and docs/supportedsites 2018-11-28 14:53:27 +01:00
Mike Fährmann
c47482b110 smaller changes, missing docs, etc.
- make 'netrc' extractor-specific
- rename 'downloader.enable' to 'enabled'
- document 'downloader.ytdl.format'
- consistent newlines in configuration.rst
2018-11-16 18:18:07 +01:00
Mike Fährmann
8ef84a6823 add option to enable/disable specific downloader modules
... and write URLs with no (active) downloader to unsupported-file
2018-11-13 18:06:36 +01:00
Mike Fährmann
2fa28a2609 update default user-agent string (closes #122) 2018-11-11 10:07:10 +01:00
Mike Fährmann
655549df7c [downloader:ytdl] add several options
The "default" downloader options (rate, retries, timeout, verify) are
mapped to corresponding youtube-dl options.

downloader.ytdl.logging tells the downloader to pass youtube-dl's output
to a Logger object.

downloader.ytdl.raw-options allows to pass arbitrary options to the
YoutubeDL constructor.
2018-10-20 18:26:49 +02:00
Mike Fährmann
6ed629f2b6 allow specifying number of skips before abort/exit (closes #115)
In addition to 'abort' and 'exit', it is now possible to specify
'abort:N' and 'exit:N' (where N is any integer) as value for 'skip'
to abort/exit after consecutively skipping N downloads.
2018-10-13 17:21:55 +02:00
Mike Fährmann
d8492df51b [deviantart] extend functionality of 'original' option 2018-10-11 22:11:49 +02:00
Mike Fährmann
392a081657 update 'docs/configuration.rst' 2018-10-08 23:09:01 +02:00
Mike Fährmann
f8b3b00249 [twitter] add experimental 'videos' option (#99)
Enabling this option will detect videos in tweets and output them as
"unsupported" URLs, so that these can then be downloaded with youtube-dl

There are a lot of improvements to be made to the current
implementation, but it works and does what it is supposed to, even if
inefficient as can be ...
2018-09-30 21:52:23 +02:00
Mike Fährmann
2cf3f53839 [yuki] add thread extractor (closes #111) 2018-09-28 12:47:32 +02:00
Mike Fährmann
17611bfec0 update build_supportedsites.py script 2018-09-28 12:43:19 +02:00
Mike Fährmann
09d2f3e5e7 [postprocessor:ugoira] improve libx264 detection 2018-09-24 13:12:54 +02:00
Mike Fährmann
c402cc4047 [hentaifoundry] add 'popular' and 'recent' extractors
for "Popular Pictures" and "Recent Pictures" listings
2018-09-24 13:11:18 +02:00
Mike Fährmann
a5fc311dfa [hentaifoundry] add 'favorite' extractor 2018-09-22 21:23:29 +02:00
Mike Fährmann
55f5c87160 [postprocessor:ugoira] add 'libx264-prevent-odd' option
A rather crude workaround for "width/height not divisible by 2"
errors when using libx264.
2018-09-21 20:01:14 +02:00
Mike Fährmann
006f75b538 [hentaifoundry] rewrite + more metadata
- extract width, height, artist per image
- improve pattern regex
- better extensibility for other listings
2018-09-21 11:23:51 +02:00
Mike Fährmann
6ea9a78588 [wallhaven] add login capabilities
Being logged in is required to access NSFW wallpapers.
2018-09-19 21:04:01 +02:00
Mike Fährmann
c9290d8212 [wallhaven] add wallpaper and search extractors
todo:
- login support to gain access to NSFW wallpapers
- extractors for tag-, similar-, latest-listings
- skip() support
2018-09-17 21:26:13 +02:00
Mike Fährmann
22ab509a70 [bobx] rename "model" to "idol" extractor 2018-09-14 18:11:36 +02:00
Mike Fährmann
69fd61ea86 [bobx] add gallery and model extractors 2018-09-13 20:13:12 +02:00
Mike Fährmann
7742cf8601 [tumblr] change 'reblogs' option (#103)
- rename "deleted" to "same-blog"
- change test for deleted original post to test if
  original post owner has the same UUID (full blog name) as the one
  being downloaded from
- add 'blog[uuid]' metadata to allow comparison with
  'reblogged_from_uuid'
2018-09-10 15:40:25 +02:00
Mike Fährmann
f1695567e8 adjust values in template config file
(#104)
2018-09-09 14:10:55 +02:00
Mike Fährmann
a666ddd16b [tumblr] extend 'reblogs' functionality (#103)
Setting 'reblogs' to "deleted" will check if the parent post of a
reblog has been deleted and download its media content if that is the
case, otherwise it will be skipped.

This is a rather costly operation (1 API request per reblogged post)
and should therefore be used with care.
2018-09-07 19:13:52 +02:00
Mike Fährmann
c9b8e6aefc [reddit] fix submission-ID parsing (#104)
Uppercase characters caused a ValueError exception
2018-09-07 18:27:54 +02:00
Mike Fährmann
3ecea4cf36 [hentaicafe] add chapter and manga extractors (#101) 2018-09-05 21:08:40 +02:00
Mike Fährmann
75862715ac [behance] add user extractor 2018-08-31 17:42:09 +02:00
Mike Fährmann
6ecb36d88c [postprocessor:ugoira] add 'ffmpeg-output' option 2018-08-31 17:37:35 +02:00