Commit Graph

395 Commits

Author SHA1 Message Date
Mike Fährmann
15e4ddf46d implement custom logging formatter
supports custom log message formats for each loglevel and, by
extension, custom ANSI codes and colors for errors and warnings

(#304)
2019-06-21 20:17:58 +02:00
Mike Fährmann
db3f52881a add 'mtime' option 2019-06-20 17:19:44 +02:00
Mike Fährmann
ec3e8601f1 [slickpic] add user extractor (#249) 2019-06-14 18:55:56 +02:00
Mike Fährmann
d997c10320 [8muses] add album extractor (#305) 2019-06-10 22:17:46 +02:00
Mike Fährmann
e05a96db5e [deviantart] rename 'stash' to 'extra' (#302)
'stash' is already used as a name for the StashExtractor and therefore
expected to be a dictionary.
2019-06-10 21:05:25 +02:00
Mike Fährmann
2184e3a86b [slickpic] add album extractor (#249) 2019-06-09 21:59:22 +02:00
Mike Fährmann
c23bf263fe [deviantart] rename 'external' to 'stash' (#302)
restrict extracted URLs to ones from https://sta.sh/...
2019-06-09 11:16:02 +02:00
Mike Fährmann
c73c2cda50 [pornhub] add gallery & user extractor (#282) 2019-06-07 16:31:20 +02:00
Mike Fährmann
035b850e82 update postprocessor entries in example config
- use whitelists
- add ugoira example (#299)
2019-06-07 13:47:02 +02:00
Mike Fährmann
2fb85178da [deviantart] add 'external' option (#302)
If a description is available, this will extract URLs from the
description text and try to find Extractors for them.
2019-06-06 18:53:50 +02:00
Mike Fährmann
096009367b [xhamster] add gallery & user extractor (#281) 2019-06-05 11:11:51 +02:00
Mike Fährmann
d33f5a7423 [wallhaven] rewrite
- use API
- remove login support, add 'api-key' option
- remove support for "alpha" subdomain - alpha.wallhaven.cc used numeric
  IDs that can't be translated to the new ID system
- support direct links to wallpapers
2019-05-31 14:53:02 +02:00
Mike Fährmann
f1893b2b5b [deviantart] add 'folders' option (#276) 2019-05-30 17:28:12 +02:00
Mike Fährmann
c849574def [keenspot] add comic extractor (#223)
Doesn't work for
- http://brawlinthefamily.keenspot.com/
- http://flipside.keenspot.com/
- http://lastblood.keenspot.com/
- http://mysticrevolution.keenspot.com/
- http://porcelain.keenspot.com/
- http://twokinds.keenspot.com/
yet, because of custom layouts.
2019-05-28 21:34:38 +02:00
Mike Fährmann
bcd1801aa8 [sankakucomplex] add 'tag' extractor (#258) 2019-05-27 23:57:44 +02:00
Mike Fährmann
74c2415138 [sankakucomplex] move article extractor to its own module (#258) 2019-05-27 23:49:23 +02:00
Mike Fährmann
4465a3ea68 [kissmanga][readcomiconline] add 'captcha' option (#279)
to configure how to handle CAPTCHA page redirects:
- either interactively wait for the user to solve the CAPTCHA
- or raise StopExtraction like before
2019-05-27 22:24:48 +02:00
Mike Fährmann
1e3e15c4f3 [sankaku] add article extractor (#258) 2019-05-26 17:42:36 +02:00
Mike Fährmann
e82cadac61 [patreon] add extractors (#226) 2019-05-17 10:47:22 +02:00
Mike Fährmann
4891f4a328 [hentainexus] add search extractor (#256) 2019-05-16 23:55:30 +02:00
Mike Fährmann
9890bfdf23 [flickr] improve code and metadata
- simplify pagination
- add more metadata and slightly change its structure
  - convert suitable values to int or list
  - move keys from ["photo"] to the base level
- proper video support (#246)
- rename method and variable names to better fit with other extractors
2019-05-14 22:10:50 +02:00
Mike Fährmann
ba8eb1ffec [hentainexus] add gallery extractor (#256) 2019-05-12 23:59:41 +02:00
Mike Fährmann
0318c610dc [sexcom] add extractor for search results (#147) 2019-04-24 22:10:01 +02:00
Mike Fährmann
a247c94c34 [sexcom] add pin and board extractors (#147) 2019-04-24 22:09:19 +02:00
Mike Fährmann
70be494161 [plurk] add a 'comments' options (#212) 2019-04-14 22:12:46 +02:00
Mike Fährmann
0b2ff406f6 [plurk] add timeline- and post-extractors (#212) 2019-04-14 21:48:38 +02:00
Mike Fährmann
e7cd5510d5 [pixnet] add extractors (closes #177)
for:
- users/blogs: http://albertayu773.pixnet.net/
- folders: https://albertayu773.pixnet.net/album/folder/1405768
- sets   : https://albertayu773.pixnet.net/album/set/15078995
- photos : https://albertayu773.pixnet.net/album/photo/159443828
2019-04-11 19:27:02 +02:00
Mike Fährmann
d9b94a585d [mangoxo] add login support (#184)
A very recent change: It is now only possible to see more
than the first 5 images of an album if you are logged in.
2019-04-10 18:55:25 +02:00
Mike Fährmann
e730fc9045 [twitter] add login support (#214) 2019-04-09 09:27:49 +02:00
Mike Fährmann
35919a9bb8 [livedoor] add blog- and post-extractors (#190) 2019-04-06 16:27:48 +02:00
Mike Fährmann
2ff043edfa [yaplog] add user- and post-extractors (#190) 2019-04-04 17:56:56 +02:00
Mike Fährmann
790f15a56f [photobucket] use HTTPS 2019-04-03 18:30:45 +02:00
Mike Fährmann
6da665f32e [mangoxo] add album- and channel-extractors (closes #184) 2019-04-03 07:55:51 +02:00
Mike Fährmann
c70b21248d [wikiart] add extractors (#179)
for
- artists:          https://www.wikiart.org/en/thomas-cole
- artist-listings:  https://www.wikiart.org/en/artists-by-century/12
- artwork-listings: https://www.wikiart.org/en/paintings-by-media/grisaille
2019-04-02 17:34:57 +02:00
Mike Fährmann
bc2020e86c release version 1.8.1 2019-03-29 17:37:11 +01:00
Mike Fährmann
369eb66125 consistently use '*' for rst lists 2019-03-28 16:21:41 +01:00
Mike Fährmann
c7a6b0ed90 [deviantart] add 'metadata' option (#189) 2019-03-21 14:49:42 +01:00
Mike Fährmann
8d96a8ce4c [500px] add user-, gallery-, and image-extractors (#185) 2019-03-20 17:32:36 +01:00
Mike Fährmann
6277a739e4 [35photo] add user-, genre-, and image-extractors (#162) 2019-03-18 01:11:30 +01:00
Mike Fährmann
12482553bd update links to youtube-dl 2019-03-13 22:03:02 +01:00
Mike Fährmann
6f57d44ec2 [seaotterscans] remove extractor
http://seaotterscans.com/ now redirects to their MangaDex profile
2019-03-13 22:02:45 +01:00
Mike Fährmann
25aaf55514 [smugmug] improve format selection (closes #183)
- use original image if available
- support video formats
- remove user info for ImageExtractor (it is no longer possible to get
  image owner information for a single image)
2019-03-10 15:20:35 +01:00
Mike Fährmann
976ccb267f [myportfolio] combine gallery and user extractors
An URL alone isn't good enough to distinguish between a gallery or a
gallery-listing, so the new extractor decides what to do based on the
page's content.
2019-03-06 19:45:01 +01:00
Mike Fährmann
9c0e2f294b [shopify] add generic collection and product extractors (#175)
with fashionnova.com  as a default domain
2019-03-05 22:33:37 +01:00
Mike Fährmann
13e0f2a78f [deviantart] add 'scraps' extractor (closes #168) 2019-03-01 14:13:34 +01:00
Mike Fährmann
280531c8ff [pururin] add gallery extractor (closes #174) 2019-02-25 14:54:57 +01:00
Mike Fährmann
3159dd79d5 [seiga] use HTTPS 2019-02-21 22:51:11 +01:00
Mike Fährmann
ea6fbfdd39 simplify build_supportedsites.py 2019-02-21 14:29:19 +01:00
Mike Fährmann
22d3a2fcc8 [artstation] add extractor for artwork listings (#80)
like https://www.artstation.com/artwork?sorting=latest
or   https://www.artstation.com/artwork?sorting=picks
2019-02-18 12:45:44 +01:00
Mike Fährmann
937a802b49 [dynastyscans] add extractors for images and image searches
(closes #163)
2019-02-18 12:25:52 +01:00