github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,798 Commits 1 Branch 0 Tags
e2d4ca495515c1943cb3eb9dd329fc0ab346e441
Commit Graph

2018 Commits

Author SHA1 Message Date
Mike Fährmann
7e0d9257a7 [hbrowse] fix manga extraction 2017-11-15 13:59:50 +01:00
Mike Fährmann
3c576d10c0 [seiga] better metadata + 'skip()' support 2017-11-15 13:58:35 +01:00
Mike Fährmann
f72318e593 [seiga] support more than 200 images 
Due to API restrictions and/or missing knowledge about and
documentation of API usage, it was only possible to retrieve the
latest 200 images of a niconico seiga user with said API.

The new approach manually visits each HTML page and gets its
information from there.
 2017-11-13 20:46:24 +01:00
Mike Fährmann
baf8094868 improve Extractor.request()'s retry behavior 2017-11-13 20:37:11 +01:00
Mike Fährmann
7e7b64162b [batoto] handle error 10031 2017-11-12 20:49:37 +01:00
Mike Fährmann
92027f67f9 use consistent names for URL constants 
root := <scheme>://<host>
base_url := <root>/<common path>
 2017-11-06 20:56:49 +01:00
Mike Fährmann
69cbc0619f [mangastream] fix 'next-page' URLs (fixes #49) 2017-11-04 11:50:40 +01:00
Mike Fährmann
980fd3616d [tumblr] use API v2 (#48) 2017-11-03 22:16:57 +01:00
Mike Fährmann
d6bed9f36f [tumblr] prevent premature exit to get all images (fixes #48) 2017-11-03 14:59:31 +01:00
Mike Fährmann
305da540c3 [mangahere] fix metadata extraction 2017-11-03 14:54:46 +01:00
Mike Fährmann
2d0cfb33e1 [xvideos] add user profile extractor (#45) 2017-11-02 17:28:35 +01:00
Mike Fährmann
a393e6e538 [xvideos] add gallery extractor (#45) 2017-11-02 15:36:53 +01:00
Mike Fährmann
3a8a0c1f35 [imgbox] rewrite / fix extraction (closes #47) 2017-11-01 13:01:59 +01:00
Mike Fährmann
035ef655f1 [imagefap] update unit tests 
old gallery/image has been deleted
 2017-10-27 12:22:16 +02:00
Mike Fährmann
239d7afea7 [hosturimage] fix extraction of larger images 2017-10-25 12:56:16 +02:00
Mike Fährmann
158e60ee89 [3dbooru] enable download continuation 
behoimi.org doesn't respect 'Range' headers and doesn't report
'Content-Length' for compressed content encodings.
 2017-10-24 13:05:31 +02:00
Mike Fährmann
c4fcdf2691 Revert "[senmanga] fix extraction and download" 
This reverts commit 2ace5c7b3c.
 2017-10-24 00:22:05 +02:00
Mike Fährmann
81a7788b40 replace space characters in unit test URLs 2017-10-23 17:00:53 +02:00
Mike Fährmann
bf82181359 [jaiminisbox] fix extraction 2017-10-22 13:26:09 +02:00
Mike Fährmann
16783e327f [common] fix UnboundLocalError in Extractor.request() 2017-10-20 18:51:06 +02:00
Mike Fährmann
2ace5c7b3c [senmanga] fix extraction and download 2017-10-19 18:25:31 +02:00
Mike Fährmann
4d8387f93b [pixiv] support mobile URLs (https://touch.pixiv.net/) 2017-10-17 16:49:42 +02:00
Mike Fährmann
ab2bf0b0dd [deviantart] replace collection unittest 2017-10-17 15:58:16 +02:00
Mike Fährmann
289d6b65d2 [danbooru] extend and improve URL regex 
- add support for danbooru mirrors:
  - hijiribe.donmai.us
  - sonohara.donmai.us
  - todo: actually use these domains instead of redirecting everything
          to danbooru itself
- improve handling of query string parameters
 2017-10-16 21:21:19 +02:00
Mike Fährmann
5fa42336a2 [sankaku] add warning for unauthenticated users 
also improve URL pattern and add missing options to default config file
 2017-10-16 21:21:08 +02:00
Mike Fährmann
6af921a952 [sankaku] rewrite/improve (fixes #44) 
- add wait-time between HTTP requests similar to exhentai
- add 'wait-min' and 'wait-max' options
- increase retry-count for HTTP requests to 10
- implement user authentication (non-authenticated users can only view
  images up to page 25)
- implement 'skip()' functionality (only works up to page 50)
- implement image-retrieval for pages >= 51
- fix issue with multiple tags
 2017-10-14 23:01:33 +02:00
Mike Fährmann
9aecc67841 [common] explicitly handle HTTP status code 429 2017-10-14 21:37:59 +02:00
Mike Fährmann
d68a24aa70 [kissmanga] fix extraction 
site changed '\n' to '\r\n' for newlines
 2017-10-12 14:49:27 +02:00
Mike Fährmann
864a63ed33 fix typo 
[skip ci]
 2017-10-10 17:42:06 +02:00
Mike Fährmann
f3fbaa5c3e [reddit] allow users to override the API User-Agent 
Only overriding the Client-ID is not enough if you want to follow
Reddit's API access rules [1].

[1] https://github.com/reddit/reddit/wiki/API#rules
 2017-10-10 17:29:46 +02:00
Mike Fährmann
31ea6001e8 [dynastyscans] improve metadata and filename formats 2017-10-10 17:14:39 +02:00
Mike Fährmann
2ef3c35c98 smaller textual changes 
- swapped doc for deviantart.mature and .original
- updated gallery-dl.conf
- "transferred" -> "delegated"
 2017-10-09 23:23:19 +02:00
Mike Fährmann
68a0a7579c fix/improve some regular expressions 2017-10-09 22:37:50 +02:00
Mike Fährmann
393755ee94 [tumblr] update tests 2017-10-09 00:10:37 +02:00
Mike Fährmann
75d3a1f72f [deviantart] always download original images 
Deviation-objects returned by the DeviantArt API don't always contain
the URL and metadata of the original image ([1]). Getting this
information requires an additional API call [2], which is indicated by
the 'is_downloadable' and 'download_filesize' metadata within a
deviation-object.

[1] https://myria-moon.deviantart.com/art/Aime-Moi-part-en-vadrouille-261986576
[2] https://www.deviantart.com/developers/http/v1/20160316/deviation_download/bed6982b88949bdb08b52cd6763fcafd
 2017-10-07 13:07:34 +02:00
Mike Fährmann
a1c8b21cfd [senmanga] improve metadata 2017-10-04 18:54:39 +02:00
Mike Fährmann
994b2fc1e7 [deviantart] replace 'author[urlname]' keyword 
author[urlname] has always only been the lowercase version of
author[username], which can now be directly converted to lowercase
using the 'l' conversion: '{author[username]!l}'
 2017-10-04 15:59:05 +02:00
Mike Fährmann
633b376f35 improve/adjust default filename formats for manga sites 2017-10-02 19:06:24 +02:00
Mike Fährmann
41adb99e9c [pawoo] fix extraction 
- changed access_token
- use account-search instead of general search
 2017-10-02 18:33:52 +02:00
Mike Fährmann
b319f4bab3 smaller code and text changes 2017-10-01 18:23:40 +02:00
Mike Fährmann
ad4580800c [pixiv] add support for more URL patterns 
- https://www.pixiv.net/mypage.php#id=USERID
- https://www.pixiv.net/#id=USERID
 2017-09-30 18:07:20 +02:00
Mike Fährmann
82ea6c0cd3 adjust format strings with optional titles 
... except for anything manga/comic related
 2017-09-28 18:00:19 +02:00
Mike Fährmann
85a2b2ae59 [khinsider] fix extraction 2017-09-28 11:47:26 +02:00
Mike Fährmann
26a866e7d8 implement (sub)category-transfer between extractors (#41) 
ImageFap- and all Manga-Extractors will transfer their (sub)category
values to other extractors instantiated by them, which will in turn
allow those to use options set for their parents.

Example:
ImagefapGalleryExtractors will use options set under
extractor.imagefap.user, if (and only if) they have been instantiated by
a ImagefapUserExtractor; and options from extractor.imagefap.gallery
otherwise.
 2017-09-26 21:05:11 +02:00
Mike Fährmann
1ab4c7986f [mangahere] fix extraction 
would switch to HTTPS, but there seem to be certificate issues
 2017-09-26 21:05:11 +02:00
Mike Fährmann
8e14714c2b [imgspice] fix extraction 2017-09-26 21:04:48 +02:00
Mike Fährmann
9c138dfc1f [common] detect empty HTTP response bodies 2017-09-26 16:49:58 +02:00
Mike Fährmann
c51616f8d8 [foolslide] fix minor chapter number 2017-09-26 12:49:50 +02:00
H R X N
77bf923c56 Update imgur.py to include 'title' of single image (#40) 
Add {title} keyword..
Images on Imgur don't necessarily have a title, but I think most of them do, and since this should not break anything else..
 2017-09-26 12:48:48 +02:00
Mike Fährmann
a85f06d2d1 [foolslide] restructure; convert suitable values to int 2017-09-24 16:57:47 +02:00