github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,075 Commits 1 Branch 0 Tags
85ed023c2e0b32d54a10223ca698ffbc5ee2cb23
Commit Graph

772 Commits

Author SHA1 Message Date
Mike Fährmann
85ed023c2e [mangadex] remove the trailing ' - MangaDex' in a better way 
str.rstrip() works differently than assumed.
 2018-03-10 15:54:50 +01:00
Mike Fährmann
32bbd12f08 update extractor tests 2018-03-08 18:04:34 +01:00
Mike Fährmann
ca326bd275 [deviantart] fix folder and collection archive IDs 
{folder[index]} and {collection[index]} are both '0' when being
delegated from Gallery- or FavoriteExtractors, as there is no
way of knowing a folder's index when getting folder-information
from the API.
 2018-03-08 14:32:23 +01:00
Mike Fährmann
e32fe1cdf1 [pinterest] cast IDs to int 
... and update test results.

Image URLs changed from
https://s-media-cache-ak0.pinimg.com/... to
https://i.pinimg.com/...
 2018-03-06 14:28:21 +01:00
Mike Fährmann
179ecee965 [turboimagehost] fix extraction 2018-03-06 14:25:10 +01:00
Mike Fährmann
1400868f53 [mangadex] general improvements 
- support >100 chapter entries per manga
- custom archive ID format
- detect non-existing chapters
 2018-03-06 14:15:15 +01:00
Mike Fährmann
749fbbfa6c [mangadex] add chapter- and manga-extractor 2018-03-05 18:37:21 +01:00
Mike Fährmann
6e38cf5aab [mangareader] use 'https://' 
The site now redirects from http://mangareader.net/
to https://mangareader.net/
 2018-03-02 17:19:17 +01:00
Mike Fährmann
1d71123f91 [pixiv] update archive IDs and add metadata-fields 
(Pixiv bookmarks actually have their own IDs, comments and tags,
independent of the bookmarked image, which makes creating an
archive ID a lot easier)
 2018-03-02 16:11:53 +01:00
Mike Fährmann
858fdbdb22 [tumblr] improve 'inline' extraction 
'quote' posts store their HTML content in the 'source' field
 2018-03-02 06:59:44 +01:00
Mike Fährmann
5008e105ee update archive IDs 
... to behave in a more straightforward way when dealing with
bookmarks/favourites/etc.

specific IDs are now grouped by their owner, album-id, ... to
allow for duplicates when it would be expected.
 2018-03-01 18:20:50 +01:00
Mike Fährmann
829ddf4ac1 [sankaku] general improvements 
- simplify regex
- unquote search tags
- increase default wait-time between HTTP requests
  - downloading several hundreds of images always resulted
    in '429 Too Many Requests' eventually
- circumvent paging restrictions for unauthenticated users by only
  using the 'next' parameter
  - setting 'page' to a constant, low value (or simply omitting it)
    does the trick
 2018-02-27 16:51:14 +01:00
Jad
49463f76bb support multi-page URL (#79) 
* support multi-page URL

* fix

* all done.

* fix, again
 2018-02-26 11:13:49 +01:00
Mike Fährmann
19aefdfde3 [directlink] update test results 2018-02-26 03:01:23 +01:00
Mike Fährmann
74029c50bb [directlink] unquote metadata fields 2018-02-26 02:12:47 +01:00
Mike Fährmann
8f338347b6 [imagehosts] cleanup 
removed
- chronos.to      - unable to resolve hostname
- coreimg.net     - same
- imgmaid.net     - same
- hosturimage.com - everything returns 404
- imageontime.org - redirects to some shady site
- imgupload.yt    - cloudflare error 522, host down
- img4ever.net    - read timeout
 2018-02-23 01:05:42 +01:00
Mike Fährmann
edfd3d9fc9 [yeet] remove module 
- archive.yeet.net returns a 500 server error
- yeet.net moved to yeet.rip, but the archive is gone
 2018-02-23 01:05:41 +01:00
Mike Fährmann
8704d850bf add explicit proxy support (#76) 
- '--proxy' as command-line argument
- 'extractor.*.proxy' as config option
 2018-02-19 18:45:06 +01:00
Mike Fährmann
367b963d37 [pixiv] fix ugoira extraction ... again (#78) 
Some animations are not available for mobile devices, so we
pretend to be a desktop browser when requesting the ugoira page.
 2018-02-19 16:50:12 +01:00
Mike Fährmann
b79f1f2ca7 [pixiv] fix ugoira extraction (closes #78) 2018-02-19 08:51:09 +01:00
Mike Fährmann
d122203be1 [mangastream] fix extraction 2018-02-17 22:40:16 +01:00
Mike Fährmann
179bcdd349 adjust archive-ids 2018-02-13 04:50:45 +01:00
Mike Fährmann
3cec533c28 Merge branch 'archive' 2018-02-12 18:07:58 +01:00
Mike Fährmann
20af86b2ea add more extractor tests 
for mangastream, reddit and imgur
 2018-02-12 17:07:18 +01:00
Mike Fährmann
7e0207bcf4 [imgur] strip trailing '?1' from 'ext' 2018-02-10 21:33:40 +01:00
Mike Fährmann
cf147dfee9 [hentai2read] fix manga extraction 
- site changed its HTML structure
 2018-02-09 22:24:34 +01:00
Mike Fährmann
f5f2d29f56 [nijie] fix dojin extraction 
- correctly extract artist_id
- set extension to "jpg" if it was empty and let filetype checks do
  the rest
 2018-02-09 22:06:26 +01:00
Mike Fährmann
d38bf2f54c [tumblr] recognize /image/... URLs 
xyz.tumblr.com/image/123 refers to the same images
as xyz.tumblr.com/post/123.
 2018-02-08 23:08:14 +01:00
Mike Fährmann
5b3c34aa96 use generic chapter-extractor in more modules 2018-02-07 12:36:39 +01:00
Mike Fährmann
7b5ba69951 [hentaihere] ensure consistent extraction results 
sometimes there is a random space before the next <a>
 2018-02-05 15:26:25 +01:00
Mike Fährmann
377b78b3c9 [hentai2read] fix manga name extraction 2018-02-04 22:12:24 +01:00
Mike Fährmann
54c36a8a34 [subapics] add chapter- and manga-extractor (#70) 2018-02-04 22:02:10 +01:00
Mike Fährmann
2dd3aeeeae [komikcast] add chapter- and manga-extractor (#70) 2018-02-04 22:02:10 +01:00
Mike Fährmann
7a412f5c32 implement generic manga-chapter extractor 2018-02-04 22:02:04 +01:00
Mike Fährmann
6a07e38366 implement extractor.add() and .add_module() 
... as a public and non-hacky way to add (external) extractors to
gallery-dl's pool and make them available for extractor.find()
 2018-02-02 00:01:41 +01:00
Mike Fährmann
34873dbd90 set 'archive_fmt' values 
These are going to be used to create an unique id for each image.
 2018-02-01 15:30:49 +01:00
Mike Fährmann
a34cebc253 [luscious] jump to first image if cover does not link to it 2018-01-30 22:39:01 +01:00
Mike Fährmann
84a52a9256 add DownloadArchive class 2018-01-30 15:23:23 +01:00
Mike Fährmann
619387cbb1 update extractor unittest results 2018-01-28 18:29:05 +01:00
Mike Fährmann
db91cf871c document message identifiers 2018-01-23 21:38:30 +01:00
Mike Fährmann
0dd48d644f update test results 
nothing broke, but things got updated or changed
 2018-01-23 21:38:29 +01:00
Mike Fährmann
1e93955170 [batoto] remove module 
Site officially shut down on 2018.01.18
 2018-01-23 21:37:32 +01:00
Mike Fährmann
76509a6d3c [imgur] update test results 2018-01-20 18:49:29 +01:00
Mike Fährmann
9fccd7b783 [tumblr] provide fallback URLs (#64) 
Each image now produces 3 URLs:
- amazonaws.com _raw (or _1280 for older images)
- amazonaws.com _500
- media.tumblr.com (URL returned by API)
 2018-01-19 23:12:15 +01:00
Mike Fährmann
9d69401391 initial support for multiple URLs per image 2018-01-17 22:08:19 +01:00
Mike Fährmann
91ed147cef [oauth] use custom key/secret values during oauth:… 2018-01-16 17:39:46 +01:00
Mike Fährmann
421a9740a3 [tumblr] add 'tumblr:' to force Tumblr extractor (#71) 2018-01-15 18:27:58 +01:00
Mike Fährmann
40d35c87bc [paheal] add tag- and post-extractors (closes #69) 2018-01-15 16:39:05 +01:00
Mike Fährmann
cc0c2cca57 [reddit] add extractor for reddit-hosted images (closes #68) 2018-01-14 18:55:42 +01:00
Mike Fährmann
f10ffc0839 update extractor blacklist to also allow classes 2018-01-14 18:47:22 +01:00