github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,326 Commits 1 Branch 0 Tags
17e24eacf00ea640b76c7d75b368dfcc1dbd1ca3
Commit Graph

115 Commits

Author SHA1 Message Date
Mike Fährmann
cd931e1139 update extractor test results 2022-12-08 18:58:29 +01:00
Mike Fährmann
202c1210d5 [exhentai] fix pagination 2022-12-06 22:39:13 +01:00
Mike Fährmann
20fbba9d7c [exhentai] add metadata to search results (#3181) 
'gallery_id' and 'gallery_token'
 2022-11-09 11:40:27 +01:00
Mike Fährmann
6a0c5e34f4 [exhentai] fix pagination (#3181) 2022-11-09 11:15:34 +01:00
Mike Fährmann
b0cb4a1b9c replace 'text.extract()' with 'text.extr()' where possible 2022-11-05 01:14:09 +01:00
Mike Fährmann
94b34f460e [exhentai] add slash to the end of gallery URLs (#2947) 2022-09-21 17:54:20 +02:00
Mike Fährmann
464ea90d14 [exhentai] guess extension for original files (#2842) 
makes it possible to sometimes, when guessed correctly ('.jpg'),
skip an original file download without costing image limit points
 2022-09-14 16:06:27 +02:00
Mike Fährmann
551fdf7ad7 [exhentai] move 509 check into its own function 2022-09-13 18:27:14 +02:00
Mike Fährmann
8b76149521 [exhentai] improve 509.gif detection (#2901) 2022-09-09 18:09:52 +02:00
Mike Fährmann
c6a9bab019 update extractor test results 2022-07-12 15:49:22 +02:00
Mike Fährmann
1f9a0e2fd8 update extractor test results 2022-04-18 17:24:00 +02:00
Mike Fährmann
dee0d22561 update extractor test results 2022-02-06 21:39:24 +01:00
Mike Fährmann
62692c6842 [exhentai] add 'source' option 
setting it to "hitomi" downloads the corresponding gallery from
hitomi.la; might be extended to other sources in the future
 2021-12-16 23:16:19 +01:00
Mike Fährmann
275543b2d2 update extractor test results 2021-11-27 19:26:44 +01:00
Mike Fährmann
f2ae179713 [exhentai] fix extraction for disowned galleries (closes #2055) 2021-11-24 21:26:16 +01:00
Mike Fährmann
57854624a1 [exhentai] improve image limits check (#1808) 
Check for a 'text/html' Content-Type instead of the very specific
137 bytes Content-Length, which might change depending on compression
or other factors.
 2021-08-29 21:19:55 +02:00
Mike Fährmann
c3b5c88b04 update extractor test results 2021-07-20 20:21:33 +02:00
Mike Fährmann
8fd8126117 fix ISO 639-1 code for Japanese 
"jp" -> "ja"
 2021-05-22 16:07:04 +02:00
Mike Fährmann
9514cb8c12 [exhentai] update 'limits' check (#1487) 
Only use 'limits' to set a custom upper bound.
Checking if the actual maximum gets exceeded is not necessary.
 2021-04-23 23:20:45 +02:00
Renan Vedovato Traba
9322c5e43b [exhentai] restore limit config (#1487) 
This partially reverts commit e9ec91c8
 2021-04-22 21:21:41 +02:00
Mike Fährmann
abafe71e04 [exhentai] fix image limit detection (closes #1437) 
check for image limit message when downloading original files
 2021-04-08 21:33:41 +02:00
Mike Fährmann
e9ec91c811 [exhentai] improve image limits check 
- check if current image is the '509 Bandwidth Exceeded' notification
  (https://ehgt.org/g/509.gif or https://exhentai.org/img/509.gif)
- remove 'limits' option
 2021-03-29 19:01:13 +02:00
Mike Fährmann
c7c3fef0bc [exhentai] support '/tag/' URLs (closes #1363) 2021-03-08 22:40:51 +01:00
Mike Fährmann
90830daf85 [exhentai] improve 'favorites' extraction (closes #1360) 
add special cases for when the favorite count is 0 (Never) or 1 (Once)
 2021-03-07 22:40:25 +01:00
Mike Fährmann
bae874f370 replace 'wait-min/-max' with 'sleep-request' 
on exhentai, idolcomplex, reactor
 2021-03-02 22:55:45 +01:00
Mike Fährmann
61fbbd2dae [exhentai] rename metadata fields to match API results (#1325) 
- gallery_id    -> gid
- gallery_token -> token
- title_jp      -> title_jpn
- visible       -> expunged
- gallery_size  -> filesize
- count         -> filecount

Also changes the function of the 'metadata' option.
It is now boolean and causes extra data fields from the API to be added
instead of completely replacing the data from HTML when activated.
 2021-02-26 18:21:04 +01:00
Mike Fährmann
e5735361ed [exhentai] add 'metadata' option (#1325) 
to select between gallery metadata from 'api' or 'html'
 2021-02-23 00:23:42 +01:00
Mike Fährmann
8f095a0980 [exhentai] extract more metadata from gallery pages (#1325) 2021-02-22 22:58:31 +01:00
Mike Fährmann
9b2e5f72d6 [exhentai] update image URL parsing (#1094) 2020-11-02 15:28:54 +01:00
Mike Fährmann
3ebb174f2c add missing extractor info when spawning new ones (fixes #1051) 
Not having this information causes the blacklist/whitelist logic to
trigger and prevents things from functioning as intended when using
default settings.

Fixes issues for 8muses, deviantart, exhentai, and mangoxo.
 2020-10-08 14:34:53 +02:00
Mike Fährmann
da87a5fb7e [exhentai] fix accessing config before main constructor 
bug introduced with 055c32e0

Making 'Extractor.config()' quite  a bit faster is worth the "cost"
of having to set _cfgpath in exhentai constructors, I think.
 2020-09-15 18:09:50 +02:00
Mike Fährmann
a0d916ed41 [exhentai] update wait time before original image download (#978) 
depend on 'wait-max', don't use a hard-coded value
 2020-09-07 23:48:28 +02:00
Mike Fährmann
0f55b8e80a [exhentai] fix type check from dbbbb21 (#940) 
'bool' is a subclass of 'int', and therefore
'isinstance(self.limits, int)' also returns True when
'self.limits' has a boolean value
 2020-08-30 20:51:22 +02:00
Mike Fährmann
dbbbb21180 [exhentai] add ability to specify custom image limit (#940) 2020-08-17 22:29:20 +02:00
Mike Fährmann
cd9de613a2 [exhentai] adjust image limit costs (#940) 
Each original file costs 10 points per 10^6 bytes,
not 10 per 2^20 == 1048576 bytes.
 2020-08-15 18:19:33 +02:00
Mike Fährmann
ecaecc4064 [exhentai] add 'domain' option (#897) 2020-07-17 22:17:46 +02:00
Mike Fährmann
6b373cb7e2 [exhentai] restrict default directory name length (#545) 2019-12-27 21:29:00 +01:00
Mike Fährmann
4409d00141 embed error messages in StopExtraction exceptions 2019-10-28 16:39:49 +01:00
Mike Fährmann
1848788970 update test results etc 2019-09-08 11:33:35 +02:00
Mike Fährmann
9ae58a6b3e [exhentai] update image limit checks 
- adjust cost of original images
- delay limit initialization until gallery and first image page have
  been requested and all cookies are available
 2019-09-07 13:29:01 +02:00
Mike Fährmann
fa60109e97 [exhentai] don't use e-hentai.org for exhentai URLs 2019-08-02 21:10:09 +02:00
Mike Fährmann
beb4fab2e6 [exhentai] improve limit and error handling (#360) 
- check image limit before opening the first gallery or image page
- prevent any further exhentai extractors from running after the image
  limit has been reached
 2019-07-30 22:58:35 +02:00
Mike Fährmann
81b35ed3cb [exhentai] catch more error states (#356, #360) 
- warn on MPV-enabled galleries
- catch parsing errors for gallery pages and image info
- write page content to debug output
 2019-07-29 16:54:31 +02:00
Mike Fährmann
6ce22f606b [exhentai] update login procedure and tests 
Logging in now follows the natural login flow that also happens in a
browser more closely and collects more cookies than just ipb_member_id
and ipb_pass_hash.

Test URLs have been updated and now point to the e-hentai.org domain.
 2019-07-28 16:51:05 +02:00
Mike Fährmann
dc73d02d87 [exhentai] always use e-hentai.org as domain + set nw cookie 2019-07-28 10:54:17 +02:00
Mike Fährmann
fdec59f8e2 replace extractor.request() 'expect' argument 
with
- 'fatal': allow 4xx status codes
- 'notfound': raise NotFoundError on 404
 2019-07-05 00:42:16 +02:00
Mike Fährmann
1c36e65e9b [exhentai] choose site version depending on input URL (#278) 
Use e-hentai.org as root and cookiedomain if the input URL is from
e-hentai (or g.e-hentai), use exhentai.org otherwise.
 2019-05-31 15:34:39 +02:00
Mike Fährmann
1f7fa9dc8e [exhentai] update data extraction code 
- parse 'date' to datetime object
- use 'text.extract_from()'
 2019-05-08 15:44:29 +02:00
Mike Fährmann
5398bfbd69 [exhentai] fix search and favorite extraction 
removes basically all metadata, but that can be compensated for with the
right search query. writing "parsers" for all 4 possible views that have
been introduced in the latest changes is too much of a hassle ...
 2019-03-28 16:22:02 +01:00
Mike Fährmann
a2af2d2965 adjust cache maxage values 2019-03-14 22:21:49 +01:00