github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,033 Commits 1 Branch 0 Tags
cb9a1176e67ed3bf41ca2aed83419621bf7ece77
Commit Graph

128 Commits

Author SHA1 Message Date
Mike Fährmann
e97d7b1c85 [exhentai] fix empty api_url with '"source": "hitomi"' (#4829) 2023-11-18 21:06:41 +01:00
Mike Fährmann
0435c6e603 [exhentai] handle 'Downloading … requires GP' errors (#4576, #4763) 2023-11-12 22:47:40 +01:00
Mike Fährmann
61d6558322 [exhentai] try to avoid 'DH_KEY_TOO_SMALL' errors (#1021, #4593) 2023-11-04 17:30:27 +01:00
Mike Fährmann
69b931b9bb [exhentai] provide fallback URLs (#1021, #4745) 2023-11-04 17:06:46 +01:00
Mike Fährmann
43d0c49d7e [exhentai] fix original image URLs (#4754) 2023-11-02 15:46:01 +01:00
Mike Fährmann
68e72a836c [exhentai] fix extraction (#4730) 
- update to new API response layout
- use proper API server URL
- fix 'filesize' metadata
 2023-10-30 13:38:49 +01:00
Mike Fährmann
3ecb512722 send Referer headers by default 2023-09-19 00:02:04 +02:00
Mike Fährmann
a453335a9f remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
5ed245317d [exhentai] add 'fav' option (#4409) 
The name 'favorite' is already taken as extractor subcategory
 2023-08-23 23:58:11 +02:00
Mike Fährmann
fd6b413f3c [exhentai] fix 'domain' option (#4458) 
regression from a383eca7
 2023-08-23 23:58:04 +02:00
Mike Fährmann
a383eca7f6 decouple extractor initialization 
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
 2023-07-25 22:16:16 +02:00
Mike Fährmann
d97b8c2fba consistent cookie-related names 
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
 2023-07-22 01:20:50 +02:00
Mike Fährmann
7499fa7075 [exhentai] remove and update sad panda check 
there hasn't been a sad panda in several years
 2023-05-03 17:39:49 +02:00
Mike Fährmann
cd931e1139 update extractor test results 2022-12-08 18:58:29 +01:00
Mike Fährmann
202c1210d5 [exhentai] fix pagination 2022-12-06 22:39:13 +01:00
Mike Fährmann
20fbba9d7c [exhentai] add metadata to search results (#3181) 
'gallery_id' and 'gallery_token'
 2022-11-09 11:40:27 +01:00
Mike Fährmann
6a0c5e34f4 [exhentai] fix pagination (#3181) 2022-11-09 11:15:34 +01:00
Mike Fährmann
b0cb4a1b9c replace 'text.extract()' with 'text.extr()' where possible 2022-11-05 01:14:09 +01:00
Mike Fährmann
94b34f460e [exhentai] add slash to the end of gallery URLs (#2947) 2022-09-21 17:54:20 +02:00
Mike Fährmann
464ea90d14 [exhentai] guess extension for original files (#2842) 
makes it possible to sometimes, when guessed correctly ('.jpg'),
skip an original file download without costing image limit points
 2022-09-14 16:06:27 +02:00
Mike Fährmann
551fdf7ad7 [exhentai] move 509 check into its own function 2022-09-13 18:27:14 +02:00
Mike Fährmann
8b76149521 [exhentai] improve 509.gif detection (#2901) 2022-09-09 18:09:52 +02:00
Mike Fährmann
c6a9bab019 update extractor test results 2022-07-12 15:49:22 +02:00
Mike Fährmann
1f9a0e2fd8 update extractor test results 2022-04-18 17:24:00 +02:00
Mike Fährmann
dee0d22561 update extractor test results 2022-02-06 21:39:24 +01:00
Mike Fährmann
62692c6842 [exhentai] add 'source' option 
setting it to "hitomi" downloads the corresponding gallery from
hitomi.la; might be extended to other sources in the future
 2021-12-16 23:16:19 +01:00
Mike Fährmann
275543b2d2 update extractor test results 2021-11-27 19:26:44 +01:00
Mike Fährmann
f2ae179713 [exhentai] fix extraction for disowned galleries (closes #2055) 2021-11-24 21:26:16 +01:00
Mike Fährmann
57854624a1 [exhentai] improve image limits check (#1808) 
Check for a 'text/html' Content-Type instead of the very specific
137 bytes Content-Length, which might change depending on compression
or other factors.
 2021-08-29 21:19:55 +02:00
Mike Fährmann
c3b5c88b04 update extractor test results 2021-07-20 20:21:33 +02:00
Mike Fährmann
8fd8126117 fix ISO 639-1 code for Japanese 
"jp" -> "ja"
 2021-05-22 16:07:04 +02:00
Mike Fährmann
9514cb8c12 [exhentai] update 'limits' check (#1487) 
Only use 'limits' to set a custom upper bound.
Checking if the actual maximum gets exceeded is not necessary.
 2021-04-23 23:20:45 +02:00
Renan Vedovato Traba
9322c5e43b [exhentai] restore limit config (#1487) 
This partially reverts commit e9ec91c8
 2021-04-22 21:21:41 +02:00
Mike Fährmann
abafe71e04 [exhentai] fix image limit detection (closes #1437) 
check for image limit message when downloading original files
 2021-04-08 21:33:41 +02:00
Mike Fährmann
e9ec91c811 [exhentai] improve image limits check 
- check if current image is the '509 Bandwidth Exceeded' notification
  (https://ehgt.org/g/509.gif or https://exhentai.org/img/509.gif)
- remove 'limits' option
 2021-03-29 19:01:13 +02:00
Mike Fährmann
c7c3fef0bc [exhentai] support '/tag/' URLs (closes #1363) 2021-03-08 22:40:51 +01:00
Mike Fährmann
90830daf85 [exhentai] improve 'favorites' extraction (closes #1360) 
add special cases for when the favorite count is 0 (Never) or 1 (Once)
 2021-03-07 22:40:25 +01:00
Mike Fährmann
bae874f370 replace 'wait-min/-max' with 'sleep-request' 
on exhentai, idolcomplex, reactor
 2021-03-02 22:55:45 +01:00
Mike Fährmann
61fbbd2dae [exhentai] rename metadata fields to match API results (#1325) 
- gallery_id    -> gid
- gallery_token -> token
- title_jp      -> title_jpn
- visible       -> expunged
- gallery_size  -> filesize
- count         -> filecount

Also changes the function of the 'metadata' option.
It is now boolean and causes extra data fields from the API to be added
instead of completely replacing the data from HTML when activated.
 2021-02-26 18:21:04 +01:00
Mike Fährmann
e5735361ed [exhentai] add 'metadata' option (#1325) 
to select between gallery metadata from 'api' or 'html'
 2021-02-23 00:23:42 +01:00
Mike Fährmann
8f095a0980 [exhentai] extract more metadata from gallery pages (#1325) 2021-02-22 22:58:31 +01:00
Mike Fährmann
9b2e5f72d6 [exhentai] update image URL parsing (#1094) 2020-11-02 15:28:54 +01:00
Mike Fährmann
3ebb174f2c add missing extractor info when spawning new ones (fixes #1051) 
Not having this information causes the blacklist/whitelist logic to
trigger and prevents things from functioning as intended when using
default settings.

Fixes issues for 8muses, deviantart, exhentai, and mangoxo.
 2020-10-08 14:34:53 +02:00
Mike Fährmann
da87a5fb7e [exhentai] fix accessing config before main constructor 
bug introduced with 055c32e0

Making 'Extractor.config()' quite  a bit faster is worth the "cost"
of having to set _cfgpath in exhentai constructors, I think.
 2020-09-15 18:09:50 +02:00
Mike Fährmann
a0d916ed41 [exhentai] update wait time before original image download (#978) 
depend on 'wait-max', don't use a hard-coded value
 2020-09-07 23:48:28 +02:00
Mike Fährmann
0f55b8e80a [exhentai] fix type check from dbbbb21 (#940) 
'bool' is a subclass of 'int', and therefore
'isinstance(self.limits, int)' also returns True when
'self.limits' has a boolean value
 2020-08-30 20:51:22 +02:00
Mike Fährmann
dbbbb21180 [exhentai] add ability to specify custom image limit (#940) 2020-08-17 22:29:20 +02:00
Mike Fährmann
cd9de613a2 [exhentai] adjust image limit costs (#940) 
Each original file costs 10 points per 10^6 bytes,
not 10 per 2^20 == 1048576 bytes.
 2020-08-15 18:19:33 +02:00
Mike Fährmann
ecaecc4064 [exhentai] add 'domain' option (#897) 2020-07-17 22:17:46 +02:00
Mike Fährmann
6b373cb7e2 [exhentai] restrict default directory name length (#545) 2019-12-27 21:29:00 +01:00