github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,921 Commits 1 Branch 0 Tags
7c43f9e152633a8a0c6d759d60c738d6efbedc25
Commit Graph

94 Commits

Author SHA1 Message Date
Mike Fährmann
ef4c1b4fc5 [sankaku] restore old 'tags' format (#6043) 
lowercase + words separated by underscores
 2024-08-17 19:25:19 +02:00
Mike Fährmann
84eefeebd6 [sankaku] match URLs with 'www' subdomain (#5907) 2024-07-30 17:05:22 +02:00
Mike Fährmann
287a7d13cf [sankaku] implement 'notes' extraction (#5865) 2024-07-18 20:44:49 +02:00
Mike Fährmann
34a4ddc399 [sankaku] add 'id-format' option (#5073) 2024-01-26 17:56:08 +01:00
Mike Fährmann
a416d4c3d5 [sankaku] support post URLs with alphanumeric IDs (#5073) 2024-01-18 16:23:14 +01:00
Mike Fährmann
57fc6fcf83 replace '24*3600' with '86400' 
and generalize cache maxage values
 2023-12-18 23:57:22 +01:00
Mike Fährmann
645b4627ef [sankaku] update URL patterns 2023-11-24 02:41:52 +01:00
Mike Fährmann
c9a2be36d4 [sankaku] support '/posts/' tag search URLs (#4740) 2023-10-29 13:48:42 +01:00
Mike Fährmann
b52fd91ac6 [sankaku] support '/posts/' URLs (#4688) 2023-10-21 13:20:35 +02:00
Mike Fährmann
3ecb512722 send Referer headers by default 2023-09-19 00:02:04 +02:00
Mike Fährmann
a453335a9f remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
a60db454af [sankaku] update/fix API headers 
'Referer' and 'Origin' were both empty
 2023-08-04 17:14:43 +02:00
Mike Fährmann
d97b8c2fba consistent cookie-related names 
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
 2023-07-22 01:20:50 +02:00
Mike Fährmann
3845c0256d [sankaku] improve warnings for unavailable posts 2023-07-01 19:11:41 +02:00
Mike Fährmann
7f25cab56e [sankaku] support post URLs with MD5 hashes (#3952) 2023-04-23 16:46:40 +02:00
Mike Fährmann
faca32a850 [sankaku] sanitize 'date:…' tags (#1790) 2023-04-19 20:09:11 +02:00
Mike Fährmann
107c60c973 [sankaku] update URL pattern (#3523) 
match tag searches with language codes without a trailing slash
 2023-01-18 21:38:01 +01:00
Mike Fährmann
b0cb4a1b9c replace 'text.extract()' with 'text.extr()' where possible 2022-11-05 01:14:09 +01:00
Mike Fährmann
775895f44b [booru] refactor 'tags' and 'notes' extraction 
- move HTML request for post pages into its own function
- move gelbooru_v02.py notes extraction to gelbooru.py
  since it only works there
- clean up some code
 2022-10-31 12:01:19 +01:00
Mike Fährmann
5fd4374036 [sankaku] improve 429 and tag limit handling 2022-10-01 11:49:47 +02:00
Mike Fährmann
4089bceddd [sankaku] implement 'refresh' option (#2958) 2022-09-30 19:55:48 +02:00
Mike Fährmann
850608551c [sankaku] detect expired links (#2958) 2022-09-23 11:51:30 +02:00
Mike Fährmann
32c75d12e8 [sankaku] rewrite URLs to s.sankakucomplex.com (#2746) 2022-07-11 12:46:04 +02:00
Mike Fährmann
05d4a0215a [sankaku] extend URL patterns (fixes #2647) 
- support URLs with ISO 639-1 language codes
- support black.… and white.… subdomains
 2022-06-01 21:31:11 +02:00
Mike Fährmann
211de95dd0 update extractor test results 2021-11-01 02:58:53 +01:00
Mike Fährmann
9ed13703cc [sankaku] handle empty tags (fixes #1617) 2021-06-14 16:20:10 +02:00
Mike Fährmann
c5ca7905ce add 'noop()' and 'identity()' functions 2021-05-04 19:27:17 +02:00
Mike Fährmann
6fa20d456b [sankaku] update invalid-token detection (fixes #1515) 2021-04-30 22:04:45 +02:00
Mike Fährmann
bdfcc9c4b1 update extractor test results 2021-04-18 20:28:15 +02:00
Mike Fährmann
0e601de67b [sankaku] simplify 'pool' tags (#1388) 
normalize 'tags' and 'artist_tags' to a string-list
 2021-03-23 18:45:45 +01:00
Mike Fährmann
d085ade9d5 [sankaku] add 'tag_string' metadata field (#1388) 
The 'join()'ed version of 'tags'.
Handling lists in format strings isn't properly supported yet.
 2021-03-23 15:42:13 +01:00
Mike Fährmann
2dffd231b7 [sankaku] add enumeration index for books (#1388) 2021-03-23 15:32:54 +01:00
Mike Fährmann
96a51ff169 [sankaku] update invalid-token detection (fixes #1309) 2021-02-11 19:49:24 +01:00
Mike Fährmann
2da9068ea8 [sankaku] simplify login process 2021-01-12 00:15:22 +01:00
Mike Fährmann
b0beed7a06 [sankaku] add support for book searches (closes #1204) 2020-12-29 17:36:37 +01:00
Mike Fährmann
47a7a51944 [sankaku] fix 'invalid_token' detection 2020-12-27 02:31:01 +01:00
Mike Fährmann
e41e2be2f9 [booru] split '_prepare_post()' 2020-12-24 01:13:54 +01:00
Mike Fährmann
b233531aaa [sankaku] use '/posts' endpoint for single posts 2020-12-22 02:44:40 +01:00
Mike Fährmann
459a0af4f8 [sankaku] add support for sankaku.app URLs (closes #1193) 2020-12-22 01:57:53 +01:00
Mike Fährmann
537742c0ee [sankaku] normalize 'created_at' metadata (closes #1190) 2020-12-21 02:06:29 +01:00
Mike Fährmann
465015f75a [sankaku] reimplement login support (#1176, #1182) 2020-12-17 16:12:59 +01:00
Mike Fährmann
8d2e4e5f13 [booru] improve error handling 
e.g. for posts without a valid 'file_url' (#1176)
 2020-12-17 01:16:45 +01:00
Mike Fährmann
b2c55f0a72 [sankaku] remove login support 
The old login method for 'https://chan.sankakucomplex.com/user/login'
and the cookies it produces have no effect on the results from
'beta.sankakucomplex.com'.
 2020-12-08 21:05:47 +01:00
Mike Fährmann
ecdea799dd [sankaku] use 'beta.sankakucomplex.com' API endpoints 2020-12-05 22:08:58 +01:00
Mike Fährmann
1e3dd7330e merge SharedConfigMixin functionality into Extractor 2020-11-17 00:34:07 +01:00
Mike Fährmann
844793847c update extractor test results 2020-10-11 18:15:41 +02:00
Mike Fährmann
4409d00141 embed error messages in StopExtraction exceptions 2019-10-28 16:39:49 +01:00
Mike Fährmann
7a5e78741c [booru] build directory path for each file (#385) 2019-08-18 23:28:33 +02:00
Mike Fährmann
40637556fa [ngomik] fix extraction 2019-07-28 10:53:46 +02:00
Mike Fährmann
7a99e85943 [kissmanga] fix download URLs and file extensions 
The current Blogspot image URLs hosted on Kissmanga end with an
"invalid" query parameter (/000.png&upx=...), which doesn't get
recognized by 'spliturl()' and 'parseurl()' as such and gets therefore
included in the 'extension' field from 'text.nameext_from_url()'.
 2019-06-28 20:34:43 +02:00