github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,033 Commits 1 Branch 0 Tags
cb9a1176e67ed3bf41ca2aed83419621bf7ece77
Commit Graph

65 Commits

Author SHA1 Message Date
Mike Fährmann
a453335a9f remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
1d2b5d0c60 update test comment positions 
always put them above the test they're referring to
 2023-09-06 18:16:09 +02:00
Mike Fährmann
1ece3b92ff [mangadex] allow multiple values for 'lang' (#4093) 
This was already possible by setting 'lang' to a list of strings,
but now it can also be done as a more command-line friendly string.

-o lang=fr,it
 2023-07-26 17:39:27 +02:00
Mike Fährmann
a383eca7f6 decouple extractor initialization 
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
 2023-07-25 22:16:16 +02:00
Mike Fährmann
f3cca50b9e [mangadex] update links to API docs 2023-05-24 12:32:13 +02:00
Mike Fährmann
2266fc8cc5 [mangadex] update and extend test results 2023-05-07 20:14:49 +02:00
Janne Alaranta
1ce5dc9e18 fix whitespaces 2023-05-07 18:47:04 +03:00
Janne Alaranta
13dedae09f add status and tags info to mangadex extractor 2023-05-07 18:35:02 +03:00
Mike Fährmann
3d8777fbc1 move user agent string to util.py 2023-02-22 11:09:17 +01:00
Mike Fährmann
253ac08203 pre-define and use 'gallery-dö/<version>' UA string 2023-02-18 22:25:54 +01:00
Mike Fährmann
cd931e1139 update extractor test results 2022-12-08 18:58:29 +01:00
Mike Fährmann
4d1896830f [mangadex] download chapters with 'externalUrl' (fixes #2503) 
if the have pages hosted on mangadex
 2022-04-18 18:09:52 +02:00
Mike Fährmann
4ea9157d51 [mangadex] fix chapters without 'translatedLanguage' (#2352) 2022-03-01 02:04:25 +01:00
Mike Fährmann
c587b678d0 [mangadex] re-enable warning for external chapters (#2193) 2022-01-16 03:21:50 +01:00
Mike Fährmann
170711af7e [mangadex] fix extraction (closes #2177) 2022-01-08 17:21:35 +01:00
Mike Fährmann
11a3d96d13 [mangadex] load additional metadata using includes[] directives 
- always provide 'artist', 'author', and 'group' metadata fields (#2049)
- remove 'metadata' option
 2021-11-22 01:16:33 +01:00
Mike Fährmann
d93b5474c3 [mangadex] update parameter handling for API requests 
- move common parameters into '_pagination()'
- add 'ratings' (#1908) and 'api-parameters' options
 2021-10-05 19:46:48 +02:00
Mike Fährmann
96215c926e [mangadex] fix retrieving chapters from 'pornographic' titles 
(fixes #1908)
 2021-10-01 16:48:53 +02:00
Mike Fährmann
e69ee41f25 implement 'page-reverse' option (#1854) 2021-09-23 18:02:19 +02:00
Mike Fährmann
6bbeaac029 [mangadex] fix extraction (fixes #1852) 2021-09-14 17:48:41 +02:00
Mike Fährmann
ddd175de77 [mangadex] prevent KeyError for manga without English title 2021-08-21 22:46:23 +02:00
Mike Fährmann
dfe1f490e9 [mangadex] use custom User-Agent header (#1535) 2021-07-15 16:39:32 +02:00
Mike Fährmann
53dab5c289 [mangadex] revert chapter handling (#1535) 
Spawn a new ChapterExtractor for each individual chapter
instead of handling them directly with a MangaExtractor.

Doing it that way broke too many features like
--chapter-filter, --chapter-range, --zip, etc.
 2021-06-09 21:54:40 +02:00
Mike Fährmann
1197ee2c20 [mangadex] add extractor for a user's followed feed (#1535) 2021-06-08 02:45:36 +02:00
Mike Fährmann
07c8adbd8b [mangadex] implement login with username & password (#1535) 2021-06-08 02:12:57 +02:00
Mike Fährmann
3e332eaf53 [mangadex] update to API v5 (#1535) 2021-06-07 01:29:42 +02:00
Mike Fährmann
69ea781d32 [mangadex] improve caching of manga results 
'manga_id' being a string or integer are treated as two different keys
 2021-02-28 01:34:58 +01:00
Mike Fährmann
e58039358d [mangadex] use 'api.mangadex.org' as default API server 
The caching issues seem to be gone.
(#1290, #1310)
 2021-02-28 01:31:50 +01:00
Mike Fährmann
79c0fc249b [mangadex] add 'api-server' option (#1309) 
and change the API server back to 'https://mangadex.org/api' for now
 2021-02-11 19:52:16 +01:00
Mike Fährmann
193dca2ce1 update extractor test results 2021-01-21 21:35:42 +01:00
Mike Fährmann
b0cf968115 [mangadex] update API URLs 
https://mangadex.org/thread/351011/9/#post_4238014
 2021-01-15 21:11:58 +01:00
Mike Fährmann
2c4d4a75db [mangadex] respect 'chapter-reverse' settings (closes #1194) 
The extractor in question doesn't inherit from MangaExtractor
and therefore didn't do this automatically.
 2020-12-22 15:08:10 +01:00
Mike Fährmann
e5d81bdc7b [mangadex] handle 'external' chapters (closes #1154) 2020-12-04 20:56:30 +01:00
Mike Fährmann
42ccae53c4 [mangadex] switch to API v2 
https://mangadex.org/api/v2/
https://mangadex.org/thread/351011
 2020-11-16 11:05:17 +01:00
Mike Fährmann
43dab3a228 [mangadex] unescape more metadata fields (fixes #1066) 
like 'manga', 'author', 'artist', etc.
 2020-10-16 00:41:15 +02:00
Mike Fährmann
7daef6ee70 update extractor test results 
- certain posts on Instagram now return
  https://static.cdninstagram.com/rsrc.php/null.jpg
  for public users
- MangaDex is deploying its new MangaDex@Home network similar to
  exhentai's Hentai@Home
- realbooru has a new site layout, but the underlying booru API still
  works like before
 2020-06-12 00:36:06 +02:00
Mike Fährmann
3bad1579ee update extractor test results 2020-05-31 17:42:07 +02:00
Mike Fährmann
45baa13615 update extractor test results 
- don't run Instagram tests on Travis anymore
- replace Twitter test because timeline was made private
- update Hiperdex domain to '.com' (again ...)
 2020-05-28 02:18:06 +02:00
Mike Fährmann
2530db3f4d [mangadex] transform 'date' timestamps to datetime objects 2020-03-18 02:19:19 +01:00
Mike Fährmann
af42c75152 [mangadex] revert domain to 'mangadex.org' 2020-01-14 09:49:57 +01:00
Mike Fährmann
988cc2ec23 [mangadex] change domain to mangadex.cc (closes #559) 2020-01-05 21:21:40 +01:00
Mike Fährmann
a2af2d2965 adjust cache maxage values 2019-03-14 22:21:49 +01:00
Mike Fährmann
5530871b5a change results of text.nameext_from_url() 
Instead of getting a complete 'filename' from an URL and splitting that
into 'name' and 'extension', the new approach gets rid of the complete
version and renames 'name' to 'filename'. (Using anything other than
{extension} for a filename extension doesn't really work anyway)

Example: "https://example.org/path/filename.ext"

before:
- filename : filename.ext
- name     : filename
- extension: ext

now:
- filename : filename
- extension: ext
 2019-02-14 16:07:17 +01:00
Mike Fährmann
61741d7333 provide type information for Queue messages 
Child extractors are now directly constructed with Extractor.from_url()
if the extractor class is known beforehand, instead of using
extractor.find() and searching through all possible extractor classes.
 2019-02-12 21:32:32 +01:00
Mike Fährmann
580baef72c change Chapter and MangaExtractor classes 
- unify and simplify constructors
- rename get_metadata and get_images to just metadata() and images()
- rename self.url to chapter_url and manga_url
 2019-02-11 18:38:47 +01:00
Mike Fährmann
4b1880fa5e propagate 'match' to base extractor constructor 2019-02-11 13:31:10 +01:00
Mike Fährmann
6284731107 simplify extractor constants 
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
 2019-02-08 13:45:40 +01:00
Mike Fährmann
a303efb597 [mangadex] handle manga pages without chapters 2019-01-03 16:22:12 +01:00
Mike Fährmann
b47af4637a [mangadex] update URL pattern 
Manga URLs now begin with /title/ instead of /manga/
 2018-08-31 20:16:50 +02:00
Mike Fährmann
2af2bb7911 [mangadex] fix relative page URLs 2018-08-25 11:07:26 +02:00