github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,376 Commits 1 Branch 0 Tags
d3003f8531610e125e4f4502f4e6c75b9d2260a2
Commit Graph

170 Commits

Author SHA1 Message Date
Mike Fährmann
db507e30c7 [pixiv] fix novel text extraction (#5285) 
change to '/webview/v2/novel'
since '/v1/novel/text' does not work anymore
 2024-03-06 02:31:26 +01:00
Se AKi
d0d199414f modify useragent of pixiv 2024-01-06 23:15:15 +09:00
Mike Fährmann
57fc6fcf83 replace '24*3600' with '86400' 
and generalize cache maxage values
 2023-12-18 23:57:22 +01:00
Mike Fährmann
1f9b16a70b replace static 'sleep-request' defaults with dynamic ones 2023-12-18 22:06:26 +01:00
Mike Fährmann
dd14adccf6 [pixiv] allow cookies for non-OAuth URLs (#4760) 2023-11-03 23:39:58 +01:00
Mike Fährmann
a453335a9f remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
8259a5abe4 flake8 2023-09-02 13:03:11 +02:00
johnsmith1202gmail
c7e31b2724 Update pixiv.py 2023-08-30 12:42:31 -07:00
johnsmith1202gmail
d3046561d4 continue downloading when the item is made private on pixiv 2023-08-30 18:21:06 +00:00
Mike Fährmann
a383eca7f6 decouple extractor initialization 
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
 2023-07-25 22:16:16 +02:00
Mike Fährmann
a45a17ddb7 [pixiv] ignore 'limit_sanity_level' images (#4328) 2023-07-22 14:57:38 +02:00
Mike Fährmann
d97b8c2fba consistent cookie-related names 
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
 2023-07-22 01:20:50 +02:00
Mike Fährmann
0b34a444e0 [pixiv:novel] only detect Pixiv embeds (#4175) 2023-06-13 18:58:35 +02:00
Mike Fährmann
0cf7282fa0 [pixiv] add 'full-series' option for novels (#4111) 2023-06-01 13:07:20 +02:00
Mike Fährmann
ffed7efb6f [pixiv] use BASE_PATTERN 2023-05-28 18:06:47 +02:00
Mike Fährmann
b286efefcc [pixiv] add 'novel-bookmark' extractor (#4111) 2023-05-28 16:30:17 +02:00
Mike Fährmann
3fca455b82 [pixiv] add 'embeds' option (#1241) 2023-05-23 12:14:06 +02:00
Mike Fährmann
56b8b8cd36 [pixiv] support short novel URLs 
https://www.pixiv.net/n/<ID>
 2023-05-21 14:26:30 +02:00
Mike Fährmann
20dc13f832 [pixiv] initial 'novel' support (#1241, #4044) 
supported URLs are
- https://www.pixiv.net/novel/show.php?id=<ID>
- https://www.pixiv.net/novel/series/<ID>
- https://www.pixiv.net/en/users/<ID>/novels
 2023-05-12 16:34:08 +02:00
Mike Fährmann
b12dad8df5 [pixiv] fix 'pixivision' extraction 2023-04-30 15:35:32 +02:00
thatfuckingbird
9f76783ac0 [pixiv] allow sorting by popularity (requires pixiv premium) 2023-04-26 22:49:29 +02:00
Mike Fährmann
362cd6991b [pixiv] implement 'metadata-bookmark' option (#3417) 2023-01-07 23:19:43 +01:00
Mike Fährmann
a6d4733e11 [pixiv] extract 'date_url' metadata (#3405) 
i.e. the datetime encoded in each file URL.

https://i.pximg.net/img-master/img/2022/12/01/13/44/55/12345678_p0.jpg
->
2022-12-01 13:44:55 +09:00
->
2022-12-01 04:44:55
 2022-12-15 11:40:20 +01:00
Mike Fährmann
5a17e15b76 [pixiv] preserve 'tags' order (#3266) 
for '"tags": "translated"'

As it turns out, set() does *not* preserve insertion order.
 2022-11-22 19:11:37 +01:00
Mike Fährmann
eaae4d9b65 [pixiv] stop with error for invalid search/ranking parameters 
instead of falling back to defaults
 2022-11-15 12:17:53 +01:00
Mike Fährmann
368f156378 [pixiv] rankings: add support for the new daily AI and daily AI R18 
(#3214, #3221)

In remembrance of @thatfuckingbird
 2022-11-15 11:47:57 +01:00
Mike Fährmann
b0cb4a1b9c replace 'text.extract()' with 'text.extr()' where possible 2022-11-05 01:14:09 +01:00
Mike Fährmann
769e6754dc [pixiv] use 'exact_match_for_tags' as default search mode (#3092) 2022-10-24 16:08:12 +02:00
Mike Fährmann
52d1eb928d [pixiv] extend 'metadata' option (#3057) 
make it usable for all 'pixiv' extractors
 2022-10-16 15:32:31 +02:00
Mike Fährmann
63e0924927 [pixiv] add 'series' extractor (#2964) 2022-09-27 23:24:03 +02:00
Mike Fährmann
d5ded11aa8 [pixiv] fix default filenames for backgrounds 2022-07-11 12:45:38 +02:00
Mike Fährmann
345199a3ec [pixiv] include '.gif' in background fallback URLs (#2495) 2022-06-03 17:25:23 +02:00
Mike Fährmann
4005171db3 [pixiv] provide more metadata fields when option enabled (#2594) 2022-05-15 14:47:14 +02:00
Mike Fährmann
6ae3a5cdb0 [pixiv] make retrieving ugoira metadata non-fatal (#2562) 2022-05-08 20:05:38 +02:00
Mike Fährmann
9adea93aef [pixiv] updates to avatar/background extractors (#2495) 
- add 'date' metadata to avatar/background files when available
  and use that in default filenames / archive ids
- remove deprecation warnings as their option names clash with
  subcategory names
 2022-05-04 17:30:54 +02:00
Mike Fährmann
84756982e9 [pixiv] implement 'include' option 
- split 'user' extractor and its 'avatar' and 'background' options into
  separate extractors ('artworks', 'avatar', 'background')
- avatars can now be downloaded with
  https://www.pixiv.net/en/users/ID/avatar
  as URL and will use a proper archive key; similar for backgrounds
- options for the 'user' subcategory must be moved to 'artworks' to have
  the same effect as before
 2022-05-02 09:03:35 +02:00
Mike Fährmann
82eee72b39 [pixiv] update API interface 
- start all endpoints with '/'
- use extractor.wait() for rate limit
- retry with while loop instead of recursion
- in case of error, write entire response to debug log
 2022-05-02 09:03:34 +02:00
Mike Fährmann
9e6ff42a9d [pixiv] implement 'background' option (#623, #1124, #2495) 2022-04-21 13:53:02 +02:00
Mike Fährmann
4bec34fc94 [pixiv] allow setting a date range for search results (#2133) 
with the 'scd' and 'ecd' query parameters
 2021-12-23 23:03:39 +01:00
Mike Fährmann
e33125ad39 [pixiv] add 'sketch' extractor (#1497) 2021-10-13 00:02:11 +02:00
Mike Fährmann
eed6ef3de0 [pixiv] fix pixivision title extraction 2021-09-02 22:34:59 +02:00
Mike Fährmann
bd08ee2859 remove most 'yield Message.Version' statements 
only leave them in oauth.py as noop results
 2021-08-16 03:10:48 +02:00
Mike Fährmann
8ecca3af58 [pixiv] add extractor for 'pixivision' articles (#1672) 2021-07-07 16:27:16 +02:00
Mike Fährmann
7273cf8536 [pixiv] support fetching privately followed users (fixes #1628) 2021-06-16 19:56:09 +02:00
thatfuckingbird
e6811c7450 [pixiv] implement 'max-posts' option (#1558) 
* implement max-rank for pixiv

* rename to max-posts and make more generic
 2021-05-24 17:49:46 +02:00
Mike Fährmann
5eeaaee01d [pixiv] add 'metadata' option (#1551) 2021-05-14 20:30:28 +02:00
Mike Fährmann
36ed1efcfb [pixiv] rename "noop" value for 'tags' option to "original" 
(#1507)
 2021-05-07 20:41:54 +02:00
Mike Fährmann
fa519f9202 [pixiv] change 'translated-tags' option (#1507) 
- rename to 'tags'
- use string-values: "japanese", "translated", "noop"
- remove duplicate entries for "translated" tags
 2021-04-29 19:30:43 +02:00
Mike Fährmann
221015e586 [downloader:http] disable filename extension changes for ugoira 
(#1507)
 2021-04-27 01:29:09 +02:00
thatfuckingbird
141ca4ac0a [pixiv] also save untranslated tags when translated-tags is enabled (#1501) 2021-04-23 23:02:41 +02:00