github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,921 Commits 1 Branch 0 Tags
7c43f9e152633a8a0c6d759d60c738d6efbedc25
Commit Graph

183 Commits

Author SHA1 Message Date
Mike Fährmann
e05b9b101e [pixiv] retrieve 'caption' from AJAX API when empty (#4327, #5191) 2024-10-09 17:50:27 +02:00
Mike Fährmann
cc38042500 [pp:ugoira] forward frame data as '_ugoira_frame_data' (#6154, #6285) 2024-10-08 11:28:44 +02:00
Mike Fährmann
8bcf7bf5ee [pixiv] add 'comments' option (#6287) 2024-10-06 20:41:36 +02:00
Mike Fährmann
7f945c44f5 [pixiv] support unlisted artworks (#5162) 2024-10-05 17:10:03 +02:00
Mike Fährmann
c5be50fdaa [pixiv] implement workaround for 'limit_sanity_level' works 
(#4327, #4747, #5054, #5435, #5651, #5655)

Metadata should be ~95% identical (there might be some 'date' differences)
and there could be issues with R-18 works, as these require some URL
manipulation to transform /c/250x250_80_a2/ thumbnail URLs into
/img-original/ ones.
 2024-10-04 21:07:56 +02:00
Mike Fährmann
d1432d02a1 [pixiv] move file/ugoira extraction into a separate function 2024-10-04 16:21:19 +02:00
Mike Fährmann
2c7a0c3ca8 add alternatives for deprecated utc datetime functions 2024-09-19 20:47:05 +02:00
Mike Fährmann
07bd967f59 [pp:ugoira] update (#6056) 
- introduce '_ugoira_frame_index' metadata field
- store Ugoira file exts separately
- add 'skip' option
 2024-09-05 20:25:20 +02:00
Mike Fährmann
9d1e5f3c9b [pixiv] allow downloading "original" ugoira frames (#6056) 2024-09-01 13:37:32 +02:00
Mike Fährmann
b33a53968d [pixiv] incorporate into '_pagination' 2024-08-12 11:50:41 +02:00
mix5003
b6a40217d6 [pixiv] use api for illust series 2024-08-10 20:29:13 +07:00
Mike Fährmann
c9d3b5e5d9 [pixiv] change 'sanity_level' debug message to a warning (#5180) 2024-04-19 16:41:31 +02:00
Mike Fährmann
35d4a706ae [pixiv:novel] add 'covers' option (#5373) 2024-04-11 22:27:49 +02:00
Mike Fährmann
db507e30c7 [pixiv] fix novel text extraction (#5285) 
change to '/webview/v2/novel'
since '/v1/novel/text' does not work anymore
 2024-03-06 02:31:26 +01:00
Se AKi
d0d199414f modify useragent of pixiv 2024-01-06 23:15:15 +09:00
Mike Fährmann
57fc6fcf83 replace '24*3600' with '86400' 
and generalize cache maxage values
 2023-12-18 23:57:22 +01:00
Mike Fährmann
1f9b16a70b replace static 'sleep-request' defaults with dynamic ones 2023-12-18 22:06:26 +01:00
Mike Fährmann
dd14adccf6 [pixiv] allow cookies for non-OAuth URLs (#4760) 2023-11-03 23:39:58 +01:00
Mike Fährmann
a453335a9f remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
8259a5abe4 flake8 2023-09-02 13:03:11 +02:00
johnsmith1202gmail
c7e31b2724 Update pixiv.py 2023-08-30 12:42:31 -07:00
johnsmith1202gmail
d3046561d4 continue downloading when the item is made private on pixiv 2023-08-30 18:21:06 +00:00
Mike Fährmann
a383eca7f6 decouple extractor initialization 
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
 2023-07-25 22:16:16 +02:00
Mike Fährmann
a45a17ddb7 [pixiv] ignore 'limit_sanity_level' images (#4328) 2023-07-22 14:57:38 +02:00
Mike Fährmann
d97b8c2fba consistent cookie-related names 
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
 2023-07-22 01:20:50 +02:00
Mike Fährmann
0b34a444e0 [pixiv:novel] only detect Pixiv embeds (#4175) 2023-06-13 18:58:35 +02:00
Mike Fährmann
0cf7282fa0 [pixiv] add 'full-series' option for novels (#4111) 2023-06-01 13:07:20 +02:00
Mike Fährmann
ffed7efb6f [pixiv] use BASE_PATTERN 2023-05-28 18:06:47 +02:00
Mike Fährmann
b286efefcc [pixiv] add 'novel-bookmark' extractor (#4111) 2023-05-28 16:30:17 +02:00
Mike Fährmann
3fca455b82 [pixiv] add 'embeds' option (#1241) 2023-05-23 12:14:06 +02:00
Mike Fährmann
56b8b8cd36 [pixiv] support short novel URLs 
https://www.pixiv.net/n/<ID>
 2023-05-21 14:26:30 +02:00
Mike Fährmann
20dc13f832 [pixiv] initial 'novel' support (#1241, #4044) 
supported URLs are
- https://www.pixiv.net/novel/show.php?id=<ID>
- https://www.pixiv.net/novel/series/<ID>
- https://www.pixiv.net/en/users/<ID>/novels
 2023-05-12 16:34:08 +02:00
Mike Fährmann
b12dad8df5 [pixiv] fix 'pixivision' extraction 2023-04-30 15:35:32 +02:00
thatfuckingbird
9f76783ac0 [pixiv] allow sorting by popularity (requires pixiv premium) 2023-04-26 22:49:29 +02:00
Mike Fährmann
362cd6991b [pixiv] implement 'metadata-bookmark' option (#3417) 2023-01-07 23:19:43 +01:00
Mike Fährmann
a6d4733e11 [pixiv] extract 'date_url' metadata (#3405) 
i.e. the datetime encoded in each file URL.

https://i.pximg.net/img-master/img/2022/12/01/13/44/55/12345678_p0.jpg
->
2022-12-01 13:44:55 +09:00
->
2022-12-01 04:44:55
 2022-12-15 11:40:20 +01:00
Mike Fährmann
5a17e15b76 [pixiv] preserve 'tags' order (#3266) 
for '"tags": "translated"'

As it turns out, set() does *not* preserve insertion order.
 2022-11-22 19:11:37 +01:00
Mike Fährmann
eaae4d9b65 [pixiv] stop with error for invalid search/ranking parameters 
instead of falling back to defaults
 2022-11-15 12:17:53 +01:00
Mike Fährmann
368f156378 [pixiv] rankings: add support for the new daily AI and daily AI R18 
(#3214, #3221)

In remembrance of @thatfuckingbird
 2022-11-15 11:47:57 +01:00
Mike Fährmann
b0cb4a1b9c replace 'text.extract()' with 'text.extr()' where possible 2022-11-05 01:14:09 +01:00
Mike Fährmann
769e6754dc [pixiv] use 'exact_match_for_tags' as default search mode (#3092) 2022-10-24 16:08:12 +02:00
Mike Fährmann
52d1eb928d [pixiv] extend 'metadata' option (#3057) 
make it usable for all 'pixiv' extractors
 2022-10-16 15:32:31 +02:00
Mike Fährmann
63e0924927 [pixiv] add 'series' extractor (#2964) 2022-09-27 23:24:03 +02:00
Mike Fährmann
d5ded11aa8 [pixiv] fix default filenames for backgrounds 2022-07-11 12:45:38 +02:00
Mike Fährmann
345199a3ec [pixiv] include '.gif' in background fallback URLs (#2495) 2022-06-03 17:25:23 +02:00
Mike Fährmann
4005171db3 [pixiv] provide more metadata fields when option enabled (#2594) 2022-05-15 14:47:14 +02:00
Mike Fährmann
6ae3a5cdb0 [pixiv] make retrieving ugoira metadata non-fatal (#2562) 2022-05-08 20:05:38 +02:00
Mike Fährmann
9adea93aef [pixiv] updates to avatar/background extractors (#2495) 
- add 'date' metadata to avatar/background files when available
  and use that in default filenames / archive ids
- remove deprecation warnings as their option names clash with
  subcategory names
 2022-05-04 17:30:54 +02:00
Mike Fährmann
84756982e9 [pixiv] implement 'include' option 
- split 'user' extractor and its 'avatar' and 'background' options into
  separate extractors ('artworks', 'avatar', 'background')
- avatars can now be downloaded with
  https://www.pixiv.net/en/users/ID/avatar
  as URL and will use a proper archive key; similar for backgrounds
- options for the 'user' subcategory must be moved to 'artworks' to have
  the same effect as before
 2022-05-02 09:03:35 +02:00
Mike Fährmann
82eee72b39 [pixiv] update API interface 
- start all endpoints with '/'
- use extractor.wait() for rate limit
- retry with while loop instead of recursion
- in case of error, write entire response to debug log
 2022-05-02 09:03:34 +02:00