github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,413 Commits 1 Branch 0 Tags
ea3d95e7e8e12d316d9479bc62d9ebcf1d96b100
Commit Graph

149 Commits

Author SHA1 Message Date
Mike Fährmann
362cd6991b [pixiv] implement 'metadata-bookmark' option (#3417) 2023-01-07 23:19:43 +01:00
Mike Fährmann
a6d4733e11 [pixiv] extract 'date_url' metadata (#3405) 
i.e. the datetime encoded in each file URL.

https://i.pximg.net/img-master/img/2022/12/01/13/44/55/12345678_p0.jpg
->
2022-12-01 13:44:55 +09:00
->
2022-12-01 04:44:55
 2022-12-15 11:40:20 +01:00
Mike Fährmann
5a17e15b76 [pixiv] preserve 'tags' order (#3266) 
for '"tags": "translated"'

As it turns out, set() does *not* preserve insertion order.
 2022-11-22 19:11:37 +01:00
Mike Fährmann
eaae4d9b65 [pixiv] stop with error for invalid search/ranking parameters 
instead of falling back to defaults
 2022-11-15 12:17:53 +01:00
Mike Fährmann
368f156378 [pixiv] rankings: add support for the new daily AI and daily AI R18 
(#3214, #3221)

In remembrance of @thatfuckingbird
 2022-11-15 11:47:57 +01:00
Mike Fährmann
b0cb4a1b9c replace 'text.extract()' with 'text.extr()' where possible 2022-11-05 01:14:09 +01:00
Mike Fährmann
769e6754dc [pixiv] use 'exact_match_for_tags' as default search mode (#3092) 2022-10-24 16:08:12 +02:00
Mike Fährmann
52d1eb928d [pixiv] extend 'metadata' option (#3057) 
make it usable for all 'pixiv' extractors
 2022-10-16 15:32:31 +02:00
Mike Fährmann
63e0924927 [pixiv] add 'series' extractor (#2964) 2022-09-27 23:24:03 +02:00
Mike Fährmann
d5ded11aa8 [pixiv] fix default filenames for backgrounds 2022-07-11 12:45:38 +02:00
Mike Fährmann
345199a3ec [pixiv] include '.gif' in background fallback URLs (#2495) 2022-06-03 17:25:23 +02:00
Mike Fährmann
4005171db3 [pixiv] provide more metadata fields when option enabled (#2594) 2022-05-15 14:47:14 +02:00
Mike Fährmann
6ae3a5cdb0 [pixiv] make retrieving ugoira metadata non-fatal (#2562) 2022-05-08 20:05:38 +02:00
Mike Fährmann
9adea93aef [pixiv] updates to avatar/background extractors (#2495) 
- add 'date' metadata to avatar/background files when available
  and use that in default filenames / archive ids
- remove deprecation warnings as their option names clash with
  subcategory names
 2022-05-04 17:30:54 +02:00
Mike Fährmann
84756982e9 [pixiv] implement 'include' option 
- split 'user' extractor and its 'avatar' and 'background' options into
  separate extractors ('artworks', 'avatar', 'background')
- avatars can now be downloaded with
  https://www.pixiv.net/en/users/ID/avatar
  as URL and will use a proper archive key; similar for backgrounds
- options for the 'user' subcategory must be moved to 'artworks' to have
  the same effect as before
 2022-05-02 09:03:35 +02:00
Mike Fährmann
82eee72b39 [pixiv] update API interface 
- start all endpoints with '/'
- use extractor.wait() for rate limit
- retry with while loop instead of recursion
- in case of error, write entire response to debug log
 2022-05-02 09:03:34 +02:00
Mike Fährmann
9e6ff42a9d [pixiv] implement 'background' option (#623, #1124, #2495) 2022-04-21 13:53:02 +02:00
Mike Fährmann
4bec34fc94 [pixiv] allow setting a date range for search results (#2133) 
with the 'scd' and 'ecd' query parameters
 2021-12-23 23:03:39 +01:00
Mike Fährmann
e33125ad39 [pixiv] add 'sketch' extractor (#1497) 2021-10-13 00:02:11 +02:00
Mike Fährmann
eed6ef3de0 [pixiv] fix pixivision title extraction 2021-09-02 22:34:59 +02:00
Mike Fährmann
bd08ee2859 remove most 'yield Message.Version' statements 
only leave them in oauth.py as noop results
 2021-08-16 03:10:48 +02:00
Mike Fährmann
8ecca3af58 [pixiv] add extractor for 'pixivision' articles (#1672) 2021-07-07 16:27:16 +02:00
Mike Fährmann
7273cf8536 [pixiv] support fetching privately followed users (fixes #1628) 2021-06-16 19:56:09 +02:00
thatfuckingbird
e6811c7450 [pixiv] implement 'max-posts' option (#1558) 
* implement max-rank for pixiv

* rename to max-posts and make more generic
 2021-05-24 17:49:46 +02:00
Mike Fährmann
5eeaaee01d [pixiv] add 'metadata' option (#1551) 2021-05-14 20:30:28 +02:00
Mike Fährmann
36ed1efcfb [pixiv] rename "noop" value for 'tags' option to "original" 
(#1507)
 2021-05-07 20:41:54 +02:00
Mike Fährmann
fa519f9202 [pixiv] change 'translated-tags' option (#1507) 
- rename to 'tags'
- use string-values: "japanese", "translated", "noop"
- remove duplicate entries for "translated" tags
 2021-04-29 19:30:43 +02:00
Mike Fährmann
221015e586 [downloader:http] disable filename extension changes for ugoira 
(#1507)
 2021-04-27 01:29:09 +02:00
thatfuckingbird
141ca4ac0a [pixiv] also save untranslated tags when translated-tags is enabled (#1501) 2021-04-23 23:02:41 +02:00
beesdotjson
5ad615f0db fix PixivFavoriteExtractor regex (#1405) 
* fix PixivFavoriteExtractor regex

* do not use lookbehind
 2021-03-25 14:59:33 +01:00
Mike Fährmann
7440d1f112 [pixiv] add 'translated-tags' option (closes #1354) 
(a lot more straight forward than I thought ...)
 2021-03-05 17:18:51 +01:00
Mike Fährmann
8974f0361c [pixiv] update (#1304) 
- remove login with username & password
- require a refresh token
- add 'oauth:pixiv' functionality

See also:
- https://github.com/upbit/pixivpy/issues/158
- https://gist.github.com/ZipFile/c9ebedb224406f4f11845ab700124362
 2021-02-12 18:07:16 +01:00
Mike Fährmann
193dca2ce1 update extractor test results 2021-01-21 21:35:42 +01:00
Mike Fährmann
c008cb5100 [pixiv] add 'related' option (#1237) 2021-01-17 22:48:32 +01:00
Mike Fährmann
3bd08acc8f [pixiv] output debug message on failed login attempt 
(#1192)
 2020-12-22 14:59:31 +01:00
Mike Fährmann
b58e605dc7 raise error when required username or password are missing 
do not try to login as 'None' (#1192)
 2020-12-22 14:40:18 +01:00
Mike Fährmann
968d3e8465 remove '&' from URL patterns 
'/?&#' -> '/?#' and '?&#' -> '?#'

According to https://www.ietf.org/rfc/rfc3986.txt, URLs are
"organized hierarchically" by using "the slash ("/"), question
mark ("?"), and number sign ("#") characters to delimit components"
 2020-10-22 23:31:25 +02:00
Mike Fährmann
844793847c update extractor test results 2020-10-11 18:15:41 +02:00
Mike Fährmann
844502cad5 update extractor test results 2020-10-03 19:24:19 +02:00
Mike Fährmann
a45fbc38ea [pixiv] implement 'avatar' option (#595, #623) 2020-03-09 21:18:16 +01:00
Mike Fährmann
ce26070231 [pixiv] reduce calls to '/user/detail' 2020-02-09 13:54:58 +01:00
Mike Fährmann
b7eb6cecbb [pixiv] handle tags at the end of new bookmark URLs 2020-02-06 23:42:13 +01:00
Mike Fährmann
40fe062851 [pixiv] fix user id for bookmarks API calls (closes #596) 2020-02-01 01:48:46 +01:00
Mike Fährmann
91aaaf1a9e [pixiv] add 'rating' metadata field (#595) 
A human-friendlier representation of 'x_restrict'
 2020-02-01 01:36:06 +01:00
Mike Fährmann
71851a6241 [pixiv] update URLs of followed users to the new format 2020-01-30 15:17:42 +01:00
Mike Fährmann
b9cbf932b4 [pixiv] update URL patterns (fixes #568) 
Pixiv now uses new URLs for
- user profiles and illustration listings:
  - https://www.pixiv.net/en/users/<ID>
  - https://www.pixiv.net/en/users/<ID>/artworks
- bookmarks:
  - https://www.pixiv.net/en/users/<ID>/bookmarks/artworks
 2020-01-10 13:38:09 +01:00
Mike Fährmann
ce54b8c04c let extractors opt-out of cookie option usage 
useful to avoid sending unnecessary cookies when all authentication
is done through OAuth tokens
 2020-01-01 21:12:37 +01:00
Mike Fährmann
1f2a69f3c5 add '_extractor' information to redirect results 2019-12-29 23:37:34 +01:00
Mike Fährmann
a27f43dad1 [pixiv] wait and retry after rate limit error (closes #535) 2019-12-28 22:06:58 +01:00
Mike Fährmann
c0f391a4e2 [pixiv] support listing followed users (#515) 2019-12-26 23:42:42 +01:00