github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,722 Commits 1 Branch 0 Tags
3a4e19d284eae3b34a8a7712521bbd2a62127ecc
Commit Graph

211 Commits

Author SHA1 Message Date
Mike Fährmann
b5c88b3d3e replace standard library 're' uses with 'util.re()' 2025-06-06 13:24:52 +02:00
Mike Fährmann
0389a622e2 [pixiv] remove '/jump.php' from AJAX caption links (#4327) 
https://github.com/mikf/gallery-dl/issues/4327#issuecomment-2924753613
 2025-05-31 19:20:39 +02:00
Mike Fährmann
e199396872 [common] simplify 'user' extractors by using 'Dispatch' mixin 2025-05-24 18:04:53 +02:00
prowlguru
b8f3ca6c4e [pixiv] Implement sanity handling for ugoira works (#4327 #6297 #7285 #7434) 
* [pixiv] Implement sanity handling for ugoira works

* [pixiv] fix PEP formatting for sanity ugoira handling

* [pixiv] update Ugoira AJAX error message

* [pixiv] respect disabled 'ugoira' option

* [tests] add 'limit_sanity_level' ugoira test

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-05-21 22:01:09 +02:00
prowlguru
5fa1e13866 [pixiv:novel] fix 'embeds' extraction with AJAX request (#7422 #7435) 
* [pixiv] fix novel embeds with AJAX request

All novel embeds seem to be broken since
`meta-preload-data` is no longer present in the
html of the php response after a site change.

* [pixiv] remove headers used for old novel embed request

* [pixiv] cleanup novel embed fix

* [pixiv] handle novel embed exceptions

* [pixiv] fix PEP formatting for novel embeds fix

* Fix error message

* [pixiv] include exception info in novel error message

---------

Co-authored-by: prowlguru <prowlguru@github.com>
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-04-28 17:14:41 +02:00
Mike Fährmann
3ac33dce61 [pixiv:novel] handle exceptions during embeds extraction (#7422) 2025-04-26 15:11:15 +02:00
prowlguru
6306b110fd [pixiv] fix 'user_details' requests not being cached 
Calls to user_detail would sometimes have an
integer user_id and sometimes string. Since this
is used as a key for caching this function, it
would cause an extra request to be made.

All calls to user_detail should be sending user_id
as a string, to avoid having multiple entries in
the cache, causing an extra request to be made.

Authored by: prowlguru
 2025-04-26 14:50:59 +02:00
Mike Fährmann
88f1541a83 [common] add 'request_location()' convenience function 2025-04-19 16:45:05 +02:00
Mike Fährmann
2ef7a05e10 [pixiv] update 'work' extractor pattern & add tests 2025-04-12 16:51:17 +02:00
w4tchdoge
1ee057285d [pixiv] match phixiv.net URLs 2025-04-12 19:30:04 +08:00
Mike Fährmann
587205bf68 [pixiv] prevent exceptions during 'comments' extraction (#6965) 
- wrap in try-except block
- do not attempt to fetch comments for 'sanity_level' works
 2025-02-10 09:56:09 +01:00
Mike Fährmann
db9cee20c2 [pixiv] fix 'sanity_level' workaround (#4327) 
The App API now returns
https://s.pximg.net/common/images/limit_unviewable_360.png
as URL for "Work cannot be displayed" artworks.
 2025-01-27 10:10:38 +01:00
Mike Fährmann
270aaea8ab [pixiv] provide fallback URLs (#6762) 2025-01-06 15:27:32 +01:00
Mike Fährmann
990907572a [pixiv] include user ID in failed AJAX warnings (#6581) 2024-12-01 20:44:17 +01:00
Mike Fährmann
c5685efbf7 [pixiv:ranking] implement filtering results by 'content' (#6574) 2024-12-01 18:01:43 +01:00
Mike Fährmann
79fd3445ee [pixiv:ranking] add 'rank' metadata field (#6531) 2024-11-28 19:34:55 +01:00
Mike Fährmann
7916c8bf77 allow passing cookies to OAuth extractors 
partially revert ce54b8c04c
 2024-11-09 18:06:27 +01:00
Mike Fährmann
b321696182 [pixiv] get ugoira frame extension from 'meta_single_page' (#6056) 
saves 1-3 HEAD requests
 2024-10-31 19:47:05 +01:00
Mike Fährmann
8dce52471a [pixiv] restructure file extraction code 2024-10-27 09:01:12 +01:00
Mike Fährmann
ae448f8ca3 [pixiv] fix/improve PHPSESSID warning code 2024-10-27 08:59:46 +01:00
Mike Fährmann
75674944f0 [pixiv] implement 'sanity_level' workaround for user artworks results 
(#4327, #5435, #6339)
 2024-10-22 20:38:00 +02:00
Mike Fährmann
33161da121 [pixiv] add 'captions' option (#4327) 
make extra requests for empty captions independent of 'sanity'
 2024-10-22 16:31:37 +02:00
Mike Fährmann
2d15e64ea9 [pixiv] fix exception when processing deleted sanity_level work (#6339) 
https://github.com/mikf/gallery-dl/issues/6339#issuecomment-2427350524
 2024-10-21 21:13:26 +02:00
Mike Fährmann
c7f0d8945b [pixiv] make retrieving ugoira metadata non-fatal (#6297) 
update to 6ae3a5cdb0
catch all exceptions instead of just StopException
 2024-10-14 14:25:09 +02:00
Mike Fährmann
f8bb0fd6cb [pixiv][danbooru] set '_ugoira_original' for ugoira results 2024-10-11 13:19:45 +02:00
Mike Fährmann
1f5f04e673 [pixiv] make '_request_ajax()' non-fatal 2024-10-11 07:55:02 +02:00
Mike Fährmann
04df813e1b [pixiv] detect works requiring 'My pixiv' access 2024-10-10 15:57:02 +02:00
Mike Fährmann
82269fdc09 [pixiv] disable 'sanity' and 'comments' for avatar and background 
fixes regression introduced in e05b9b10 and 8bcf7bf5
 2024-10-10 15:00:11 +02:00
Mike Fährmann
e05b9b101e [pixiv] retrieve 'caption' from AJAX API when empty (#4327, #5191) 2024-10-09 17:50:27 +02:00
Mike Fährmann
cc38042500 [pp:ugoira] forward frame data as '_ugoira_frame_data' (#6154, #6285) 2024-10-08 11:28:44 +02:00
Mike Fährmann
8bcf7bf5ee [pixiv] add 'comments' option (#6287) 2024-10-06 20:41:36 +02:00
Mike Fährmann
7f945c44f5 [pixiv] support unlisted artworks (#5162) 2024-10-05 17:10:03 +02:00
Mike Fährmann
c5be50fdaa [pixiv] implement workaround for 'limit_sanity_level' works 
(#4327, #4747, #5054, #5435, #5651, #5655)

Metadata should be ~95% identical (there might be some 'date' differences)
and there could be issues with R-18 works, as these require some URL
manipulation to transform /c/250x250_80_a2/ thumbnail URLs into
/img-original/ ones.
 2024-10-04 21:07:56 +02:00
Mike Fährmann
d1432d02a1 [pixiv] move file/ugoira extraction into a separate function 2024-10-04 16:21:19 +02:00
Mike Fährmann
2c7a0c3ca8 add alternatives for deprecated utc datetime functions 2024-09-19 20:47:05 +02:00
Mike Fährmann
07bd967f59 [pp:ugoira] update (#6056) 
- introduce '_ugoira_frame_index' metadata field
- store Ugoira file exts separately
- add 'skip' option
 2024-09-05 20:25:20 +02:00
Mike Fährmann
9d1e5f3c9b [pixiv] allow downloading "original" ugoira frames (#6056) 2024-09-01 13:37:32 +02:00
Mike Fährmann
b33a53968d [pixiv] incorporate into '_pagination' 2024-08-12 11:50:41 +02:00
mix5003
b6a40217d6 [pixiv] use api for illust series 2024-08-10 20:29:13 +07:00
Mike Fährmann
c9d3b5e5d9 [pixiv] change 'sanity_level' debug message to a warning (#5180) 2024-04-19 16:41:31 +02:00
Mike Fährmann
35d4a706ae [pixiv:novel] add 'covers' option (#5373) 2024-04-11 22:27:49 +02:00
Mike Fährmann
db507e30c7 [pixiv] fix novel text extraction (#5285) 
change to '/webview/v2/novel'
since '/v1/novel/text' does not work anymore
 2024-03-06 02:31:26 +01:00
Se AKi
d0d199414f modify useragent of pixiv 2024-01-06 23:15:15 +09:00
Mike Fährmann
57fc6fcf83 replace '24*3600' with '86400' 
and generalize cache maxage values
 2023-12-18 23:57:22 +01:00
Mike Fährmann
1f9b16a70b replace static 'sleep-request' defaults with dynamic ones 2023-12-18 22:06:26 +01:00
Mike Fährmann
dd14adccf6 [pixiv] allow cookies for non-OAuth URLs (#4760) 2023-11-03 23:39:58 +01:00
Mike Fährmann
a453335a9f remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
8259a5abe4 flake8 2023-09-02 13:03:11 +02:00
johnsmith1202gmail
c7e31b2724 Update pixiv.py 2023-08-30 12:42:31 -07:00
johnsmith1202gmail
d3046561d4 continue downloading when the item is made private on pixiv 2023-08-30 18:21:06 +00:00