github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,316 Commits 1 Branch 0 Tags
c1d21e8cb9fab9c3c19ecdfede65db3722e69fb9
Commit Graph

229 Commits

Author SHA1 Message Date
Mike Fährmann
602c84e1fa [pixiv] implement searching past 'offset=5000' (#1686 #7082 #8298) 
by adjusting 'start_date' or 'end_date' depending on sort order
 2025-09-30 21:56:25 +02:00
Mike Fährmann
c71766e6ab [pixiv] distinguish empty from deleted profiles (#8066) 
fixes regression introduced in d5e2fe4f44
 2025-08-17 23:51:50 +02:00
Mike Fährmann
a70b4ab693 [pixiv] improve API error messages 2025-08-08 11:57:48 +02:00
Mike Fährmann
d5e2fe4f44 [pixiv] detect suspended/deleted accounts (#7990) 2025-08-08 11:56:57 +02:00
Mike Fährmann
f2a58118eb [pixiv] fix artworks downloads when using expired cookies (#7987) 2025-08-07 20:58:29 +02:00
Mike Fährmann
ba174b1a06 [pixiv] remove redundant cookies initialization 
7916c8bf77
75674944f0
 2025-08-07 20:58:29 +02:00
Mike Fährmann
9a50588792 [pixiv] fix 'IndexError' for unviewable works (#7940) 2025-08-01 17:59:39 +02:00
Mike Fährmann
8e33d0b985 [pixiv] improve AJAX error handling (#7896) 2025-07-26 07:33:58 +02:00
Mike Fährmann
a097a373a9 simplify if statements by using walrus operators (#7671) 2025-07-22 20:57:54 +02:00
Mike Fährmann
e68dc960c2 [pixiv] add workaround for 'novel-*' includes (#7746) 2025-07-11 22:27:45 +02:00
Mike Fährmann
d8ef1d693f rename 'StopExtraction' to 'AbortExtraction' 
for cases where StopExtraction was used to report errors
 2025-07-09 21:07:28 +02:00
Mike Fährmann
82891b4d0c [pixiv] move 'novel' extractors to a 'pixiv-novel' category (#7746) 
TODO:
- restore full 'include' functionality
- allow remapping category:subcategory pairs
 2025-07-04 20:13:19 +02:00
Mike Fährmann
f2a72d8d1e replace 'request(…).json()' with 'request_json(…)' 2025-06-29 17:50:19 +02:00
Mike Fährmann
9dbe33b6de replace old %-formatted and .format(…) strings with f-strings (#7671) 
mostly using flynt
https://github.com/ikamensh/flynt
 2025-06-29 17:50:19 +02:00
Mike Fährmann
41191bb60a 'match.group(N)' -> 'match[N]' (#7671) 
2.5x faster
 2025-06-18 13:05:58 +02:00
Mike Fährmann
b583891df6 [pixiv] remove '/jump.php' from *all* AJAX caption links (#4327) 
https://github.com/mikf/gallery-dl/issues/4327#issuecomment-2969765775
 2025-06-13 21:10:51 +02:00
Mike Fährmann
e08ec7e083 update copyright notices 2025-06-13 00:03:41 +02:00
Mike Fährmann
811b665e33 remove @staticmethod decorators 
There might have been a time when calling a static method was faster
than a regular method, but that is no longer the case. According to
micro-benchmarks, it is 70% slower in CPython 3.13 and it also makes
executing the code of a class definition slower.
 2025-06-12 22:50:52 +02:00
Mike Fährmann
b5c88b3d3e replace standard library 're' uses with 'util.re()' 2025-06-06 13:24:52 +02:00
Mike Fährmann
0389a622e2 [pixiv] remove '/jump.php' from AJAX caption links (#4327) 
https://github.com/mikf/gallery-dl/issues/4327#issuecomment-2924753613
 2025-05-31 19:20:39 +02:00
Mike Fährmann
e199396872 [common] simplify 'user' extractors by using 'Dispatch' mixin 2025-05-24 18:04:53 +02:00
prowlguru
b8f3ca6c4e [pixiv] Implement sanity handling for ugoira works (#4327 #6297 #7285 #7434) 
* [pixiv] Implement sanity handling for ugoira works

* [pixiv] fix PEP formatting for sanity ugoira handling

* [pixiv] update Ugoira AJAX error message

* [pixiv] respect disabled 'ugoira' option

* [tests] add 'limit_sanity_level' ugoira test

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-05-21 22:01:09 +02:00
prowlguru
5fa1e13866 [pixiv:novel] fix 'embeds' extraction with AJAX request (#7422 #7435) 
* [pixiv] fix novel embeds with AJAX request

All novel embeds seem to be broken since
`meta-preload-data` is no longer present in the
html of the php response after a site change.

* [pixiv] remove headers used for old novel embed request

* [pixiv] cleanup novel embed fix

* [pixiv] handle novel embed exceptions

* [pixiv] fix PEP formatting for novel embeds fix

* Fix error message

* [pixiv] include exception info in novel error message

---------

Co-authored-by: prowlguru <prowlguru@github.com>
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-04-28 17:14:41 +02:00
Mike Fährmann
3ac33dce61 [pixiv:novel] handle exceptions during embeds extraction (#7422) 2025-04-26 15:11:15 +02:00
prowlguru
6306b110fd [pixiv] fix 'user_details' requests not being cached 
Calls to user_detail would sometimes have an
integer user_id and sometimes string. Since this
is used as a key for caching this function, it
would cause an extra request to be made.

All calls to user_detail should be sending user_id
as a string, to avoid having multiple entries in
the cache, causing an extra request to be made.

Authored by: prowlguru
 2025-04-26 14:50:59 +02:00
Mike Fährmann
88f1541a83 [common] add 'request_location()' convenience function 2025-04-19 16:45:05 +02:00
Mike Fährmann
2ef7a05e10 [pixiv] update 'work' extractor pattern & add tests 2025-04-12 16:51:17 +02:00
w4tchdoge
1ee057285d [pixiv] match phixiv.net URLs 2025-04-12 19:30:04 +08:00
Mike Fährmann
587205bf68 [pixiv] prevent exceptions during 'comments' extraction (#6965) 
- wrap in try-except block
- do not attempt to fetch comments for 'sanity_level' works
 2025-02-10 09:56:09 +01:00
Mike Fährmann
db9cee20c2 [pixiv] fix 'sanity_level' workaround (#4327) 
The App API now returns
https://s.pximg.net/common/images/limit_unviewable_360.png
as URL for "Work cannot be displayed" artworks.
 2025-01-27 10:10:38 +01:00
Mike Fährmann
270aaea8ab [pixiv] provide fallback URLs (#6762) 2025-01-06 15:27:32 +01:00
Mike Fährmann
990907572a [pixiv] include user ID in failed AJAX warnings (#6581) 2024-12-01 20:44:17 +01:00
Mike Fährmann
c5685efbf7 [pixiv:ranking] implement filtering results by 'content' (#6574) 2024-12-01 18:01:43 +01:00
Mike Fährmann
79fd3445ee [pixiv:ranking] add 'rank' metadata field (#6531) 2024-11-28 19:34:55 +01:00
Mike Fährmann
7916c8bf77 allow passing cookies to OAuth extractors 
partially revert ce54b8c04c
 2024-11-09 18:06:27 +01:00
Mike Fährmann
b321696182 [pixiv] get ugoira frame extension from 'meta_single_page' (#6056) 
saves 1-3 HEAD requests
 2024-10-31 19:47:05 +01:00
Mike Fährmann
8dce52471a [pixiv] restructure file extraction code 2024-10-27 09:01:12 +01:00
Mike Fährmann
ae448f8ca3 [pixiv] fix/improve PHPSESSID warning code 2024-10-27 08:59:46 +01:00
Mike Fährmann
75674944f0 [pixiv] implement 'sanity_level' workaround for user artworks results 
(#4327, #5435, #6339)
 2024-10-22 20:38:00 +02:00
Mike Fährmann
33161da121 [pixiv] add 'captions' option (#4327) 
make extra requests for empty captions independent of 'sanity'
 2024-10-22 16:31:37 +02:00
Mike Fährmann
2d15e64ea9 [pixiv] fix exception when processing deleted sanity_level work (#6339) 
https://github.com/mikf/gallery-dl/issues/6339#issuecomment-2427350524
 2024-10-21 21:13:26 +02:00
Mike Fährmann
c7f0d8945b [pixiv] make retrieving ugoira metadata non-fatal (#6297) 
update to 6ae3a5cdb0
catch all exceptions instead of just StopException
 2024-10-14 14:25:09 +02:00
Mike Fährmann
f8bb0fd6cb [pixiv][danbooru] set '_ugoira_original' for ugoira results 2024-10-11 13:19:45 +02:00
Mike Fährmann
1f5f04e673 [pixiv] make '_request_ajax()' non-fatal 2024-10-11 07:55:02 +02:00
Mike Fährmann
04df813e1b [pixiv] detect works requiring 'My pixiv' access 2024-10-10 15:57:02 +02:00
Mike Fährmann
82269fdc09 [pixiv] disable 'sanity' and 'comments' for avatar and background 
fixes regression introduced in e05b9b10 and 8bcf7bf5
 2024-10-10 15:00:11 +02:00
Mike Fährmann
e05b9b101e [pixiv] retrieve 'caption' from AJAX API when empty (#4327, #5191) 2024-10-09 17:50:27 +02:00
Mike Fährmann
cc38042500 [pp:ugoira] forward frame data as '_ugoira_frame_data' (#6154, #6285) 2024-10-08 11:28:44 +02:00
Mike Fährmann
8bcf7bf5ee [pixiv] add 'comments' option (#6287) 2024-10-06 20:41:36 +02:00
Mike Fährmann
7f945c44f5 [pixiv] support unlisted artworks (#5162) 2024-10-05 17:10:03 +02:00