github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,891 Commits 1 Branch 0 Tags
master
Commit Graph

245 Commits

Author SHA1 Message Date
Mike Fährmann
53cdfaac37 [common] add reference to 'exception' module to Extractor class 
- remove 'exception' imports
- replace with 'self.exc'
 2026-02-15 10:57:22 +01:00
Mike Fährmann
12f5e24ab5 use sets for ' in { ... }' checks 2026-02-11 22:55:01 +01:00
Mike Fährmann
fdc59efdda [pixiv] fix errors when using metadata options for avatar/background 
(#9002)
 2026-02-05 12:07:42 +01:00
Mike Fährmann
cd83be41c5 [common] allow Dispatch 'alt' extractors to use custom URLs 2026-01-25 11:15:30 +01:00
Mike Fährmann
5e94bae906 [pixiv] support 'sketch' include (#8789) 2026-01-01 10:49:41 +01:00
Mike Fährmann
4e56be4def [pixiv] provide 'count' metadata (#8794) 2026-01-01 10:15:04 +01:00
Mike Fährmann
00c6821a3f replace 2-element f-strings with simple '+' concatenations 
Python's 'ast' module and its 'NodeVisitor' class
were incredibly helpful in identifying these
 2025-12-22 11:26:04 +01:00
Mike Fährmann
e006d26c8e Revert "use f-strings when building 'pattern'" 
revert d7c97d5a97.
 2025-12-20 22:07:37 +01:00
Mike Fährmann
a8ca947154 [pixiv] warn about invalid 'PHPSESSID' cookie (#8689) 2025-12-13 16:00:02 +01:00
Mike Fährmann
968597a302 yield 3-tuples for Message.Directory 
adapt tuples to the same length and semantics as other messages
 2025-12-05 21:39:52 +01:00
Mike Fährmann
d7c97d5a97 use f-strings when building 'pattern' 2025-10-20 21:23:11 +02:00
Mike Fährmann
9bf76c1352 replace 'util.re()' with 'text.re()' 
remove unnecessary 'util' imports
 2025-10-20 17:44:58 +02:00
Mike Fährmann
c8fc790028 merge branch 'dt': move datetime utils into separate module 
- use 'datetime.fromisoformat()' when possible (#7671)
- return a datetime-compatible object for invalid datetimes
  (instead of a 'str' value)
 2025-10-20 09:30:05 +02:00
Mike Fährmann
0eb3c8a994 [dt] replace 'util' datetime functions 2025-10-15 21:30:38 +02:00
Mike Fährmann
8c62be343e [output] add 'Logger.traceback()' helper 2025-10-14 18:44:29 +02:00
Mike Fährmann
6e4f9e2ce3 [pixiv] fix "KeyError: 'is_bookmarked'" (#8398) 2025-10-13 12:21:11 +02:00
Mike Fährmann
602c84e1fa [pixiv] implement searching past 'offset=5000' (#1686 #7082 #8298) 
by adjusting 'start_date' or 'end_date' depending on sort order
 2025-09-30 21:56:25 +02:00
Mike Fährmann
c71766e6ab [pixiv] distinguish empty from deleted profiles (#8066) 
fixes regression introduced in d5e2fe4f44
 2025-08-17 23:51:50 +02:00
Mike Fährmann
a70b4ab693 [pixiv] improve API error messages 2025-08-08 11:57:48 +02:00
Mike Fährmann
d5e2fe4f44 [pixiv] detect suspended/deleted accounts (#7990) 2025-08-08 11:56:57 +02:00
Mike Fährmann
f2a58118eb [pixiv] fix artworks downloads when using expired cookies (#7987) 2025-08-07 20:58:29 +02:00
Mike Fährmann
ba174b1a06 [pixiv] remove redundant cookies initialization 
7916c8bf77
75674944f0
 2025-08-07 20:58:29 +02:00
Mike Fährmann
9a50588792 [pixiv] fix 'IndexError' for unviewable works (#7940) 2025-08-01 17:59:39 +02:00
Mike Fährmann
8e33d0b985 [pixiv] improve AJAX error handling (#7896) 2025-07-26 07:33:58 +02:00
Mike Fährmann
a097a373a9 simplify if statements by using walrus operators (#7671) 2025-07-22 20:57:54 +02:00
Mike Fährmann
e68dc960c2 [pixiv] add workaround for 'novel-*' includes (#7746) 2025-07-11 22:27:45 +02:00
Mike Fährmann
d8ef1d693f rename 'StopExtraction' to 'AbortExtraction' 
for cases where StopExtraction was used to report errors
 2025-07-09 21:07:28 +02:00
Mike Fährmann
82891b4d0c [pixiv] move 'novel' extractors to a 'pixiv-novel' category (#7746) 
TODO:
- restore full 'include' functionality
- allow remapping category:subcategory pairs
 2025-07-04 20:13:19 +02:00
Mike Fährmann
f2a72d8d1e replace 'request(…).json()' with 'request_json(…)' 2025-06-29 17:50:19 +02:00
Mike Fährmann
9dbe33b6de replace old %-formatted and .format(…) strings with f-strings (#7671) 
mostly using flynt
https://github.com/ikamensh/flynt
 2025-06-29 17:50:19 +02:00
Mike Fährmann
41191bb60a 'match.group(N)' -> 'match[N]' (#7671) 
2.5x faster
 2025-06-18 13:05:58 +02:00
Mike Fährmann
b583891df6 [pixiv] remove '/jump.php' from *all* AJAX caption links (#4327) 
https://github.com/mikf/gallery-dl/issues/4327#issuecomment-2969765775
 2025-06-13 21:10:51 +02:00
Mike Fährmann
e08ec7e083 update copyright notices 2025-06-13 00:03:41 +02:00
Mike Fährmann
811b665e33 remove @staticmethod decorators 
There might have been a time when calling a static method was faster
than a regular method, but that is no longer the case. According to
micro-benchmarks, it is 70% slower in CPython 3.13 and it also makes
executing the code of a class definition slower.
 2025-06-12 22:50:52 +02:00
Mike Fährmann
b5c88b3d3e replace standard library 're' uses with 'util.re()' 2025-06-06 13:24:52 +02:00
Mike Fährmann
0389a622e2 [pixiv] remove '/jump.php' from AJAX caption links (#4327) 
https://github.com/mikf/gallery-dl/issues/4327#issuecomment-2924753613
 2025-05-31 19:20:39 +02:00
Mike Fährmann
e199396872 [common] simplify 'user' extractors by using 'Dispatch' mixin 2025-05-24 18:04:53 +02:00
prowlguru
b8f3ca6c4e [pixiv] Implement sanity handling for ugoira works (#4327 #6297 #7285 #7434) 
* [pixiv] Implement sanity handling for ugoira works

* [pixiv] fix PEP formatting for sanity ugoira handling

* [pixiv] update Ugoira AJAX error message

* [pixiv] respect disabled 'ugoira' option

* [tests] add 'limit_sanity_level' ugoira test

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-05-21 22:01:09 +02:00
prowlguru
5fa1e13866 [pixiv:novel] fix 'embeds' extraction with AJAX request (#7422 #7435) 
* [pixiv] fix novel embeds with AJAX request

All novel embeds seem to be broken since
`meta-preload-data` is no longer present in the
html of the php response after a site change.

* [pixiv] remove headers used for old novel embed request

* [pixiv] cleanup novel embed fix

* [pixiv] handle novel embed exceptions

* [pixiv] fix PEP formatting for novel embeds fix

* Fix error message

* [pixiv] include exception info in novel error message

---------

Co-authored-by: prowlguru <prowlguru@github.com>
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-04-28 17:14:41 +02:00
Mike Fährmann
3ac33dce61 [pixiv:novel] handle exceptions during embeds extraction (#7422) 2025-04-26 15:11:15 +02:00
prowlguru
6306b110fd [pixiv] fix 'user_details' requests not being cached 
Calls to user_detail would sometimes have an
integer user_id and sometimes string. Since this
is used as a key for caching this function, it
would cause an extra request to be made.

All calls to user_detail should be sending user_id
as a string, to avoid having multiple entries in
the cache, causing an extra request to be made.

Authored by: prowlguru
 2025-04-26 14:50:59 +02:00
Mike Fährmann
88f1541a83 [common] add 'request_location()' convenience function 2025-04-19 16:45:05 +02:00
Mike Fährmann
2ef7a05e10 [pixiv] update 'work' extractor pattern & add tests 2025-04-12 16:51:17 +02:00
w4tchdoge
1ee057285d [pixiv] match phixiv.net URLs 2025-04-12 19:30:04 +08:00
Mike Fährmann
587205bf68 [pixiv] prevent exceptions during 'comments' extraction (#6965) 
- wrap in try-except block
- do not attempt to fetch comments for 'sanity_level' works
 2025-02-10 09:56:09 +01:00
Mike Fährmann
db9cee20c2 [pixiv] fix 'sanity_level' workaround (#4327) 
The App API now returns
https://s.pximg.net/common/images/limit_unviewable_360.png
as URL for "Work cannot be displayed" artworks.
 2025-01-27 10:10:38 +01:00
Mike Fährmann
270aaea8ab [pixiv] provide fallback URLs (#6762) 2025-01-06 15:27:32 +01:00
Mike Fährmann
990907572a [pixiv] include user ID in failed AJAX warnings (#6581) 2024-12-01 20:44:17 +01:00
Mike Fährmann
c5685efbf7 [pixiv:ranking] implement filtering results by 'content' (#6574) 2024-12-01 18:01:43 +01:00
Mike Fährmann
79fd3445ee [pixiv:ranking] add 'rank' metadata field (#6531) 2024-11-28 19:34:55 +01:00