github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,469 Commits 1 Branch 0 Tags
36fa543715ff6830bff3b02409e91c315429f062
Commit Graph

93 Commits

Author SHA1 Message Date
Mike Fährmann
d7c97d5a97 use f-strings when building 'pattern' 2025-10-20 21:23:11 +02:00
Mike Fährmann
9bf76c1352 replace 'util.re()' with 'text.re()' 
remove unnecessary 'util' imports
 2025-10-20 17:44:58 +02:00
Mike Fährmann
c8fc790028 merge branch 'dt': move datetime utils into separate module 
- use 'datetime.fromisoformat()' when possible (#7671)
- return a datetime-compatible object for invalid datetimes
  (instead of a 'str' value)
 2025-10-20 09:30:05 +02:00
Mike Fährmann
989136bc59 [dt] replace 'parse_compat' with compat workaround 2025-10-19 18:26:32 +02:00
Mike Fährmann
6c71b279b6 [dt] update 'parse_datetime' calls with one argument 2025-10-17 22:49:41 +02:00
Mike Fährmann
085616e0a8 [dt] replace 'text.parse_datetime()' & 'text.parse_timestamp()' 2025-10-17 17:43:06 +02:00
Mike Fährmann
8c62be343e [output] add 'Logger.traceback()' helper 2025-10-14 18:44:29 +02:00
Mike Fährmann
7c10fce562 [newgrounds] extract 'slug' metadata (#8064) 2025-08-17 23:52:28 +02:00
Mike Fährmann
6d65c65647 [newgrounds] add fallback for 'full_image_text' with empty 'src' 2025-08-16 15:44:34 +02:00
Mike Fährmann
a097a373a9 simplify if statements by using walrus operators (#7671) 2025-07-22 20:57:54 +02:00
Mike Fährmann
d8ef1d693f rename 'StopExtraction' to 'AbortExtraction' 
for cases where StopExtraction was used to report errors
 2025-07-09 21:07:28 +02:00
Mike Fährmann
f2a72d8d1e replace 'request(…).json()' with 'request_json(…)' 2025-06-29 17:50:19 +02:00
Mike Fährmann
9dbe33b6de replace old %-formatted and .format(…) strings with f-strings (#7671) 
mostly using flynt
https://github.com/ikamensh/flynt
 2025-06-29 17:50:19 +02:00
Mike Fährmann
41191bb60a 'match.group(N)' -> 'match[N]' (#7671) 
2.5x faster
 2025-06-18 13:05:58 +02:00
Mike Fährmann
e08ec7e083 update copyright notices 2025-06-13 00:03:41 +02:00
Mike Fährmann
811b665e33 remove @staticmethod decorators 
There might have been a time when calling a static method was faster
than a regular method, but that is no longer the case. According to
micro-benchmarks, it is 70% slower in CPython 3.13 and it also makes
executing the code of a class definition slower.
 2025-06-12 22:50:52 +02:00
Mike Fährmann
b5c88b3d3e replace standard library 're' uses with 'util.re()' 2025-06-06 13:24:52 +02:00
Mike Fährmann
f9dd8318f1 [newgrounds] ignore '<script>' content in 'tags' (#7604) 2025-05-30 21:52:48 +02:00
Mike Fährmann
e199396872 [common] simplify 'user' extractors by using 'Dispatch' mixin 2025-05-24 18:04:53 +02:00
Mike Fährmann
57937c3b68 [newgrounds] provide 'comment_html' metadata (#7038) 2025-02-22 10:08:58 +01:00
Mike Fährmann
b069783578 [newgrounds] fix metadata extraction (#6463) 
- fix 'comment' metadata
- fix 'following' extractor pattern
- use own 'type' values, since 'og:type' is no longer available
- update test results
 2024-11-18 16:21:59 +01:00
Mike Fährmann
4358799833 [newgrounds] support page numbers in URLs (#6320) 
https://USER.newgrounds.com/art/?page=5
https://USER.newgrounds.com/art/page/5
 2024-10-14 14:25:09 +02:00
Mike Fährmann
7d0e5877ce [newgrounds] add more fallback URLs for 'art-images' files 2024-10-01 08:22:16 +02:00
Mike Fährmann
f3f27496d6 [newgrounds] support more comment embed formats (#6253) 2024-09-30 08:20:50 +02:00
Mike Fährmann
a051e1c955 directly pass exception instances as 'exc_info' logger argument 2024-09-19 14:50:08 +02:00
Mike Fährmann
c495bfe460 [newgrounds] fix warning for age-restricted posts (#6005) 2024-08-13 08:16:15 +02:00
Mike Fährmann
c6fc0281e8 [newgrounds] extend 'format' option (#5709) 
- check more extensions for original formats (mp4, webm, m4v, mov, mkv)
- allow specifying which extensions and recoded formats to check
 2024-06-12 20:46:45 +02:00
Mike Fährmann
329da4db5e [newgrounds] update and fix login procedure (#5109) 2024-05-23 17:06:22 +02:00
Mike Fährmann
57fc6fcf83 replace '24*3600' with '86400' 
and generalize cache maxage values
 2023-12-18 23:57:22 +01:00
Mike Fährmann
1f9b16a70b replace static 'sleep-request' defaults with dynamic ones 2023-12-18 22:06:26 +01:00
Mike Fährmann
7958ab1946 [newgrounds] support 'imageData' files (#4642) 2023-10-21 13:22:55 +02:00
Mike Fährmann
c4c4e4d2f4 [newgrounds] improve 'art-image' extraction (#4642) 
- download files in original resolution
- replace .webp with extension of first file
 2023-10-13 20:10:55 +02:00
Mike Fährmann
15f940819b [newgrounds] support 'art-image' files (#4642) 2023-10-09 11:20:10 +02:00
Mike Fährmann
3ecb512722 send Referer headers by default 2023-09-19 00:02:04 +02:00
Mike Fährmann
27ec653991 fix bug in test_init and update example URLs 2023-09-14 13:27:03 +02:00
Mike Fährmann
a453335a9f remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
a383eca7f6 decouple extractor initialization 
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
 2023-07-25 22:16:16 +02:00
Mike Fährmann
d97b8c2fba consistent cookie-related names 
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
 2023-07-22 01:20:50 +02:00
Mike Fährmann
a16d7c59cb [newgrounds] access 'response.text' only once 2023-07-04 21:49:57 +02:00
FrostTheFox
9576652fa5 extract & pass auth token for newgrounds 2023-07-04 02:35:48 -04:00
Mike Fährmann
c698c3de44 [newgrounds] add default delay between requests (#4046) 2023-05-11 16:04:37 +02:00
Mike Fährmann
c9a7345228 [newgrounds] prevent archive ID overlap (#3681) 
add an 'i' and 'a' prefix to image and audio files
(/art/view/, /audio/listen/)
since their numeric ID may conflict with movies and other media
 2023-03-06 15:03:49 +01:00
Mike Fährmann
dd884b02ee replace json.loads with direct calls to JSONDecoder.decode 2023-02-09 15:22:00 +01:00
Mike Fährmann
b0cb4a1b9c replace 'text.extract()' with 'text.extr()' where possible 2022-11-05 01:14:09 +01:00
Mike Fährmann
ff532d6c3c [newgrounds] extract 'type' metadata 2022-09-24 20:29:43 +02:00
Mike Fährmann
0393e59535 [newgrounds] add 'games' extractor (#2955) 2022-09-24 12:34:37 +02:00
Mike Fährmann
c794777600 [newgrounds] prevent exception on empty results (#2727) 2022-07-03 11:44:46 +02:00
Mike Fährmann
37453a9528 [newgrounds] only login if necessary (#2715) 2022-06-29 11:46:07 +02:00
Mike Fährmann
c1768972c2 [newgrounds] update and fix pagination (#2456) 2022-04-07 15:38:41 +02:00
Mike Fährmann
a53cfc845e [newgrounds] warn about age-restricted posts (#2456) 2022-03-30 16:18:33 +02:00