github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,457 Commits 1 Branch 0 Tags
89be2f45e4e88be6069a8d3a341ebaa84f1236df
Commit Graph

190 Commits

Author SHA1 Message Date
featherbutt
06e3126bba [pp:metadata] add 'newline' option (#8439) 
* Add configuration setting to control newline characters in metadata.
* update docs/configuration
* introduce 'open()' method
* add 'newline' test
 2025-10-21 18:47:39 +02:00
Mike Fährmann
3deb5a413d improve arguments for relative '__import__()' 
using None for 'fromlist' is slightly faster than an empty tuple
 2025-10-21 12:06:50 +02:00
Mike Fährmann
c8fc790028 merge branch 'dt': move datetime utils into separate module 
- use 'datetime.fromisoformat()' when possible (#7671)
- return a datetime-compatible object for invalid datetimes
  (instead of a 'str' value)
 2025-10-20 09:30:05 +02:00
Mike Fährmann
0eb3c8a994 [dt] replace 'util' datetime functions 2025-10-15 21:30:38 +02:00
Mike Fährmann
8c62be343e [output] add 'Logger.traceback()' helper 2025-10-14 18:44:29 +02:00
Mike Fährmann
ca4e061386 [pp:exec] support '{_temppath}' replacement field (#8329) 2025-10-09 20:59:06 +02:00
Mike Fährmann
99d5c521d1 use 'encoding="utf-8"' when opening files in text mode (#8376) 2025-10-09 09:54:18 +02:00
Mike Fährmann
eee78f8148 [pp:python] restore archive functionality 
fixes regression introduced in 09f0ba8e9c
 2025-09-27 09:59:47 +02:00
Mike Fährmann
4052068d5a [pp:metadata] add '"mode": "print"' (#2691) 
https://github.com/mikf/gallery-dl/issues/2691#issuecomment-3276158980
 2025-09-22 17:54:53 +02:00
Mike Fährmann
2480ba2a55 [pp:python] add '"mode": "eval"' 2025-09-22 17:54:37 +02:00
Mike Fährmann
09f0ba8e9c [pp:python] move '_init_archive()' after events (#8243) 2025-09-22 11:21:17 +02:00
Mike Fährmann
e5db6efca9 [pp] close archive database connections (#8243) 2025-09-19 22:12:35 +02:00
cenodis
8b6fffe866 [pp:exec] use 'False' as default for 'start_new_session' to avoid TypeError (#7899) 
in Python <= 3.11

* Use False as default instead of None to avoid TypeError
* Update postprocessor tests
 2025-07-27 09:41:27 +02:00
Mike Fährmann
a097a373a9 simplify if statements by using walrus operators (#7671) 2025-07-22 20:57:54 +02:00
Mike Fährmann
9bbfb09952 [pp:exec] add 'session' option (#6582) 
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3067297033
 2025-07-20 19:55:33 +02:00
Mike Fährmann
3810555bbd do not use 'append = list.append' 2025-06-30 11:42:44 +02:00
Mike Fährmann
9dbe33b6de replace old %-formatted and .format(…) strings with f-strings (#7671) 
mostly using flynt
https://github.com/ikamensh/flynt
 2025-06-29 17:50:19 +02:00
Mike Fährmann
9e56d81292 fix Last-Modified mtime overwriting post processor mtime (#7529) 
https://github.com/mikf/gallery-dl/issues/7529#issuecomment-2989955455

- split '_mtime' into '_mtime_http' and '_mtime_meta'
- add PathFormat.set_mtime() method
 2025-06-20 16:19:37 +02:00
Mike Fährmann
41191bb60a 'match.group(N)' -> 'match[N]' (#7671) 
2.5x faster
 2025-06-18 13:05:58 +02:00
Mike Fährmann
fa1fc39a36 [pp:exec] implement 'commands' option 
to run multiple commands in succession
and stopping if one fails
 2025-06-13 20:19:01 +02:00
Mike Fährmann
e08ec7e083 update copyright notices 2025-06-13 00:03:41 +02:00
Mike Fährmann
811b665e33 remove @staticmethod decorators 
There might have been a time when calling a static method was faster
than a regular method, but that is no longer the case. According to
micro-benchmarks, it is 70% slower in CPython 3.13 and it also makes
executing the code of a class definition slower.
 2025-06-12 22:50:52 +02:00
Mike Fährmann
9b43a3677e [pp:directory] add 'directory' post processor (#7432) 2025-04-28 09:08:35 +02:00
Mike Fährmann
763bbfae8b [pp:ugoira] support '"keep-files": true' + custom extension (#7304) 
for '"mode": "archive"'
 2025-04-17 20:03:18 +02:00
Mike Fährmann
fc561c2936 [pp:ugoira] restore 'keep-files' functionality (#7304) 
fixes regression introduced in a08c35ad79
 2025-04-17 19:25:31 +02:00
Mike Fährmann
f3ababc3be [pp:ugoira] use '_ugoira_frame_index' to detect '.zip' files 2025-04-17 18:37:08 +02:00
Mike Fährmann
a08c35ad79 [pp:ugoira] preserve 'extension' when using "mode": "archive" (#7304) 2025-04-11 16:33:11 +02:00
Mike Fährmann
edd90fbf02 [pp:metadata] add 'metadata-path' option (#6582) 
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-2781643835
 2025-04-09 20:24:14 +02:00
Mike Fährmann
8eb365d4b6 [pp:metadata] fix handling of empty directory paths (#7296) 2025-04-08 17:03:27 +02:00
Mike Fährmann
fd4de02e67 [archive] support PostgreSQL archives for post processors (#6152) 2025-02-17 14:58:14 +01:00
Mike Fährmann
800cf5beb5 replace 'print()' with 'output.stderr_write("\n")' 2025-02-15 18:01:05 +01:00
Mike Fährmann
d8cf381904 [archive] use defaults when 'prefix'/'format' are 'null' 2024-11-29 16:36:35 +01:00
Mike Fährmann
9b2d782cb7 [pp:classify] rewrite & simplify (#5213) 
Do not manually build paths, which get later overwritten by
pathfmt.build_path() anyway. Just update the target directory and let
the rest of the "path logic" handle it.

Fixes skipping previously downloaded and categorized files,
which was broken since 8124c16a50
 2024-11-19 08:05:11 +01:00
Mike Fährmann
4b7a24122e [pp:ugoira] prevent BadZipFile exceptions (#6285) 
reset '_convert_…' flags  on every 'prepare' event
 2024-10-10 09:20:06 +02:00
Mike Fährmann
cc38042500 [pp:ugoira] forward frame data as '_ugoira_frame_data' (#6154, #6285) 2024-10-08 11:28:44 +02:00
Mike Fährmann
c0835a0cd1 [pp:ugoira] catch all exceptions when extracting zip archive (#6285) 2024-10-08 10:06:19 +02:00
Mike Fährmann
a051e1c955 directly pass exception instances as 'exc_info' logger argument 2024-09-19 14:50:08 +02:00
Mike Fährmann
319116c923 [pp:ugoira] update mtime of zipped files (#6147) 
in zip archives generated by '"mode": "archive"'
 2024-09-12 21:51:55 +02:00
Mike Fährmann
ff07aef776 [pp:ugoira] implement storing "original" frames in archives (#6147) 
… by using '"mode": "archive"'

- rename 'ffmpeg-demuxer' option to 'mode'
- add 'metadata' option
- add 'zip' as a possible `--ugoira` format

TODO: adjust file mtimes inside archives when 'mtime' is enabled
 2024-09-09 21:41:37 +02:00
Mike Fährmann
061cc12fdd [pp:ugoira] fix "KeyError: '_ugoira_frame_index'" (#6154) 
fixes regression introduced in 07bd967f

Files with a non-ugoira related 'frames' metadata field would cause an
exception when an ugoira post processor was active.
 2024-09-06 21:29:12 +02:00
Mike Fährmann
07bd967f59 [pp:ugoira] update (#6056) 
- introduce '_ugoira_frame_index' metadata field
- store Ugoira file exts separately
- add 'skip' option
 2024-09-05 20:25:20 +02:00
Mike Fährmann
8072dcf717 [pp:rename] recheck if file exists only when necessary 2024-09-05 17:42:29 +02:00
Mike Fährmann
359572162b [pp:rename] improve renaming files 'to' a format (#5846, #6044) 2024-09-03 21:17:31 +02:00
Mike Fährmann
57da9ebfb5 [pp:ugoira] initial support for "original" frames (#6056) 2024-09-01 21:32:00 +02:00
Mike Fährmann
ae9b0da755 [pp:hash] add 'hash' post processor (#6099) 2024-08-31 17:04:44 +02:00
Mike Fährmann
17f5ba43a8 [pp:rename] add 'rename' post processor (#5846, #6044) 
renames previously downloaded files to a different filename format
 2024-08-30 18:21:36 +02:00
Mike Fährmann
eef5beb841 [pp:metadata] fix using '..' in directories on Windows (#5942, #6094) 2024-08-27 14:38:40 +02:00
Mike Fährmann
4b94b7d477 [pp:metadata] add 'include' and 'exclude' options (#6058) 2024-08-19 21:58:57 +02:00
Mike Fährmann
605f6fba2c [pp:metadata] implement 'base-directory' option (#5262, #5728) 2024-08-07 10:15:06 +02:00
Mike Fährmann
1656c9287d [pp:metadata] fix 'directory' when file directory has conditions (#5728) 2024-07-06 22:07:17 +02:00