github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,825 Commits 1 Branch 0 Tags
04442e262eedcbfd737bf7778a44cb66e00d2ddf
Commit Graph

57 Commits

Author SHA1 Message Date
Mike Fährmann
9fd732afe8 [config] support setting default file type via GDL_CONFIG_TYPE 2026-01-31 17:55:17 +01:00
brerk
e00c717b15 [turbo] update 'saint' extractors (#8893 #8896) 
* Implements turbo.py & remove from domain pattern from saints.py
* Remove leftover commented pattern from saints.py
* Make turbo.py comply with flake8
* Add album support
* Improved metadata extracion for albums and single files & created turbo.py tests using saints.py test
* Align turbo.py extractor with flake8 rules
* Fix #class name on turbo.py tests
* Fix #category test
* Fix #category test x2
* Fix #category tests
* Fix #category tests
* Fix TurboMediaExtractor self.groups unpacking

* update basic module formatting
* replace 'saint' with 'turbo' in modules list
* remove saint extractors and tests
* update & simplify 'media' extractor
* update & simplify 'album' extractor
* update tests
* update supportedsites
* update 'category-map' & 'config-map'

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2026-01-19 22:20:13 +01:00
Mike Fährmann
ed3ac5a402 [sizebooru] add support (#7667) 2025-08-24 10:39:17 +02:00
djsigmann
04fa29fa5a [config] add 'conf' argument to 'config.load()' (#8084) 
to allow using an alternative config dict
 2025-08-21 18:23:35 +02:00
Mike Fährmann
a097a373a9 simplify if statements by using walrus operators (#7671) 2025-07-22 20:57:54 +02:00
Mike Fährmann
82891b4d0c [pixiv] move 'novel' extractors to a 'pixiv-novel' category (#7746) 
TODO:
- restore full 'include' functionality
- allow remapping category:subcategory pairs
 2025-07-04 20:13:19 +02:00
Mike Fährmann
e7922ababd [naver] change categories (#7746) 
- 'naver'        -> 'naver-blog'
- 'chzzk'        -> 'naver-chzzk'
- 'naverwebtoon' -> 'naver-webtoon'
 2025-07-02 23:20:40 +02:00
Mike Fährmann
9dbe33b6de replace old %-formatted and .format(…) strings with f-strings (#7671) 
mostly using flynt
https://github.com/ikamensh/flynt
 2025-06-29 17:50:19 +02:00
Mike Fährmann
e08ec7e083 update copyright notices 2025-06-13 00:03:41 +02:00
Mike Fährmann
99bf92364a rename 'category-map' to 'config-map' (#7612) 2025-06-04 07:57:27 +02:00
Mike Fährmann
783f1c1b11 add 'category-map' option 
primarily to achieve some form of backwards compatibility for
922c296482
 2025-05-30 17:55:49 +02:00
Mike Fährmann
bf927cbd4f [config] fix using same key multiple times with 'apply' (#7127) 2025-03-16 19:37:04 +01:00
Mike Fährmann
80454460ce [config] support accumulating non-list values 
fixes 1264fc518b
 2024-11-16 21:13:57 +01:00
Mike Fährmann
1df09292b5 consistent __exit__() argument names 2024-10-11 16:43:28 +02:00
Mike Fährmann
9c65db2a92 consistent 'with open(…) as fp:' syntax 2024-06-14 01:22:00 +02:00
Mike Fährmann
3fc2e61818 add '--config-open' and '--config-status' (#5713) 
adapted from 119a25a59cd07781844136ed3d360e7d820b112c
 2024-06-13 23:22:56 +02:00
Mike Fährmann
92f98e6f5e 'sys.exit' -> 'SystemExit' 2023-08-21 23:46:39 +02:00
Mike Fährmann
410f783a33 implement 'subconfigs' option (#4440) 2023-08-21 21:18:40 +02:00
Mike Fährmann
076380e079 remove '*' indicating keyword-only arguments 
they are kind of unnecessary and
cause a non-insignificant function call overhead (~10%)
 2023-05-02 22:23:33 +02:00
Mike Fährmann
57cf942bb1 [config] include exception type in error message 2023-05-02 18:36:13 +02:00
Mike Fährmann
075c965512 add '--config-create' command-line option 
(#2333)
 2023-03-01 14:49:40 +01:00
Mike Fährmann
26d06e0bb2 move executable check into util.py 2023-02-28 23:10:23 +01:00
Mike Fährmann
de2f35d068 simplify config.load() 2023-02-28 22:02:15 +01:00
Mike Fährmann
632d5d7745 allow loading config files in TOML format with --config-toml 2023-02-28 21:54:46 +01:00
Mike Fährmann
dd884b02ee replace json.loads with direct calls to JSONDecoder.decode 2023-02-09 15:22:00 +01:00
Mike Fährmann
46d811bac0 add loaded config files to debug output 2022-11-18 17:15:32 +01:00
Mike Fährmann
3a7c3ff138 support XDG_CONFIG_HOME (closes #1545) 
This will only load either
${XDG_CONFIG_HOME}/gallery-dl/config.json or
${HOME}/.config/gallery-dl/config.json
if XDG_CONFIG_HOME is empty, never both.
 2021-05-16 02:48:16 +02:00
Mike Fährmann
392d022b04 implement 'config.accumulate()' (#994) 2020-09-14 21:13:08 +02:00
Mike Fährmann
53cc498d9c improve config lookup when there are multiple possible locations 
This specifically applies to all Mastodon extractors and all
extractors with a 'basecategory', i.e. 'booru', 'foolslide', etc.

Values inside those general config locations wouldn't be recognized
when a value with the same was set on the 'extractor' level.

For example 'extractor.mastodon.directory' should be used over
'extractor.directory' when both are set, but this was impossible
with the previous implementation.

(fixes #843)
 2020-06-21 00:07:10 +02:00
Mike Fährmann
da22ea8ced use %APPDATA%\gallery-dl for config/cache on Windows 2020-05-28 02:56:38 +02:00
Mike Fährmann
3201fe3521 add global SENTINEL object 2020-05-19 22:32:53 +02:00
Mike Fährmann
c8787647ed add global WINDOWS bool 2020-05-19 22:32:53 +02:00
Mike Fährmann
300264f676 read config files from PyInstaller exe directory (closes #682) 2020-04-08 21:53:50 +02:00
Mike Fährmann
c4702ec9b6 simplify some logging calls 2019-12-10 21:30:08 +01:00
Mike Fährmann
b5c964332b improve config.py test coverage 2019-11-25 17:20:00 +01:00
Mike Fährmann
f5604492c3 update interface of config functions 2019-11-24 00:42:28 +01:00
Mike Fährmann
1578013efc remove unused default config path 2019-03-04 20:53:58 +01:00
Mike Fährmann
176b7253a1 update function signature for config.load() 2019-03-01 14:13:34 +01:00
Mike Fährmann
513d807632 explicitly open config files as utf-8 2018-05-25 16:29:46 +02:00
Mike Fährmann
b50bdbf3d7 change config specifiers in input file format 
Instead of a dictionary/object, input file options are now specified
by a 'key=value' pair starting with '-' for options only applying to
the next URL or '-G' for Global options applying to all following URLs.

See the docstring of parse_inputfile() for details.

Example option specifiers:

- filename = "{id}.{extension}"
- extractor.pixiv.user.directory = ["Pixiv Users", "{user[id]}"]
-spaces="are_optional"
-G keywords = {"global": "option"}
 2018-02-16 03:10:41 +01:00
Mike Fährmann
057668e17e extend input-file format with per-URL config and comments 
- see docstring of parse_inputfile() for details
- TODO: unittests, recursion (currently setting for example
  {"extractor": {"key": "value"}} will override the whole "extractor"
  branch instead of merging {"key": "value"} into the already existing
  dictionary)
 2018-02-07 21:47:27 +01:00
Mike Fährmann
ea8ca4cfa4 add 'util.expand_path()' 2017-10-26 00:04:28 +02:00
Mike Fährmann
b319f4bab3 smaller code and text changes 2017-10-01 18:23:40 +02:00
Mike Fährmann
004456d5d5 properly update the config-dictionary 
When using 2 or more config files, the values of the second would
improperly overwrite nested dictionaries of the first one.
The new method properly combines these nested dictionaries as well.
 2017-08-12 20:07:27 +02:00
Mike Fährmann
e3212dd98f fix some smaller stuff 
- remove support for old windows config paths
- catch exception if cache-database can't be opened
- fix username/password settings for unit tests
- rename variable 'max_tries' to 'retries'
 2017-03-27 14:30:32 +02:00
Mike Fährmann
e4b3077168 improve config module 
- speed improvements, especially in the 'interpolate' function
- 'interpolate' now prioritizes base-level values if they exist
  - "username" is chosen before "extractor.<category>.username"
  - -u/--username & co can now override config-file values
 2017-03-27 11:59:27 +02:00
Mike Fährmann
27ae152f57 use logging to report errors 2017-03-11 01:47:57 +01:00
Mike Fährmann
0cfe51dc78 add '--config-yaml' option 
(#8)
 2017-03-08 16:57:42 +01:00
Mike Fährmann
4f123b8513 code adjustments according to pep8 2017-01-30 19:40:15 +01:00
Mike Fährmann
7c8fae62f6 update default config file paths 2016-11-10 16:24:08 +01:00