* [VSCO] Fix JSON returned by VSCO
The JSON returned contains multiple `undefined` values that aren't valid
JSON, changing these values to `null` fixes the issue.
* [VSCO] Make JSON fix more targeted
Change only occurrences that are values of a key-value
* [iwara] Add initial support
* [iwara] Add search support
* [iwara] Code cleanup
* [iwara] Small fixes and additions
* [iwara] Add tag support
* [iwara] Add mime-type to metadata
* [iwara] Refactor patterns/matching using urllib
* [iwara] Add unit tests
* [iwara] Update docs
* [iwara] Fix linting on older Python versions
* [iwara] update 'IwaraAPI' interface class
- define endpoints inside methods
- implement and use _call() and _pagination()
- cache auth tokens
* [iwara] split and rename 'profile' extractor
TODO:
- update test results
- simplify code
* [iwara] simplify '_user_params()' usage
* [iwara] update 'video' extractor
and move user data extraction into 'yield_video'
* [iwara] update 'image' extractor
and move user info extraction into 'yield_image()'
* [iwara] update 'playlist' extractor
* [iwara] update 'search' extractor
* [iwara] update 'tag' extractor
* [iwara] simplify 'yield_image' usage
perform API calls to get full 'files' list inside the function
* [iwara] add video "image" test
* [iwara] provide 'date' metadata
* [iwara] simplify 'source()'
remove urllib.parse usage
* [iwara] small optimizations
* get("key", {}) -> get("key") or {}
* split("…", 1) -> partition("…")
* use f-strings for all patterns
* [iwara] add missing 'keyarg=1' to profile() memcache decorator
* [tests/iwara] update results
* [iwara] extract more 'user' metadata
* [iwara] update default format strings
include 'date' in filenames to order them chronologically
* [iwara] restructure image/video handling
- use less generators
- make processing individual media items non-fatal
* [iwara] fix login and token handling
* [iwara] add 'favorite' extractor
* [iwara] add 'following' and 'followers' extractors
---------
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
- init.py:
- generate initial extractor module code and test result file
- insert new entries into modules list and site names
- generate_test_result.py:
- generate test result for a given URL
- insert it into the test result file generated by init.py
(or an already existing one)
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-2973285775
To stop gracefully after the current file finishes processing when
Ctrl+C was pressed, or after the current post finishes processing when
SIGUSR1 was received:
{
"signals-actions": {
"SIGINT" : "file",
"SIGUSR1": "post"
}
}
- add InputError and ControlException base classes
- add ChallengeError, LoginRequired, AbortExtraction
- change code of NotFoundError from 8 to 4
and NoExtractorError from 64 to 32
This makes it possible to specify 'mode' and/or 'event' options of a
postprocessor in its 'name' as
"NAME/MODE@EVENT" or "NAME/MODE" or "NAME@EVENT"
For example
"postprocessors": "metadata/jsonl@file,skip"
is equivalent to
"postprocessors": {
"name" : "metadata",
"mode" : "jsonl",
"event": ["file", "skip"]
}
allows reporting error codes for exceptions that are not handled
by the Job.run() try-except block
- fixes Job.status being 0 in certain situations even when errors occurred
- fixes some URLs not getting written to -e/--error-file (#7758)
