github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,788 Commits 1 Branch 0 Tags
df5c7ee03ef05ab73a034075102d6674d516236c
Commit Graph

602 Commits

Author SHA1 Message Date
Mike Fährmann
d50c312ff0 prevent test failure when there's no 'ytdl' module (#4364) 
split of ytdl into its own test function and
skip it when there's an ImportError similar to test_ytdl.py
 2023-07-29 13:48:31 +02:00
Mike Fährmann
48ef062867 fix issues with 'Extractor.finalize()' 
- prevent crash in InstagramUserExtractor (#4359)
- call it at the end of every DownloadJob
- add it to tests
 2023-07-29 13:43:27 +02:00
Mike Fährmann
255d08b79e add test for 'Extractor.initialize()' (#4359) 2023-07-28 16:58:16 +02:00
Mike Fährmann
a383eca7f6 decouple extractor initialization 
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
 2023-07-25 22:16:16 +02:00
Mike Fährmann
f0203b7559 [postprocessor:python] add tests 2023-07-24 15:22:57 +02:00
Mike Fährmann
d97b8c2fba consistent cookie-related names 
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
 2023-07-22 01:20:50 +02:00
Mike Fährmann
c5565f79f7 merge #4096: [danbooru] add support for booru.borvar.art instance 2023-07-18 18:33:08 +02:00
Mike Fährmann
63326e3168 [danbooru] add tests for booruvar 2023-07-18 18:29:57 +02:00
Mike Fährmann
5171d8975c [E621] support 'e6ai.net' (#4320) 2023-07-18 18:16:30 +02:00
Mike Fährmann
7444fc125b [gfycat] implement login support (#3770, #4271) 
For the record: '/webtoken' and '/weblogin' are not the same ...
 2023-07-06 18:56:34 +02:00
Mike Fährmann
25c5a6ffcb no f-strings 2023-06-25 14:01:26 +02:00
Mike Fährmann
ec64cbefeb [postprocessor:exec] add tests 2023-06-21 23:54:35 +02:00
Mike Fährmann
ce93c460a6 [formatter] implement 'H' conversion (#4164) 
to remove HTML tags and unescape HTML entities
 2023-06-15 13:07:51 +02:00
Mike Fährmann
deff3b434d [vipergirls] implement login support (#4166) 2023-06-13 21:05:09 +02:00
Mike Fährmann
69865dcc05 [formatter] implement slicing strings as bytes (#4087) 
prefixing a slice '[10:30]' with a lowercase b '[b10:30]' encodes
the string to bytes in filesystem encoding before applying the slice
 2023-05-22 18:30:45 +02:00
Mike Fährmann
df11214281 [ytdl] improve --xff/--geo-bypass detection (#3989) 
check if --xff is supported in a try-except block
and select expected results accordingly
 2023-05-01 18:26:37 +02:00
Mike Fährmann
aa731c4298 [ytdl] run yt-dlp tests with latest code from master (#3989) 
Only use PyPI version for Python 3.6, since that's no longer supported
by the current codebase.
 2023-05-01 16:42:57 +02:00
Mike Fährmann
43f4bd9faa [ytdl] fix tests 
tests pass with latest Git HEAD, but not with the current PyPI version
 2023-04-29 18:05:45 +02:00
Mike Fährmann
61a65d5bb9 [ytdl] fix crash due to --geo-bypass deprecation (#3975) 2023-04-29 17:25:38 +02:00
Mike Fährmann
a96745368e "fix" tests on Python 3.4 and 3.5 
can't rely on dict insertion order
 2023-04-26 19:31:27 +02:00
Mike Fährmann
3905f05f00 [postprocessor:metadata] support putting keys in quotes 
for mode 'modify' and 'delete'
based on fe41a2b1
 2023-04-25 14:30:18 +02:00
Mike Fährmann
7459e4abce [postprocessor:metadata] fix traversing more than 1 level deep 
for mode 'modify' and 'delete'
 2023-04-25 14:17:25 +02:00
Mike Fährmann
2edcdee32f [downloader:http] add MIME type and signature for .heic files 
(#3915)
https://github.com/strukturag/libheif/issues/83
 2023-04-15 17:09:22 +02:00
Mike Fährmann
082d55de16 fix circular reference detection for -K 2023-03-21 23:46:36 +01:00
Mike Fährmann
2ab66ad899 update -K output to include quotes around keys 2023-03-21 22:28:04 +01:00
Mike Fährmann
fe41a2b159 [formatter] support putting keys in quotes 
i.e. obj["key"] or obj['key']
as in f-strings
 2023-03-21 22:06:54 +01:00
Mike Fährmann
46fdf46f21 [formatter] support loading an f-string from a template file 
"\fTF ~/path/to/file.txt"
 2023-03-20 22:05:33 +01:00
Mike Fährmann
1a4d4a799b [formatter] support filesystem paths for \fM 2023-03-20 22:01:33 +01:00
Mike Fährmann
00f0233b28 [postprocessor:metadata] add 'skip' option (#3786) 2023-03-17 23:30:11 +01:00
Mike Fährmann
8f8b4de0e8 [ytdl] fix '--parse-metadata' (#3663) 2023-03-05 19:57:23 +01:00
Mike Fährmann
7610d9cf82 merge #3675: [pixiv] fix --write-tags for '"tags": "original"' 2023-03-02 21:48:31 +01:00
Mike Fährmann
83e7a25b6b extend OAuth tests 2023-03-02 17:26:51 +01:00
Mike Fährmann
d788e6c60c implement 'globals' option 2023-02-28 18:18:55 +01:00
Mike Fährmann
56039d2456 add 'hash_md5' and 'hash_sha1' functions (#3679) 
... to global eval namespace
 2023-02-22 10:58:44 +01:00
Mike Fährmann
e1df7f73b1 [deviantart] add 'search' extractor 
(#538, #1264, #2954, #2970, #3577)

Requires login to fetch any results, since the API endpoint raises an
error for not logged in requests.

TODO: parse HTML search results
 2023-02-20 20:54:46 +01:00
Gray Manley
38a6389e2c Fix lint. 2023-02-20 00:33:30 -06:00
Gray Manley
56cbae92ec Use more pythony naming. 2023-02-19 06:14:34 -06:00
Gray Manley
8e2ba4f32e Add test. 2023-02-19 06:13:21 -06:00
Mike Fährmann
dd884b02ee replace json.loads with direct calls to JSONDecoder.decode 2023-02-09 15:22:00 +01:00
Mike Fährmann
b7337d810e [postprocessor:metadata] add 'sort' and 'separators' options 2023-02-07 18:28:14 +01:00
Mike Fährmann
3436c6b117 [postprocessor:metadata] speed up JSON encoding 2023-02-06 12:35:28 +01:00
Mike Fährmann
925b467496 split e621 from danbooru module (#3425) 2023-02-03 19:24:31 +01:00
Mike Fährmann
c2bc70593e implement ability to load external extractor classes 
- -X/--extractors
- extractor.module-sources
 2023-01-30 23:10:10 +01:00
ClosedPort22
b6706b373a [downloader:http] add signature checks for some formats 
also add the MIME type for .obj files
 2023-01-15 23:40:55 +08:00
Mike Fährmann
71d3143c35 fix bug in test_extractors.py 
pattern matching tests would succeed
if there is exactly one match
but for the wrong extractor
 2023-01-08 15:35:05 +01:00
Mike Fährmann
fa144f38ed [ytdl} fix dfe4f00c for legacy yt-dlp 2023-01-04 21:42:22 +01:00
Mike Fährmann
dfe4f00ca2 [ytdl] update for yt-dlp changes 2023-01-04 13:12:24 +01:00
Mike Fährmann
d651d45239 implement specifying ranges in slice notation (#918, #2865) 
e.g.
- '1:101'   or ':101' or ':101:'  for files 1 to 100
- '1::2'    or '::2'              for every second file
- '1:101:5' or ':101:5'           for files 1, 6, 11, ..., 91, 96

(the second argument specifies the first index NOT included)
 2022-12-27 18:21:12 +01:00
Mike Fährmann
3616adfc75 implement '--range' with Python ranges 2022-12-26 18:32:34 +01:00
Mike Fährmann
1800bd7d14 allow '*-filter' options to be a list of expressions 2022-12-23 22:20:21 +01:00