github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,304 Commits 1 Branch 0 Tags
a050fe0d94de7e9caac0216b53fa682d9b45b8a9
Commit Graph

59 Commits

Author SHA1 Message Date
Mike Fährmann
c87bd1a752 [danbooru] extend 'metadata' option 
make it possible to specify a custom list of metadata includes
 2023-01-13 16:20:01 +01:00
blankie
f82ee93676 [danbooru] extract uploader metadata (if metadata is set) 2022-12-25 11:00:59 +07:00
ClosedPort22
dd4a4a3fa6 [e621] softcode the pagination threshold 2022-12-17 21:06:47 +08:00
ClosedPort22
9faa4ed738 [e621] refactor pagination control 
as suggested by @mikf
 2022-12-17 20:05:34 +08:00
ClosedPort22
d0ad6d0e67 [e621] implement manual pagination mode 2022-12-16 16:15:36 +08:00
Mike Fährmann
e99ce99284 [danbooru] remove stray 'print()' 2022-11-25 18:27:45 +01:00
Mike Fährmann
4e26bf98f5 [aibooru] support 'safe' subdomain (#3110) 2022-10-28 12:10:18 +02:00
thatfuckingbird
062ef238a6 add support for aibooru (using danbooru extractor) (#3075) 2022-10-19 11:53:59 +02:00
Mike Fährmann
d0d4ce1a13 [danbooru] fix ugoira metadata extraction (#3056) 2022-10-15 13:29:09 +02:00
Mike Fährmann
f362d4a3c7 [e621] fix 'popular' extraction 2022-08-26 17:57:17 +02:00
Mike Fährmann
c6a9bab019 update extractor test results 2022-07-12 15:49:22 +02:00
Mike Fährmann
3f02e483c6 [e621] fix applying request_interval_min (#2533) 
Setting this property after calling Extractor.__init__() has no effect.
 2022-04-27 21:10:34 +02:00
Mike Fährmann
563bd0ecf4 [danbooru] inherit from BaseExtractor 
- merge danbooru and e621 code
- support booru.allthefallen.moe (closes #2283)
- remove support for old e621 tag search URLs
 2022-02-11 21:01:51 +01:00
Mike Fährmann
cadfad4eea [danbooru] add 'external' option (closes #1747) 2021-08-08 19:18:41 +02:00
Mike Fährmann
221015e586 [downloader:http] disable filename extension changes for ugoira 
(#1507)
 2021-04-27 01:29:09 +02:00
thatfuckingbird
224b883ff4 [danbooru] add option for extended metadata extraction (#1458) 
* [danbooru] add option for extended metadata extraction

* appease linter

* [danbooru] update docs/configuration.rst

* [danbooru] rename extended-metadata -> metadata
 2021-04-13 23:41:30 +02:00
Mike Fährmann
d781e6ac44 [e621] return pool posts in order (closes #1195) 
… and add a 'num' enumeration index.

A bit more code than the PR version, but it prints some helpful messages
and doesn't call 'metadata()' twice.
 2020-12-26 19:00:29 +01:00
Mike Fährmann
e7d446a8f7 [danbooru] slight code refactoring 2020-12-25 22:06:25 +01:00
Mike Fährmann
1e3dd7330e merge SharedConfigMixin functionality into Extractor 2020-11-17 00:34:07 +01:00
Mike Fährmann
2626629117 [danbooru] handle posts without 'id' (fixes #1004) 2020-09-16 21:35:27 +02:00
Mike Fährmann
e19f665a44 [danbooru] change default for 'ugoira' to 'false' 
Downloading the pre-rendered versions should be a better default
than .zip files with individual frames.
 2020-05-20 19:57:28 +02:00
Mike Fährmann
ad2efa8509 [e621] derive from Danbooru extractors (#651) 
- use extractor implementations from 'danbooru'
- use "page": "b[ID]" to paginate over results instead of
  "tags": "id:<[ID]", avoiding infinite loops with certain
  post orders
- bump User-Agent version
 2020-03-22 21:08:45 +01:00
Mike Fährmann
5bcc7184c9 [danbooru][e621] increase page limits 2020-03-17 15:53:28 +01:00
Mike Fährmann
f117e32910 [danbooru] restore 'popular' functionality 2020-02-29 23:37:53 +01:00
Mike Fährmann
86c00f9e66 [danbooru] move extractor logic from booru.py 2020-02-28 22:53:45 +01:00
Mike Fährmann
f02a768b5c [danbooru] add 'ugoira' option (#406) 
to choose between ZIP archives or converted video files
for Ugoira posts
 2019-08-31 21:51:52 +02:00
Mike Fährmann
6284731107 simplify extractor constants 
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
 2019-02-08 13:45:40 +01:00
Mike Fährmann
1e4d351ad3 [danbooru] add authentication support (closes #151) 
... via HTTP Basic Auth with username and "password".

The password value in this case is not the account password itself,
but the"api_key" found in your user profile.
 2019-01-09 14:19:07 +01:00
Mike Fährmann
e1d306cc48 update unit test results 2018-10-13 16:54:30 +02:00
Mike Fährmann
6ac403c5d3 add postprocessor config example 2018-06-08 18:31:59 +02:00
Mike Fährmann
3905474805 [booru] call update_page() with correct dict (closes #82) 2018-03-19 11:33:19 +01:00
Mike Fährmann
974e73bdbb [booru] smaller code adjustments 2018-01-06 17:48:49 +01:00
Mike Fährmann
1219ebb7f5 [danbooru] use alternate subdomains; support safebooru 2018-01-04 00:51:04 +01:00
Mike Fährmann
9e8a84ab6c [booru] rewrite using Mixin classes (#59) 
- improved code structure
- improved URL patterns
- better pagination to work around page limits on
  - Danbooru
  - e621
  - 3dbooru
 2018-01-04 00:01:39 +01:00
Mike Fährmann
289d6b65d2 [danbooru] extend and improve URL regex 
- add support for danbooru mirrors:
  - hijiribe.donmai.us
  - sonohara.donmai.us
  - todo: actually use these domains instead of redirecting everything
          to danbooru itself
- improve handling of query string parameters
 2017-10-16 21:21:19 +02:00
Mike Fährmann
65997d835b replace popular/ranking tests with older ones 
Metadata of several year old lists shouldn't change as much as it
would for newer ones, which makes metadata-comparisons of the output
of build_testresult_db.oy easier.
 2017-08-31 15:09:18 +02:00
Mike Fährmann
07214f4007 [booru] place subcategories into base classes 2017-08-26 22:27:55 +02:00
Mike Fährmann
47bcf53ec1 implement support for additional unit test result types 
- "pattern" matches all resulting URLs against the given regex
- "count" allows to specify the amount of returned URLs
 2017-08-25 22:01:14 +02:00
Mike Fährmann
18e6ed1c7e [booru] add extractors for "Popular" images 2017-08-24 21:29:22 +02:00
Mike Fährmann
c921b4f32a code cleanup and fixing tests 2017-06-02 09:10:58 +02:00
Mike Fährmann
cf79a47b59 update unit tests 2017-04-08 11:02:32 +02:00
Mike Fährmann
ed94d9b92d fix/improve various things 2017-03-17 09:39:46 +01:00
Mike Fährmann
94e10f249a code adjustments according to pep8 nr2 2017-02-01 00:53:19 +01:00
Mike Fährmann
d7e168799d consistent extractor naming scheme + docstrings 2016-09-12 10:34:31 +02:00
Mike Fährmann
6f7d42b974 update tests 2016-07-12 12:08:36 +02:00
Mike Fährmann
616e0aedd6 update booru testdata 2015-12-22 03:10:52 +01:00
Mike Fährmann
ba99506c72 more extractor test-cases 2015-12-14 03:00:58 +01:00
Mike Fährmann
2449532132 add more extractor test-cases 2015-12-13 04:36:44 +01:00
Mike Fährmann
c8fdfbec42 [danbooru] add test cases 2015-12-12 15:59:26 +01:00
Mike Fährmann
f7c47a6018 add subcategories to extractors 2015-11-30 01:11:13 +01:00