github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,743 Commits 1 Branch 0 Tags
e0f65be36b14a0b484867722b6887c7365d0ea2f
Commit Graph

44 Commits

Author SHA1 Message Date
Mike Fährmann
e08ec7e083 update copyright notices 2025-06-13 00:03:41 +02:00
Mike Fährmann
c567bf12e6 [danbooru][e621] remove constructors 2025-03-02 20:38:12 +01:00
Mike Fährmann
6e919a3695 [e621] support e621.cc and e621.anthro.fr frontend URLs (#6809) 2025-01-15 14:35:37 +01:00
Mike Fährmann
89276c5b3e [e621] match 'tag' search URLs with empty tag (#6783) 2025-01-07 20:00:26 +01:00
Mike Fährmann
5412b22dae [common] allow overriding more default 'User-Agent' headers (#6496) 
ignore 'extractor.user-agent' if it is the default useragent value
and an extractor wants to set its own custom value
 2024-11-26 21:50:28 +01:00
Mike Fährmann
eb5955bacc [e621] reorder 2024-08-12 18:01:29 +02:00
ClosedPort22
0f0ded53ae [e621] cache pool metadata results 2024-08-12 21:03:35 +08:00
Mike Fährmann
a453335a9f remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
5171d8975c [E621] support 'e6ai.net' (#4320) 2023-07-18 18:16:30 +02:00
Mike Fährmann
494acabd38 [danbooru] refactor pagination logic (#4002) 
- only use 'b<ID>' when no other order is specified
- support 'a<ID>' when  using 'order:id' as tag
 2023-05-14 18:39:59 +02:00
Mike Fährmann
708f478d15 [danbooru][e621] add 'date' metadata field (#4047) 2023-05-11 15:59:42 +02:00
Mike Fährmann
3d8777fbc1 move user agent string to util.py 2023-02-22 11:09:17 +01:00
Mike Fährmann
253ac08203 pre-define and use 'gallery-dö/<version>' UA string 2023-02-18 22:25:54 +01:00
Mike Fährmann
bbf0911a46 [e621] implement 'notes' and 'pools' metadata extraction 
(#3425)
 2023-02-04 21:20:38 +01:00
Mike Fährmann
925b467496 split e621 from danbooru module (#3425) 2023-02-03 19:24:31 +01:00
Mike Fährmann
563bd0ecf4 [danbooru] inherit from BaseExtractor 
- merge danbooru and e621 code
- support booru.allthefallen.moe (closes #2283)
- remove support for old e621 tag search URLs
 2022-02-11 21:01:51 +01:00
Mike Fährmann
02e18f56be [e621] add 'favorite' extractor (closes #2250) 2022-02-04 20:54:48 +01:00
Mike Fährmann
d781e6ac44 [e621] return pool posts in order (closes #1195) 
… and add a 'num' enumeration index.

A bit more code than the PR version, but it prints some helpful messages
and doesn't call 'metadata()' twice.
 2020-12-26 19:00:29 +01:00
Mike Fährmann
8ca7f54750 rename '_request_…' variables 
- remove '_' at the beginning
- _request_last -> request_timestamp
 2020-12-05 00:09:15 +01:00
Mike Fährmann
c57a918f4a [e621] implement delay via '_request_interval_min' 2020-11-25 00:19:32 +01:00
Mike Fährmann
ad2efa8509 [e621] derive from Danbooru extractors (#651) 
- use extractor implementations from 'danbooru'
- use "page": "b[ID]" to paginate over results instead of
  "tags": "id:<[ID]", avoiding infinite loops with certain
  post orders
- bump User-Agent version
 2020-03-22 21:08:45 +01:00
Mike Fährmann
9b39e1cd7e [e621] fix bug in API rate limiting (#651) 2020-03-22 14:01:23 +01:00
Mike Fährmann
5bcc7184c9 [danbooru][e621] increase page limits 2020-03-17 15:53:28 +01:00
Mike Fährmann
ebc70e87ce [e621] update to new interface / API endpoints (closes #635) 2020-03-06 21:12:58 +01:00
Mike Fährmann
6284731107 simplify extractor constants 
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
 2019-02-08 13:45:40 +01:00
Mike Fährmann
34bab080ae rewrite URL patterns to use only 1 per extractor 2019-02-08 12:03:10 +01:00
Mike Fährmann
4a57509392 generalize tag-splitting option (#92) 
- extend functionality to other booru sites:
  - http://behoimi.org/
  - https://konachan.com/
  - https://e621.net/
  - https://rule34.xxx/
  - https://safebooru.org/
  - https://yande.re/
 2018-07-04 12:21:16 +02:00
Mike Fährmann
974e73bdbb [booru] smaller code adjustments 2018-01-06 17:48:49 +01:00
Mike Fährmann
9e8a84ab6c [booru] rewrite using Mixin classes (#59) 
- improved code structure
- improved URL patterns
- better pagination to work around page limits on
  - Danbooru
  - e621
  - 3dbooru
 2018-01-04 00:01:39 +01:00
Mike Fährmann
65997d835b replace popular/ranking tests with older ones 
Metadata of several year old lists shouldn't change as much as it
would for newer ones, which makes metadata-comparisons of the output
of build_testresult_db.oy easier.
 2017-08-31 15:09:18 +02:00
Mike Fährmann
88a386977e [booru] add "popular" extractors for more sites 
- konachan.com
- behoimi.org
- e621.net
 2017-08-26 23:08:52 +02:00
Mike Fährmann
07214f4007 [booru] place subcategories into base classes 2017-08-26 22:27:55 +02:00
Mike Fährmann
94e10f249a code adjustments according to pep8 nr2 2017-02-01 00:53:19 +01:00
Mike Fährmann
d7e168799d consistent extractor naming scheme + docstrings 2016-09-12 10:34:31 +02:00
Mike Fährmann
616e0aedd6 update booru testdata 2015-12-22 03:10:52 +01:00
Mike Fährmann
ba99506c72 more extractor test-cases 2015-12-14 03:00:58 +01:00
Mike Fährmann
f7c47a6018 add subcategories to extractors 2015-11-30 01:11:13 +01:00
Mike Fährmann
23208a49b8 [e621] update to new format 2015-11-21 01:56:49 +01:00
Mike Fährmann
3b0fe8f544 unify booru filename-patterns 2015-11-06 16:48:33 +01:00
Mike Fährmann
3c13548f29 rewrite extractors to use config-module 2015-10-05 15:51:08 +02:00
Mike Fährmann
3c6161eea7 small whitespace fixes 2015-04-11 16:22:15 +02:00
Mike Fährmann
758fe00441 [e621] update to new extractor interface 2015-04-11 00:39:26 +02:00
Mike Fährmann
34d13bc906 added extractor 'danbooru' + split BooruExtractor to handle XML and JSON 2014-12-30 21:34:55 +01:00
Mike Fährmann
099fe7515e added extractor 'e621' 2014-12-22 16:23:41 +01:00