github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,889 Commits 1 Branch 0 Tags
5548cff730ed50436271e7057371b5a5d74161fc
Commit Graph

59 Commits

Author SHA1 Message Date
Mike Fährmann
a28fbbc3c6 [util] rename 'USERAGENT' to 'USERAGENT_GALLERYDL' 2026-01-03 10:31:48 +01:00
Mike Fährmann
d0f06be0d2 use 'operator +' when building 'pattern' 2025-12-20 22:07:44 +01:00
Mike Fährmann
e006d26c8e Revert "use f-strings when building 'pattern'" 
revert d7c97d5a97.
 2025-12-20 22:07:37 +01:00
Mike Fährmann
968597a302 yield 3-tuples for Message.Directory 
adapt tuples to the same length and semantics as other messages
 2025-12-05 21:39:52 +01:00
Mike Fährmann
62e9d75611 [common] fix 'AttributeError' 
fixes regression introduced in 98d3354575
 2025-10-29 11:38:26 +01:00
Mike Fährmann
98d3354575 [wikimedia] implement config lookup for fandom/wikigg sites (#7283) 
{
    "extractor": {
        "fandom": {
            "filename": "..."
        }
    }
}
 2025-10-23 20:14:56 +02:00
Mike Fährmann
df1b6204c6 [e621] add 'artist' & 'artist-search' extractors (#8448) 2025-10-22 10:33:52 +02:00
Mike Fährmann
d7c97d5a97 use f-strings when building 'pattern' 2025-10-20 21:23:11 +02:00
Mike Fährmann
c38856bd3f [dt] use 'parse_datetime_iso()' for ISO formats 2025-10-19 21:52:05 +02:00
Mike Fährmann
085616e0a8 [dt] replace 'text.parse_datetime()' & 'text.parse_timestamp()' 2025-10-17 17:43:06 +02:00
Mike Fährmann
a097a373a9 simplify if statements by using walrus operators (#7671) 2025-07-22 20:57:54 +02:00
Mike Fährmann
1aa8fe7c10 [e621] support 'e621.cc/posts' URLs (#6809) 
https://github.com/mikf/gallery-dl/issues/6809#issuecomment-3043180010
 2025-07-07 08:44:31 +02:00
Mike Fährmann
3810555bbd do not use 'append = list.append' 2025-06-30 11:42:44 +02:00
Mike Fährmann
f2a72d8d1e replace 'request(…).json()' with 'request_json(…)' 2025-06-29 17:50:19 +02:00
Mike Fährmann
9dbe33b6de replace old %-formatted and .format(…) strings with f-strings (#7671) 
mostly using flynt
https://github.com/ikamensh/flynt
 2025-06-29 17:50:19 +02:00
Mike Fährmann
e08ec7e083 update copyright notices 2025-06-13 00:03:41 +02:00
Mike Fährmann
c567bf12e6 [danbooru][e621] remove constructors 2025-03-02 20:38:12 +01:00
Mike Fährmann
6e919a3695 [e621] support e621.cc and e621.anthro.fr frontend URLs (#6809) 2025-01-15 14:35:37 +01:00
Mike Fährmann
89276c5b3e [e621] match 'tag' search URLs with empty tag (#6783) 2025-01-07 20:00:26 +01:00
Mike Fährmann
5412b22dae [common] allow overriding more default 'User-Agent' headers (#6496) 
ignore 'extractor.user-agent' if it is the default useragent value
and an extractor wants to set its own custom value
 2024-11-26 21:50:28 +01:00
Mike Fährmann
eb5955bacc [e621] reorder 2024-08-12 18:01:29 +02:00
ClosedPort22
0f0ded53ae [e621] cache pool metadata results 2024-08-12 21:03:35 +08:00
Mike Fährmann
a453335a9f remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
5171d8975c [E621] support 'e6ai.net' (#4320) 2023-07-18 18:16:30 +02:00
Mike Fährmann
494acabd38 [danbooru] refactor pagination logic (#4002) 
- only use 'b<ID>' when no other order is specified
- support 'a<ID>' when  using 'order:id' as tag
 2023-05-14 18:39:59 +02:00
Mike Fährmann
708f478d15 [danbooru][e621] add 'date' metadata field (#4047) 2023-05-11 15:59:42 +02:00
Mike Fährmann
3d8777fbc1 move user agent string to util.py 2023-02-22 11:09:17 +01:00
Mike Fährmann
253ac08203 pre-define and use 'gallery-dö/<version>' UA string 2023-02-18 22:25:54 +01:00
Mike Fährmann
bbf0911a46 [e621] implement 'notes' and 'pools' metadata extraction 
(#3425)
 2023-02-04 21:20:38 +01:00
Mike Fährmann
925b467496 split e621 from danbooru module (#3425) 2023-02-03 19:24:31 +01:00
Mike Fährmann
563bd0ecf4 [danbooru] inherit from BaseExtractor 
- merge danbooru and e621 code
- support booru.allthefallen.moe (closes #2283)
- remove support for old e621 tag search URLs
 2022-02-11 21:01:51 +01:00
Mike Fährmann
02e18f56be [e621] add 'favorite' extractor (closes #2250) 2022-02-04 20:54:48 +01:00
Mike Fährmann
d781e6ac44 [e621] return pool posts in order (closes #1195) 
… and add a 'num' enumeration index.

A bit more code than the PR version, but it prints some helpful messages
and doesn't call 'metadata()' twice.
 2020-12-26 19:00:29 +01:00
Mike Fährmann
8ca7f54750 rename '_request_…' variables 
- remove '_' at the beginning
- _request_last -> request_timestamp
 2020-12-05 00:09:15 +01:00
Mike Fährmann
c57a918f4a [e621] implement delay via '_request_interval_min' 2020-11-25 00:19:32 +01:00
Mike Fährmann
ad2efa8509 [e621] derive from Danbooru extractors (#651) 
- use extractor implementations from 'danbooru'
- use "page": "b[ID]" to paginate over results instead of
  "tags": "id:<[ID]", avoiding infinite loops with certain
  post orders
- bump User-Agent version
 2020-03-22 21:08:45 +01:00
Mike Fährmann
9b39e1cd7e [e621] fix bug in API rate limiting (#651) 2020-03-22 14:01:23 +01:00
Mike Fährmann
5bcc7184c9 [danbooru][e621] increase page limits 2020-03-17 15:53:28 +01:00
Mike Fährmann
ebc70e87ce [e621] update to new interface / API endpoints (closes #635) 2020-03-06 21:12:58 +01:00
Mike Fährmann
6284731107 simplify extractor constants 
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
 2019-02-08 13:45:40 +01:00
Mike Fährmann
34bab080ae rewrite URL patterns to use only 1 per extractor 2019-02-08 12:03:10 +01:00
Mike Fährmann
4a57509392 generalize tag-splitting option (#92) 
- extend functionality to other booru sites:
  - http://behoimi.org/
  - https://konachan.com/
  - https://e621.net/
  - https://rule34.xxx/
  - https://safebooru.org/
  - https://yande.re/
 2018-07-04 12:21:16 +02:00
Mike Fährmann
974e73bdbb [booru] smaller code adjustments 2018-01-06 17:48:49 +01:00
Mike Fährmann
9e8a84ab6c [booru] rewrite using Mixin classes (#59) 
- improved code structure
- improved URL patterns
- better pagination to work around page limits on
  - Danbooru
  - e621
  - 3dbooru
 2018-01-04 00:01:39 +01:00
Mike Fährmann
65997d835b replace popular/ranking tests with older ones 
Metadata of several year old lists shouldn't change as much as it
would for newer ones, which makes metadata-comparisons of the output
of build_testresult_db.oy easier.
 2017-08-31 15:09:18 +02:00
Mike Fährmann
88a386977e [booru] add "popular" extractors for more sites 
- konachan.com
- behoimi.org
- e621.net
 2017-08-26 23:08:52 +02:00
Mike Fährmann
07214f4007 [booru] place subcategories into base classes 2017-08-26 22:27:55 +02:00
Mike Fährmann
94e10f249a code adjustments according to pep8 nr2 2017-02-01 00:53:19 +01:00
Mike Fährmann
d7e168799d consistent extractor naming scheme + docstrings 2016-09-12 10:34:31 +02:00
Mike Fährmann
616e0aedd6 update booru testdata 2015-12-22 03:10:52 +01:00