Commit Graph

31 Commits

Author SHA1 Message Date
Mike Fährmann
b4a897aede [skeb] rename 'sent-requests' subcategory to 'sentrequests' (#8036)
to avoid clashes with 'sent-requests' option
2025-08-14 17:40:20 +02:00
Mike Fährmann
ab54f71511 [skeb] implement 'include' option (#6558 #7267)
- split 'user' extractor into 'works' & 'sent-requests' extractors
- use BASE_PATTERN & USER_PATTERN
- use self.groups
2025-08-12 22:11:03 +02:00
Mike Fährmann
a097a373a9 simplify if statements by using walrus operators (#7671) 2025-07-22 20:57:54 +02:00
Mike Fährmann
b77e18317b [skeb] fix "KeyError - 'frame_rate'" (#7798) 2025-07-09 12:40:37 +02:00
Mike Fährmann
f2a72d8d1e replace 'request(…).json()' with 'request_json(…)' 2025-06-29 17:50:19 +02:00
Mike Fährmann
9dbe33b6de replace old %-formatted and .format(…) strings with f-strings (#7671)
mostly using flynt
https://github.com/ikamensh/flynt
2025-06-29 17:50:19 +02:00
Mike Fährmann
41191bb60a 'match.group(N)' -> 'match[N]' (#7671)
2.5x faster
2025-06-18 13:05:58 +02:00
Mike Fährmann
93675b8e97 [skeb] make exceptions when extracting posts non-fatal (#7250) 2025-03-26 12:35:38 +01:00
Mike Fährmann
f0267f969a [skeb] prevent 429 errors and need for 'request_key' cookie
https://github.com/danbooru/danbooru/pull/5795
2024-09-26 15:15:37 +02:00
Mike Fährmann
11421cf940 [skeb] fix '429 Too Many Requests' errors (#5766)
Introduce '_handle_429' method to make it easier for Extractors to react
to 429 errors regardless of 'sleep-429' settings.
2024-06-21 00:12:05 +02:00
Mike Fährmann
6d9e3c0eb1 [skeb] add extractor for followed users (#5290)
needs 'Authorization' header from browser session
-o headers.Authorization="Bearer ey…"
2024-03-06 22:43:01 +01:00
Mike Fährmann
a2b55d5dde [skeb] retry 429 responses containing a 'request_key' cookie (#5210) 2024-02-24 00:54:15 +01:00
Mike Fährmann
4cfceb23cb [skeb] rename 'data' -> 'file' & add tests 2024-02-10 19:35:50 +01:00
blankie
f9a8e8cacf [skeb] add 'num' and 'count' metadata fields 2024-02-10 21:51:23 +11:00
Mike Fährmann
3ecb512722 send Referer headers by default 2023-09-19 00:02:04 +02:00
Mike Fährmann
a453335a9f remove test results in extractor modules
and add generic example URLs
2023-09-11 16:30:55 +02:00
Mike Fährmann
a383eca7f6 decouple extractor initialization
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
2023-07-25 22:16:16 +02:00
Mike Fährmann
f81dd5297a [skeb] fix extraction (#3112)
'completed_at' is no longer included in API responses
2022-10-29 15:07:21 +02:00
Mike Fährmann
ac45ed2764 [skeb] implement 'filters' option (#2945) 2022-09-26 15:17:28 +02:00
Mike Fährmann
68f11e02a9 [skeb] add 'search_tags' metadata to search results (#2945) 2022-09-23 13:56:00 +02:00
Mike Fährmann
4b2a006871 [skeb] add 'search' extractor (#2945) 2022-09-21 17:57:55 +02:00
Mike Fährmann
8672f8a2b9 [skeb] fix archive_ids for thumbnails and article images
8cf5981ded (commitcomment-82316040)
2022-08-27 16:46:53 +02:00
Mike Fährmann
70dc4ce911 [skeb] ignore article images with empty URL
8cf5981ded (commitcomment-81980633)
2022-08-26 17:57:17 +02:00
Mike Fährmann
8cf5981ded [skeb] add option to download 'article' images (#1031) 2022-07-29 16:32:00 +02:00
Mike Fährmann
c6a9bab019 update extractor test results 2022-07-12 15:49:22 +02:00
Mike Fährmann
f8cfc3b08a [skeb] add 'following' extractor (#2698) 2022-06-27 11:31:49 +02:00
Mike Fährmann
48cc4853be [skeb] refactor 'sent-requests' and add tests 2022-03-28 11:26:24 +02:00
Mike Fährmann
3ddc620ef6 [skeb] fix post extractor (#2330) 2022-03-09 18:45:07 +01:00
Alice
f1cab23724 [skeb] add 'sent-requests' option (#2322) (#2330)
* Update skeb.py

* Update configuration.rst

* flake8
2022-02-28 22:42:15 +01:00
Alice
612850438e [skeb] add 'thumbnails' option (#2047) (#2051) 2021-11-23 21:16:42 +01:00
Alice
bfd7401b1e [skeb] add 'user' and 'post' extractors (#1031) (#1971)
* Create skeb.py

* Update __init__.py

* Update supportedsites.py

* Update supportedsites.md

* Update supportedsites.py

* Update skeb.py
2021-10-26 20:00:41 +02:00