github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[kemono][coomer][schalenetwork] rename modules & extractors

Browse Source 
category changes:

- kemonoparty -> kemono
- coomerparty -> coomer
- koharu      -> schalenetwork

also wanted to rename '2chan' -> 'sturdychan',
but the site's main page is still titled '2chen'
This commit is contained in:
Mike Fährmann
2025-05-30 17:12:45 +02:00
parent 2a5eec953e
commit 922c296482
10 changed files with 211 additions and 223 deletions
Download Patch File Download Diff File Expand all files Collapse all files

70
docs/configuration.rst
View File

@@ -390,13 +390,13 @@ Default
``[E621]``,
``[foolfuuka]:search``,
``itaku``,
``koharu``,
``newgrounds``,
``[philomena]``,
``pixiv:novel``,
``plurk``,
``poipiku`` ,
``pornpics``,
``schalenetwork``,
``scrolller``,
``soundgasm``,
``urlgalleries``,
@@ -453,7 +453,7 @@ Description
* ``atfbooru`` (*)
* ``bluesky``
* ``booruvar`` (*)
* ``coomerparty``
* ``coomer``
* ``danbooru`` (*)
* ``deviantart``
* ``e621`` (*)
@@ -464,14 +464,14 @@ Description
* ``idolcomplex``
* ``imgbb``
* ``inkbunny``
* ``kemonoparty``
* ``koharu``
* ``kemono``
* ``mangadex``
* ``mangoxo``
* ``newgrounds``
* ``nijie`` (R)
* ``pillowfort``
* ``sankaku``
* ``schalenetwork``
* ``scrolller``
* ``seiga``
* ``subscribestar``
@@ -3106,8 +3106,8 @@ Description
Download video files.
extractor.kemonoparty.archives
------------------------------
extractor.kemono.archives
-------------------------
Type
``bool``
Default
@@ -3119,8 +3119,8 @@ Description
Note: This requires 1 additional HTTP request per ``archives`` file.
extractor.kemonoparty.comments
------------------------------
extractor.kemono.comments
-------------------------
Type
``bool``
Default
@@ -3131,8 +3131,8 @@ Description
Note: This requires 1 additional HTTP request per post.
extractor.kemonoparty.duplicates
--------------------------------
extractor.kemono.duplicates
---------------------------
Type
``bool``
Default
@@ -3144,8 +3144,8 @@ Description
* ``false``: Ignore duplicates
extractor.kemonoparty.dms
-------------------------
extractor.kemono.dms
--------------------
Type
``bool``
Default
@@ -3154,8 +3154,8 @@ Description
Extract a user's direct messages as ``dms`` metadata.
extractor.kemonoparty.announcements
-----------------------------------
extractor.kemono.announcements
------------------------------
Type
``bool``
Default
@@ -3164,8 +3164,8 @@ Description
Extract a user's announcements as ``announcements`` metadata.
extractor.kemonoparty.endpoint
------------------------------
extractor.kemono.endpoint
-------------------------
Type
``string``
Default
@@ -3191,8 +3191,8 @@ Description
| Provides more metadata, but might not return a creator's first/last posts.
extractor.kemonoparty.favorites
-------------------------------
extractor.kemono.favorites
--------------------------
Type
``string``
Default
@@ -3203,8 +3203,8 @@ Description
Available types are ``artist``, and ``post``.
extractor.kemonoparty.files
---------------------------
extractor.kemono.files
----------------------
Type
``list`` of ``strings``
Default
@@ -3215,8 +3215,8 @@ Description
Available types are ``file``, ``attachments``, and ``inline``.
extractor.kemonoparty.max-posts
-------------------------------
extractor.kemono.max-posts
--------------------------
Type
``integer``
Default
@@ -3225,8 +3225,8 @@ Description
Limit the number of posts to download.
extractor.kemonoparty.metadata
------------------------------
extractor.kemono.metadata
-------------------------
Type
``bool``
Default
@@ -3235,8 +3235,8 @@ Description
Extract ``username`` and ``user_profile`` metadata.
extractor.kemonoparty.revisions
-------------------------------
extractor.kemono.revisions
--------------------------
Type
* ``bool``
* ``string``
@@ -3250,15 +3250,15 @@ Description
Note: This requires 1 additional HTTP request per post.
extractor.kemonoparty.order-revisions
-------------------------------------
extractor.kemono.order-revisions
--------------------------------
Type
``string``
Default
``"desc"``
Description
Controls the order in which
`revisions <extractor.kemonoparty.revisions_>`__
`revisions <extractor.kemono.revisions_>`__
are returned.
* ``"asc"``: Ascending order (oldest first)
@@ -3292,8 +3292,8 @@ Description
the first in the list gets chosen (usually `mp3`).
extractor.koharu.cbz
--------------------
extractor.schalenetwork.cbz
---------------------------
Type
``bool``
Default
@@ -3305,8 +3305,8 @@ Description
to be downloaded as individual image files.
extractor.koharu.format
-----------------------
extractor.schalenetwork.format
------------------------------
Type
* ``string``
* ``list`` of ``strings``
@@ -3321,8 +3321,8 @@ Description
| ``"780"``, ``"980"``, ``"1280"``, ``"1600"``, ``"0"`` (original)
extractor.koharu.tags
---------------------
extractor.schalenetwork.tags
----------------------------
Type
``bool``
Default

24
docs/gallery-dl.conf
View File

@@ -191,7 +191,7 @@
"quality" : "original=true",
"quality-videos": "quality=100"
},
"coomerparty":
"coomer":
{
"username": "",
"password": "",
@@ -376,7 +376,7 @@
"sleep-request": "0.5-1.5",
"videos": true
},
"kemonoparty":
"kemono":
{
"username": "",
"password": "",
@@ -399,16 +399,6 @@
"covers": false,
"format": "mp3"
},
"koharu":
{
"username": "",
"password": "",
"sleep-request": "0.5-1.5",
"cbz" : true,
"format": ["0", "1600", "1280", "980", "780"],
"tags" : false
},
"luscious":
{
"gif": false
@@ -571,6 +561,16 @@
"embeds": false,
"videos": true
},
"schalenetwork":
{
"username": "",
"password": "",
"sleep-request": "0.5-1.5",
"cbz" : true,
"format": ["0", "1600", "1280", "980", "780"],
"tags" : false
},
"scrolller":
{
"username": "",

2
docs/supportedsites.md
View File

@@ -836,7 +836,7 @@ Consider all listed sites to potentially be NSFW.
<td></td>
</tr>
<tr>
<td>SchaleNetwork</td>
<td>Schale Network</td>
<td>https://niyaniya.moe/</td>
<td>Favorites, Galleries, Search Results</td>
<td></td>

4
gallery_dl/extractor/__init__.py
View File

@@ -91,9 +91,8 @@ modules = [
"jschan",
"kabeuchi",
"keenspot",
"kemonoparty",
"kemono",
"khinsider",
"koharu",
"komikcast",
"lensdump",
"lexica",
@@ -153,6 +152,7 @@ modules = [
"saint",
"sankaku",
"sankakucomplex",
"schalenetwork",
"scrolller",
"seiga",
"senmanga",

41
gallery_dl/extractor/kemonoparty.py → gallery_dl/extractor/kemono.py
View File

@@ -20,9 +20,9 @@ USER_PATTERN = BASE_PATTERN + r"/([^/?#]+)/user/([^/?#]+)"
HASH_PATTERN = r"/[0-9a-f]{2}/[0-9a-f]{2}/([0-9a-f]{64})"
class KemonopartyExtractor(Extractor):
"""Base class for kemonoparty extractors"""
category = "kemonoparty"
class KemonoExtractor(Extractor):
"""Base class for kemono extractors"""
category = "kemono"
root = "https://kemono.su"
directory_fmt = ("{category}", "{service}", "{user}")
filename_fmt = "{id}_{title[:180]}_{num:>02}_{filename[:180]}.{extension}"
@@ -30,9 +30,8 @@ class KemonopartyExtractor(Extractor):
cookies_domain = ".kemono.su"
def __init__(self, match):
domain = match.group(1)
tld = match.group(2)
self.category = domain + "party"
self.category = domain = match.group(1)
self.root = text.root_from_url(match.group(0))
self.cookies_domain = ".{}.{}".format(domain, tld)
Extractor.__init__(self, match)
@@ -46,7 +45,7 @@ class KemonopartyExtractor(Extractor):
self.revisions_reverse = order[0] in ("r", "a") if order else False
self._find_inline = re.compile(
r'src="(?:https?://(?:kemono|coomer)\.(?:su|party))?(/inline/[^"]+'
r'src="(?:https?://(?:kemono|coomer)\.su)?(/inline/[^"]+'
r'|/[0-9a-f]{2}/[0-9a-f]{2}/[0-9a-f]{64}\.[^"]+)').findall
self._json_dumps = json.JSONEncoder(
ensure_ascii=False, check_circular=False,
@@ -305,7 +304,7 @@ def _validate(response):
response.content != b"not found")
class KemonopartyUserExtractor(KemonopartyExtractor):
class KemonoUserExtractor(KemonoExtractor):
"""Extractor for all posts from a kemono.su user listing"""
subcategory = "user"
pattern = USER_PATTERN + r"/?(?:\?([^#]+))?(?:$|\?|#)"
@@ -313,7 +312,7 @@ class KemonopartyUserExtractor(KemonopartyExtractor):
def __init__(self, match):
self.subcategory = match.group(3)
KemonopartyExtractor.__init__(self, match)
KemonoExtractor.__init__(self, match)
def posts(self):
endpoint = self.config("endpoint")
@@ -337,7 +336,7 @@ class KemonopartyUserExtractor(KemonopartyExtractor):
service, creator_id, post["id"])["post"]
class KemonopartyPostsExtractor(KemonopartyExtractor):
class KemonoPostsExtractor(KemonoExtractor):
"""Extractor for kemono.su post listings"""
subcategory = "posts"
pattern = BASE_PATTERN + r"/posts()()(?:/?\?([^#]+))?"
@@ -349,7 +348,7 @@ class KemonopartyPostsExtractor(KemonopartyExtractor):
params.get("o"), params.get("q"), params.get("tag"))
class KemonopartyPostExtractor(KemonopartyExtractor):
class KemonoPostExtractor(KemonoExtractor):
"""Extractor for a single kemono.su post"""
subcategory = "post"
pattern = USER_PATTERN + r"/post/([^/?#]+)(/revisions?(?:/(\d*))?)?"
@@ -357,7 +356,7 @@ class KemonopartyPostExtractor(KemonopartyExtractor):
def __init__(self, match):
self.subcategory = match.group(3)
KemonopartyExtractor.__init__(self, match)
KemonoExtractor.__init__(self, match)
def posts(self):
_, _, service, creator_id, post_id, revision, revision_id = self.groups
@@ -378,7 +377,7 @@ class KemonopartyPostExtractor(KemonopartyExtractor):
raise exception.NotFoundError("revision")
class KemonopartyDiscordExtractor(KemonopartyExtractor):
class KemonoDiscordExtractor(KemonoExtractor):
"""Extractor for kemono.su discord servers"""
subcategory = "discord"
directory_fmt = ("{category}", "discord",
@@ -451,7 +450,7 @@ class KemonopartyDiscordExtractor(KemonopartyExtractor):
yield Message.Url, url, post
class KemonopartyDiscordServerExtractor(KemonopartyExtractor):
class KemonoDiscordServerExtractor(KemonoExtractor):
subcategory = "discord-server"
pattern = BASE_PATTERN + r"/discord/server/(\d+)$"
example = "https://kemono.su/discord/server/12345"
@@ -465,7 +464,7 @@ class KemonopartyDiscordServerExtractor(KemonopartyExtractor):
yield Message.Queue, url, {
"server" : server,
"channel" : channel,
"_extractor": KemonopartyDiscordExtractor,
"_extractor": KemonoDiscordExtractor,
}
@@ -478,7 +477,7 @@ def discord_server_info(extr, server_id):
}
class KemonopartyFavoriteExtractor(KemonopartyExtractor):
class KemonoFavoriteExtractor(KemonoExtractor):
"""Extractor for kemono.su favorites"""
subcategory = "favorite"
pattern = BASE_PATTERN + r"/(?:account/)?favorites()()(?:/?\?([^#]+))?"
@@ -504,11 +503,11 @@ class KemonopartyFavoriteExtractor(KemonopartyExtractor):
for user in users:
service = user["service"]
if service == "discord":
user["_extractor"] = KemonopartyDiscordServerExtractor
user["_extractor"] = KemonoDiscordServerExtractor
url = "{}/discord/server/{}".format(
self.root, user["id"])
else:
user["_extractor"] = KemonopartyUserExtractor
user["_extractor"] = KemonoUserExtractor
url = "{}/{}/user/{}".format(
self.root, service, user["id"])
yield Message.Queue, url, user
@@ -522,13 +521,13 @@ class KemonopartyFavoriteExtractor(KemonopartyExtractor):
reverse=(order == "desc"))
for post in posts:
post["_extractor"] = KemonopartyPostExtractor
post["_extractor"] = KemonoPostExtractor
url = "{}/{}/user/{}/post/{}".format(
self.root, post["service"], post["user"], post["id"])
yield Message.Queue, url, post
class KemonopartyArtistsExtractor(KemonopartyExtractor):
class KemonoArtistsExtractor(KemonoExtractor):
"""Extractor for kemono artists"""
subcategory = "artists"
pattern = BASE_PATTERN + r"/artists(?:\?([^#]+))?"
@@ -556,11 +555,11 @@ class KemonopartyArtistsExtractor(KemonopartyExtractor):
for user in users:
service = user["service"]
if service == "discord":
user["_extractor"] = KemonopartyDiscordServerExtractor
user["_extractor"] = KemonoDiscordServerExtractor
url = "{}/discord/server/{}".format(
self.root, user["id"])
else:
user["_extractor"] = KemonopartyUserExtractor
user["_extractor"] = KemonoUserExtractor
url = "{}/{}/user/{}".format(
self.root, service, user["id"])
yield Message.Queue, url, user

20
gallery_dl/extractor/koharu.py → gallery_dl/extractor/schalenetwork.py
View File

@@ -22,9 +22,9 @@ BASE_PATTERN = (
)
class KoharuExtractor(Extractor):
"""Base class for koharu extractors"""
category = "koharu"
class SchalenetworkExtractor(Extractor):
"""Base class for schale.network extractors"""
category = "schalenetwork"
root = "https://niyaniya.moe"
root_api = "https://api.schale.network"
request_interval = (0.5, 1.5)
@@ -51,7 +51,7 @@ class KoharuExtractor(Extractor):
for entry in entries:
url = "{}/g/{}/{}".format(
self.root, entry["id"], entry["public_key"])
entry["_extractor"] = KoharuGalleryExtractor
entry["_extractor"] = SchalenetworkGalleryExtractor
yield Message.Queue, url, entry
try:
@@ -62,8 +62,8 @@ class KoharuExtractor(Extractor):
params["page"] += 1
class KoharuGalleryExtractor(KoharuExtractor, GalleryExtractor):
"""Extractor for koharu galleries"""
class SchalenetworkGalleryExtractor(SchalenetworkExtractor, GalleryExtractor):
"""Extractor for schale.network galleries"""
filename_fmt = "{num:>03}.{extension}"
directory_fmt = ("{category}", "{id} {title}")
archive_fmt = "{id}_{num}"
@@ -205,8 +205,8 @@ class KoharuGalleryExtractor(KoharuExtractor, GalleryExtractor):
return fmt
class KoharuSearchExtractor(KoharuExtractor):
"""Extractor for koharu search results"""
class SchalenetworkSearchExtractor(SchalenetworkExtractor):
"""Extractor for schale.network search results"""
subcategory = "search"
pattern = BASE_PATTERN + r"/\?([^#]*)"
example = "https://niyaniya.moe/?s=QUERY"
@@ -217,8 +217,8 @@ class KoharuSearchExtractor(KoharuExtractor):
return self._pagination("/books", params)
class KoharuFavoriteExtractor(KoharuExtractor):
"""Extractor for koharu favorites"""
class SchalenetworkFavoriteExtractor(SchalenetworkExtractor):
"""Extractor for schale.network favorites"""
subcategory = "favorite"
pattern = BASE_PATTERN + r"/favorites(?:\?([^#]*))?"
example = "https://niyaniya.moe/favorites"

28
scripts/supportedsites.py
View File

@@ -40,8 +40,6 @@ CATEGORY_MAP = {
"cien" : "Ci-en",
"cohost" : "cohost!",
"comicvine" : "Comic Vine",
"coomerparty" : "Coomer",
"deltaporno" : "DeltaPorno",
"deviantart" : "DeviantArt",
"drawfriends" : "Draw Friends",
"dynastyscans" : "Dynasty Reader",
@@ -90,8 +88,7 @@ CATEGORY_MAP = {
"itchio" : "itch.io",
"jpgfish" : "JPG Fish",
"kabeuchi" : "かべうち",
"kemonoparty" : "Kemono",
"koharu" : "SchaleNetwork",
"schalenetwork" : "Schale Network",
"livedoor" : "livedoor Blog",
"lofter" : "LOFTER",
"ohpolly" : "Oh Polly",
@@ -197,9 +194,10 @@ SUBCATEGORY_MAP = {
"tweets" : "",
"user" : "User Profiles",
"watch" : "Watches",
"following" : "Followed Users",
"related-pin" : "related Pins",
"related-board": "",
"direct-messages": "DMs",
"following" : "Followed Users",
"related-pin" : "related Pins",
"related-board" : "",
"ao3": {
"user-works" : "",
@@ -221,7 +219,6 @@ SUBCATEGORY_MAP = {
"posts": "",
},
"boosty": {
"direct-messages": "DMs",
"feed": "Subscriptions Feed",
},
"civitai": {
@@ -232,7 +229,7 @@ SUBCATEGORY_MAP = {
"user-posts" : "User Posts",
"user-videos": "User Videos",
},
"coomerparty": {
"coomer": {
"discord" : "",
"discord-server": "",
"posts" : "",
@@ -251,7 +248,6 @@ SUBCATEGORY_MAP = {
"watch-posts": "",
},
"discord": {
"direct-messages": "DMs",
"direct-message" : "",
},
"fanbox": {
@@ -283,7 +279,7 @@ SUBCATEGORY_MAP = {
"saved": "Saved Posts",
"tagged": "Tagged Posts",
},
"kemonoparty": {
"kemono": {
"discord" : "Discord Servers",
"discord-server": "",
"posts" : "",
@@ -443,7 +439,7 @@ AUTH_MAP = {
"bluesky" : "Supported",
"booruvar" : "Supported",
"boosty" : _COOKIES,
"coomerparty" : "Supported",
"coomer" : "Supported",
"danbooru" : "Supported",
"derpibooru" : _APIKEY_DB,
"deviantart" : _OAUTH,
@@ -463,7 +459,7 @@ AUTH_MAP = {
"imgbb" : "Supported",
"inkbunny" : "Supported",
"instagram" : _COOKIES,
"kemonoparty" : "Supported",
"kemono" : "Supported",
"mangadex" : "Supported",
"mangoxo" : "Supported",
"mastodon.social": _OAUTH,
@@ -603,9 +599,9 @@ def build_extractor_list():
default["e-hentai"] = default["exhentai"]
domains["e-hentai"] = domains["exhentai"].replace("x", "-")
# add coomer.party
default["coomerparty"] = default["kemonoparty"]
domains["coomerparty"] = domains["kemonoparty"].replace("kemono", "coomer")
# add coomer.su
default["coomer"] = default["kemono"]
domains["coomer"] = domains["kemono"].replace("kemono", "coomer")
# add manga4life.com
default["mangalife"] = default["mangasee"]

10
test/results/coomerparty.py → test/results/coomer.py
View File

@@ -4,22 +4,22 @@
# it under the terms of the GNU General Public License version 2 as
# published by the Free Software Foundation.
from gallery_dl.extractor import kemonoparty
from gallery_dl.extractor import kemono
__tests__ = (
{
"#url" : "https://coomer.su/onlyfans/user/alinity/post/125962203",
"#comment" : "coomer (#2100)",
"#category": ("", "coomerparty", "onlyfans"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "coomer", "onlyfans"),
"#class" : kemono.KemonoPostExtractor,
"#urls" : "https://coomer.su/data/7d/3f/7d3fd9804583dc224968c0591163ec91794552b04f00a6c2f42a15b68231d5a8.jpg",
},
{
"#url" : "https://coomer.party/onlyfans/user/alinity/post/125962203",
"#category": ("", "coomerparty", "onlyfans"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "coomer", "onlyfans"),
"#class" : kemono.KemonoPostExtractor,
"#urls" : "https://coomer.party/data/7d/3f/7d3fd9804583dc224968c0591163ec91794552b04f00a6c2f42a15b68231d5a8.jpg",
},

196
test/results/kemonoparty.py → test/results/kemono.py
View File

@@ -4,15 +4,15 @@
# it under the terms of the GNU General Public License version 2 as
# published by the Free Software Foundation.
from gallery_dl.extractor import kemonoparty
from gallery_dl.extractor import kemono
from gallery_dl import exception
__tests__ = (
{
"#url" : "https://kemono.su/fanbox/user/6993449",
"#category": ("", "kemonoparty", "fanbox"),
"#class" : kemonoparty.KemonopartyUserExtractor,
"#category": ("", "kemono", "fanbox"),
"#class" : kemono.KemonoUserExtractor,
"#options" : {"endpoint": "legacy"},
"#range" : "1-500",
"#count" : 500,
@@ -44,8 +44,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/fanbox/user/6993449",
"#comment" : "endpoint: legacy+ (#7438 #7450 #7462)",
"#category": ("", "kemonoparty", "fanbox"),
"#class" : kemonoparty.KemonopartyUserExtractor,
"#category": ("", "kemono", "fanbox"),
"#class" : kemono.KemonoUserExtractor,
"#options" : {"endpoint": "legacy+"},
"#range" : "1-10",
@@ -81,8 +81,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/patreon/user/881792?o=150",
"#comment" : "'max-posts' and 'endpoint' option, 'o' query parameter (#1674)",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyUserExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoUserExtractor,
"#options" : {"max-posts": 100, "endpoint": "posts"},
"#count" : range(200, 400),
@@ -111,8 +111,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/fanbox/user/6993449?q=お蔵入りになった",
"#comment" : "search / 'q' query parameter (#3385, #4057)",
"#category": ("", "kemonoparty", "fanbox"),
"#class" : kemonoparty.KemonopartyUserExtractor,
"#category": ("", "kemono", "fanbox"),
"#class" : kemono.KemonoUserExtractor,
"#urls" : (
"https://kemono.su/data/ef/7b/ef7b4398a2f4ada597421fd3c116cff86e85695911f7cd2a459b0e566b864e46.png",
"https://kemono.su/data/73/e6/73e615f6645b9d1af6329448601673c9275f07fd11eb37670c97e307e29a9ee9.png",
@@ -124,8 +124,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/patreon/user/3161935?tag=pin-up",
"#comment" : "'tag' query parameter",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyUserExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoUserExtractor,
"#urls" : (
"https://kemono.su/data/83/61/8361560887a09c7b828d326b3e1a2f0288673741569a09d74bcd01e602d20db1.png",
"https://kemono.su/data/03/e6/03e62592c3b616b8906c1aaa130bd9ceaa24d7f601b31f90cc11956a57ca1d82.png",
@@ -141,20 +141,20 @@ __tests__ = (
{
"#url" : "https://kemono.su/subscribestar/user/alcorart",
"#category": ("", "kemonoparty", "subscribestar"),
"#class" : kemonoparty.KemonopartyUserExtractor,
"#category": ("", "kemono", "subscribestar"),
"#class" : kemono.KemonoUserExtractor,
},
{
"#url" : "https://kemono.su/subscribestar/user/alcorart",
"#category": ("", "kemonoparty", "subscribestar"),
"#class" : kemonoparty.KemonopartyUserExtractor,
"#category": ("", "kemono", "subscribestar"),
"#class" : kemono.KemonoUserExtractor,
},
{
"#url" : "https://kemono.su/fanbox/user/6993449/post/506575",
"#category": ("", "kemonoparty", "fanbox"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "fanbox"),
"#class" : kemono.KemonoPostExtractor,
"#pattern" : r"https://kemono.su/data/21/0f/210f35388e28bbcf756db18dd516e2d82ce75[0-9a-f]+\.jpg",
"#sha1_content": "900949cefc97ab8dc1979cc3664785aac5ba70dd",
@@ -182,8 +182,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/fanbox/user/7356311/post/802343",
"#comment" : "inline image (#1286)",
"#category": ("", "kemonoparty", "fanbox"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "fanbox"),
"#class" : kemono.KemonoPostExtractor,
"#pattern" : r"https://kemono\.su/data/47/b5/47b5c014ecdcfabdf2c85eec53f1133a76336997ae8596f332e97d956a460ad2\.jpg",
"hash": "47b5c014ecdcfabdf2c85eec53f1133a76336997ae8596f332e97d956a460ad2",
@@ -191,16 +191,16 @@ __tests__ = (
{
"#url" : "https://kemono.su/gumroad/user/3101696181060/post/tOWyf",
"#category": ("", "kemonoparty", "gumroad"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "gumroad"),
"#class" : kemono.KemonoPostExtractor,
"#count" : 12,
},
{
"#url" : "https://kemono.party/gumroad/user/3252870377455/post/aJnAH",
"#comment" : "username (#1548, #1652)",
"#category": ("", "kemonoparty", "gumroad"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "gumroad"),
"#class" : kemono.KemonoPostExtractor,
"#options" : {"metadata": True},
"username": "Kudalyn's Creations",
@@ -209,16 +209,16 @@ __tests__ = (
{
"#url" : "https://kemono.su/patreon/user/4158582/post/32099982",
"#comment" : "allow duplicates (#2440)",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoPostExtractor,
"#count" : 2,
},
{
"#url" : "https://kemono.su/patreon/user/4158582/post/32099982",
"#comment" : "allow duplicates (#2440)",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoPostExtractor,
"#options" : {"duplicates": True},
"#count" : 3,
},
@@ -226,8 +226,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/patreon/user/3161935/post/23445732",
"#comment" : "comments (#2008)",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoPostExtractor,
"#options" : {"comments": True},
"comments": "len:12",
@@ -236,8 +236,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/patreon/user/34134344/post/38129255",
"#comment" : "DMs (#2008); no comments",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoPostExtractor,
"#options" : {"dms": True, "comments": True},
"comments": (),
@@ -259,8 +259,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/patreon/user/3161935/post/68231671",
"#comment" : "announcements",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoPostExtractor,
"#options" : {"announcements": True},
"announcements": [
@@ -278,41 +278,41 @@ __tests__ = (
{
"#url" : "https://kemono.su/patreon/user/19623797/post/29035449",
"#comment" : "invalid file (#3510)",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoPostExtractor,
"#pattern" : r"907ba78b4545338d3539683e63ecb51cf51c10adc9dabd86e92bd52339f298b9\.txt",
"#sha1_content": "da39a3ee5e6b4b0d3255bfef95601890afd80709",
},
{
"#url" : "https://kemono.su/subscribestar/user/alcorart/post/184330",
"#category": ("", "kemonoparty", "subscribestar"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "subscribestar"),
"#class" : kemono.KemonoPostExtractor,
},
{
"#url" : "https://kemono.su/subscribestar/user/alcorart/post/184330",
"#category": ("", "kemonoparty", "subscribestar"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "subscribestar"),
"#class" : kemono.KemonoPostExtractor,
},
{
"#url" : "https://www.kemono.su/subscribestar/user/alcorart/post/184330",
"#category": ("", "kemonoparty", "subscribestar"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "subscribestar"),
"#class" : kemono.KemonoPostExtractor,
},
{
"#url" : "https://beta.kemono.su/subscribestar/user/alcorart/post/184330",
"#category": ("", "kemonoparty", "subscribestar"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "subscribestar"),
"#class" : kemono.KemonoPostExtractor,
},
{
"#url" : "https://kemono.su/patreon/user/3161935/post/68231671/revision/142470",
"#comment" : "revisions (#4498)",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoPostExtractor,
"#urls" : "https://kemono.su/data/88/52/88521f71822dfa2f42df3beba319ea4fceda2a2d6dc59da0276a75238f743f86.jpg",
"file": {
@@ -341,8 +341,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/patreon/user/3161935/post/68231671",
"#comment" : "unique revisions (#5013)",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoPostExtractor,
"#options" : {"revisions": "unique"},
"#urls" : "https://kemono.su/data/e3/e6/e3e6287dbc0468dd2a9d28ed276ae86788907143acf2ba10ab886a3add4c436c.jpg",
"#archive" : False,
@@ -365,8 +365,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/patreon/user/3161935/post/68231671/revisions",
"#comment" : "revisions (#4498)",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoPostExtractor,
"#pattern" : r"https://kemono\.su/data/88/52/88521f71822dfa2f42df3beba319ea4fceda2a2d6dc59da0276a75238f743f86\.jpg",
"#count" : 11,
"#archive" : False,
@@ -384,16 +384,16 @@ __tests__ = (
{
"#url" : "https://kemono.su/patreon/user/3161935/post/68231671/revision/12345",
"#comment" : "revisions (#4498)",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoPostExtractor,
"#exception": exception.NotFoundError,
},
{
"#url" : "https://kemono.su/patreon/user/6298789/post/69764693",
"#comment" : "'published' metadata with extra microsecond data",
"#category": ("", "kemonoparty", "patreon"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "patreon"),
"#class" : kemono.KemonoPostExtractor,
"date" : "dt:2022-07-29 21:12:11",
"published": "2022-07-29T21:12:11.483000",
@@ -402,8 +402,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/gumroad/user/3267960360326/post/jwwag",
"#comment" : "empty 'file' with no 'path' (#5368)",
"#category": ("", "kemonoparty", "gumroad"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "gumroad"),
"#class" : kemono.KemonoPostExtractor,
"#count" : 8,
"type" : "attachment",
@@ -412,8 +412,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/fanbox/user/49494721/post/9457614",
"#comment" : "archives",
"#category": ("", "kemonoparty", "fanbox"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "fanbox"),
"#class" : kemono.KemonoPostExtractor,
"#options" : {"archives": True},
"#range" : "1-2",
@@ -494,8 +494,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/boosty/user/felixf/post/d9d8d670-16be-4e06-8ff9-65b13e322ba8",
"#comment" : r"'\' in file paths",
"#category": ("", "kemonoparty", "boosty"),
"#class" : kemonoparty.KemonopartyPostExtractor,
"#category": ("", "kemono", "boosty"),
"#class" : kemono.KemonoPostExtractor,
"#urls" : (
"https://kemono.su/data/dd/35/dd35c43d8a93f1806f094d9331a17c5037ed5d93e0f30c28d3cca2056b400aa6.png",
"https://kemono.su/data/25/48/254864eb2523ab48be8d3fb7ad21ab3a127d61736b76602f8421cde88700a174.png",
@@ -513,8 +513,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/discord/server/488668827274444803/608504710906904576",
"#category": ("", "kemonoparty", "discord"),
"#class" : kemonoparty.KemonopartyDiscordExtractor,
"#category": ("", "kemono", "discord"),
"#class" : kemono.KemonoDiscordExtractor,
"#count" : 4,
"channel" : "finish-work",
@@ -528,8 +528,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/discord/server/488668827274444803#608504710906904576",
"#category": ("", "kemonoparty", "discord"),
"#class" : kemonoparty.KemonopartyDiscordExtractor,
"#category": ("", "kemono", "discord"),
"#class" : kemono.KemonoDiscordExtractor,
"#count" : 4,
"channel" : "finish-work",
@@ -543,8 +543,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/discord/server/488668827274444803/channel/608504710906904576#finish-work",
"#category": ("", "kemonoparty", "discord"),
"#class" : kemonoparty.KemonopartyDiscordExtractor,
"#category": ("", "kemono", "discord"),
"#class" : kemono.KemonoDiscordExtractor,
"#count" : 4,
"channel" : "finish-work",
@@ -560,8 +560,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/discord/server/818188637329031199/818343747275456522",
"#comment" : "pagination",
"#category": ("", "kemonoparty", "discord"),
"#class" : kemonoparty.KemonopartyDiscordExtractor,
"#category": ("", "kemono", "discord"),
"#class" : kemono.KemonoDiscordExtractor,
"#range" : "1-250",
"#count" : 250,
@@ -576,8 +576,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/discord/server/256559665620451329/channel/462437519519383555#",
"#category": ("", "kemonoparty", "discord"),
"#class" : kemonoparty.KemonopartyDiscordExtractor,
"#category": ("", "kemono", "discord"),
"#class" : kemono.KemonoDiscordExtractor,
"#pattern" : r"https://kemono\.su/data/(e3/77/e377e3525164559484ace2e64425b0cec1db08.*\.png|51/45/51453640a5e0a4d23fbf57fb85390f9c5ec154.*\.gif)",
"#count" : ">= 2",
@@ -590,8 +590,8 @@ __tests__ = (
{
"#url" : "https://kemono.su/discord/server/315262215055736843/channel/315262215055736843#general",
"#comment" : "'inline' files",
"#category": ("", "kemonoparty", "discord"),
"#class" : kemonoparty.KemonopartyDiscordExtractor,
"#category": ("", "kemono", "discord"),
"#class" : kemono.KemonoDiscordExtractor,
"#options" : {"image-filter": "type == 'inline'"},
"#pattern" : r"https://cdn\.discordapp\.com/attachments/\d+/\d+/.+$",
"#range" : "1-5",
@@ -601,24 +601,24 @@ __tests__ = (
{
"#url" : "https://kemono.su/discord/server/488668827274444803",
"#category": ("", "kemonoparty", "discord-server"),
"#class" : kemonoparty.KemonopartyDiscordServerExtractor,
"#pattern" : kemonoparty.KemonopartyDiscordExtractor.pattern,
"#category": ("", "kemono", "discord-server"),
"#class" : kemono.KemonoDiscordServerExtractor,
"#pattern" : kemono.KemonoDiscordExtractor.pattern,
"#count" : 26,
},
{
"#url" : "https://kemono.su/posts?q=foobar",
"#category": ("", "kemonoparty", "posts"),
"#class" : kemonoparty.KemonopartyPostsExtractor,
"#category": ("", "kemono", "posts"),
"#class" : kemono.KemonoPostsExtractor,
"#count" : range(60, 100),
},
{
"#url" : "https://kemono.su/favorites",
"#category": ("", "kemonoparty", "favorite"),
"#class" : kemonoparty.KemonopartyFavoriteExtractor,
"#pattern" : kemonoparty.KemonopartyUserExtractor.pattern,
"#category": ("", "kemono", "favorite"),
"#class" : kemono.KemonoFavoriteExtractor,
"#pattern" : kemono.KemonoUserExtractor.pattern,
"#auth" : True,
"#urls" : (
"https://kemono.su/patreon/user/881792",
@@ -630,9 +630,9 @@ __tests__ = (
{
"#url" : "https://kemono.su/favorites?type=artist&sort=faved_seq&order=asc",
"#category": ("", "kemonoparty", "favorite"),
"#class" : kemonoparty.KemonopartyFavoriteExtractor,
"#pattern" : kemonoparty.KemonopartyUserExtractor.pattern,
"#category": ("", "kemono", "favorite"),
"#class" : kemono.KemonoFavoriteExtractor,
"#pattern" : kemono.KemonoUserExtractor.pattern,
"#auth" : True,
"#urls" : (
"https://kemono.su/fanbox/user/6993449",
@@ -644,9 +644,9 @@ __tests__ = (
{
"#url" : "https://kemono.su/favorites?type=post",
"#category": ("", "kemonoparty", "favorite"),
"#class" : kemonoparty.KemonopartyFavoriteExtractor,
"#pattern" : kemonoparty.KemonopartyPostExtractor.pattern,
"#category": ("", "kemono", "favorite"),
"#class" : kemono.KemonoFavoriteExtractor,
"#pattern" : kemono.KemonoPostExtractor.pattern,
"#auth" : True,
"#urls" : (
"https://kemono.su/subscribestar/user/alcorart/post/184329",
@@ -657,9 +657,9 @@ __tests__ = (
{
"#url" : "https://kemono.su/favorites?type=post&sort=published&order=asc",
"#category": ("", "kemonoparty", "favorite"),
"#class" : kemonoparty.KemonopartyFavoriteExtractor,
"#pattern" : kemonoparty.KemonopartyPostExtractor.pattern,
"#category": ("", "kemono", "favorite"),
"#class" : kemono.KemonoFavoriteExtractor,
"#pattern" : kemono.KemonoPostExtractor.pattern,
"#auth" : True,
"#urls" : (
"https://kemono.su/patreon/user/881792/post/4769638",
@@ -670,21 +670,21 @@ __tests__ = (
{
"#url" : "https://kemono.su/account/favorites/artists",
"#category": ("", "kemonoparty", "favorite"),
"#class" : kemonoparty.KemonopartyFavoriteExtractor,
"#category": ("", "kemono", "favorite"),
"#class" : kemono.KemonoFavoriteExtractor,
},
{
"#url" : "https://kemono.su/account/favorites/posts?sort_by=published&order=asc",
"#category": ("", "kemonoparty", "favorite"),
"#class" : kemonoparty.KemonopartyFavoriteExtractor,
"#category": ("", "kemono", "favorite"),
"#class" : kemono.KemonoFavoriteExtractor,
},
{
"#url" : "https://kemono.su/artists?q=aMSa",
"#category": ("", "kemonoparty", "artists"),
"#class" : kemonoparty.KemonopartyArtistsExtractor,
"#pattern" : kemonoparty.KemonopartyUserExtractor.pattern,
"#category": ("", "kemono", "artists"),
"#class" : kemono.KemonoArtistsExtractor,
"#pattern" : kemono.KemonoUserExtractor.pattern,
"#urls" : (
"https://kemono.su/patreon/user/91205314",
"https://kemono.su/patreon/user/51528107",
@@ -713,9 +713,9 @@ __tests__ = (
{
"#url" : "https://kemono.su/artists?q=Axe&service=discord&sort_by=name&order=asc",
"#category": ("", "kemonoparty", "artists"),
"#class" : kemonoparty.KemonopartyArtistsExtractor,
"#pattern" : kemonoparty.KemonopartyDiscordServerExtractor.pattern,
"#category": ("", "kemono", "artists"),
"#class" : kemono.KemonoArtistsExtractor,
"#pattern" : kemono.KemonoDiscordServerExtractor.pattern,
"#urls" : "https://kemono.su/discord/server/1168450323023663164",
"favorited": range(40, 80),

39
test/results/koharu.py → test/results/schalenetwork.py
View File

@@ -4,14 +4,13 @@
# it under the terms of the GNU General Public License version 2 as
# published by the Free Software Foundation.
from gallery_dl.extractor import koharu
from gallery_dl.extractor import schalenetwork
__tests__ = (
{
"#url" : "https://niyaniya.moe/g/14216/6c67076fdd45",
"#category": ("", "koharu", "gallery"),
"#class" : koharu.KoharuGalleryExtractor,
"#class" : schalenetwork.SchalenetworkGalleryExtractor,
"#options" : {"tags": True},
"#pattern" : r"https://kisakisexo.xyz/download/59896/a4fbd1828229/f47639c6abaf1903dd69c36a3d961da84741a1831aa07a2906ce9c74156a5d75\?v=1721626410802&w=0",
"#count" : 1,
@@ -80,8 +79,7 @@ __tests__ = (
{
"#url" : "https://niyaniya.moe/g/14216/6c67076fdd45",
"#category": ("", "koharu", "gallery"),
"#class" : koharu.KoharuGalleryExtractor,
"#class" : schalenetwork.SchalenetworkGalleryExtractor,
"#options" : {"cbz": False, "format": "780"},
"#pattern" : r"https://koharusexo.xyz/data/59905/2df9110af7f1/a7cbeca3fb9c83aa87582a8a74cc8f8ce1b9e9b434dc1af293628871642f42df/[0-9a-f]+/.+",
"#count" : 22,
@@ -89,8 +87,7 @@ __tests__ = (
{
"#url" : "https://niyaniya.moe/g/14216/6c67076fdd45",
"#category": ("", "koharu", "gallery"),
"#class" : koharu.KoharuGalleryExtractor,
"#class" : schalenetwork.SchalenetworkGalleryExtractor,
"#options" : {"cbz": False, "format": "780"},
"#range" : "1",
"#sha1_content": "08954e0ae18a900ee7ca144d1661c664468c2525",
@@ -98,44 +95,41 @@ __tests__ = (
{
"#url" : "https://koharu.to/g/14216/6c67076fdd45",
"#class": koharu.KoharuGalleryExtractor,
"#class": schalenetwork.SchalenetworkGalleryExtractor,
},
{
"#url" : "https://anchira.to/g/14216/6c67076fdd45",
"#class": koharu.KoharuGalleryExtractor,
"#class": schalenetwork.SchalenetworkGalleryExtractor,
},
{
"#url" : "https://seia.to/g/14216/6c67076fdd45",
"#class": koharu.KoharuGalleryExtractor,
"#class": schalenetwork.SchalenetworkGalleryExtractor,
},
{
"#url" : "https://shupogaki.moe/g/14216/6c67076fdd45",
"#class": koharu.KoharuGalleryExtractor,
"#class": schalenetwork.SchalenetworkGalleryExtractor,
},
{
"#url" : "https://hoshino.one/g/14216/6c67076fdd45",
"#class": koharu.KoharuGalleryExtractor,
"#class": schalenetwork.SchalenetworkGalleryExtractor,
},
{
"#url" : "https://niyaniya.moe/reader/14216/6c67076fdd45",
"#category": ("", "koharu", "gallery"),
"#class" : koharu.KoharuGalleryExtractor,
"#class" : schalenetwork.SchalenetworkGalleryExtractor,
},
{
"#url" : "https://niyaniya.moe/?s=tag:^beach$",
"#category": ("", "koharu", "search"),
"#class" : koharu.KoharuSearchExtractor,
"#pattern" : koharu.KoharuGalleryExtractor.pattern,
"#class" : schalenetwork.SchalenetworkSearchExtractor,
"#pattern" : schalenetwork.SchalenetworkGalleryExtractor.pattern,
"#count" : ">= 50",
},
{
"#url" : "https://niyaniya.moe/favorites",
"#category": ("", "koharu", "favorite"),
"#class" : koharu.KoharuFavoriteExtractor,
"#pattern" : koharu.KoharuGalleryExtractor.pattern,
"#class" : schalenetwork.SchalenetworkFavoriteExtractor,
"#pattern" : schalenetwork.SchalenetworkGalleryExtractor.pattern,
"#auth" : True,
"#urls" : [
"https://niyaniya.moe/g/14216/6c67076fdd45",
@@ -144,9 +138,8 @@ __tests__ = (
{
"#url" : "https://niyaniya.moe/favorites?cat=6&sort=4",
"#category": ("", "koharu", "favorite"),
"#class" : koharu.KoharuFavoriteExtractor,
"#pattern" : koharu.KoharuGalleryExtractor.pattern,
"#class" : schalenetwork.SchalenetworkFavoriteExtractor,
"#pattern" : schalenetwork.SchalenetworkGalleryExtractor.pattern,
"#auth" : True,
"#urls" : [
"https://niyaniya.moe/g/14216/6c67076fdd45",