github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,580 Commits 1 Branch 0 Tags
bd7fcdab4cee2e26300119ad5232adb65479adf1
Commit Graph

6580 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
bd7fcdab4c [flickr] provide human-readable 'license_name' metadata 2025-05-12 16:44:22 +02:00
Mike Fährmann
cf6eff7ff7 [flickr] remove constructors 2025-05-12 16:27:14 +02:00
Mike Fährmann
0dce9ad7c6 [mangadex] send Bearer token only when necessary 
> Do not send authentication headers unless necessary for your API call
> Authenticated requests cannot be cached, so you're slowing yourself down.

https://api.mangadex.org/docs/02-authentication/
 2025-05-12 15:08:13 +02:00
Mike Fährmann
7907d0d3bd [mangadex] add 'following' extractor (#7487) 
also fixes the URL pattern for the Updates feed at
https://mangadex.org/titles/feed
 2025-05-12 12:58:22 +02:00
Mike Fährmann
f88e42c194 [mangadex] implement login with client credentials 
unless the legacy method still works for your account,
login now requires 4 values:

- client-id
- client-secret
- username
- password

where 'client-id' and 'client-secret' are the credentials
of a personal API client.

> Public clients are not yet available.
 2025-05-12 11:57:34 +02:00
Mike Fährmann
d92bc06f90 [twitter:ctid] reduce chance of generating the same ID 
Querying the same endpoint more than once per second could result in a
"404 Not Found" error if the same transaction ID is generated by
'random.randrange(256)' yielding the same result.

This commit tries to significantly reduce the chance of this happening
by incorporating the fractional part of the current timestamp into the
random number, making it only possible to generate the same ID for
one-sixteenth of a second.
 2025-05-10 21:27:22 +02:00
Mike Fährmann
98fdcd4d72 [subscribestar] fix 'content' extraction (#7486) 
and extract 'tags' metadata

Authored by: prowlguru

Co-authored-by: prowlguru <183935626+prowlguru@users.noreply.github.com>
 2025-05-10 21:04:27 +02:00
Mike Fährmann
1da44d8fe6 [bluesky] handle exception for posts without 'record' (#7499) 2025-05-10 20:24:25 +02:00
Mike Fährmann
2755425ddd [fanbox] update API headers (#7490) 2025-05-10 20:23:04 +02:00
Mike Fährmann
cd01eacd3d [mastodon] support Akkoma/Pleroma '/objects/:uuid' URLs (#7497) 2025-05-10 18:01:45 +02:00
Mike Fährmann
18eac5ca70 merge #7496: [mastodon] support Akkoma/Pleroma '/notice/:id' URLs 2025-05-10 17:50:34 +02:00
Mike Fährmann
1f7c29aee9 [tests:mastodon] move comments into '#comments' fields 2025-05-10 17:49:52 +02:00
Mike Fährmann
e233360015 [vipergirls] extract more metadata & rename fields (#7479) 
- forum_title
- thread_title

- id     -> post_id
- title  -> post_title
- number -> post_num
 2025-05-10 17:40:43 +02:00
Mike Fährmann
d872509060 [vipergirls] provide 'num' and 'count' metadata (#7479) 2025-05-10 17:10:32 +02:00
Mike Fährmann
f0a46bf979 [docs] update 'civitai.nsfw' example 
include XXX-rated files
 2025-05-10 16:43:39 +02:00
Mike Fährmann
f343c8c0b6 [civitai] fix/improve video downloads (#7502) 
- add 'quality-videos' option
- fix downloading static .webp files instead of video
 2025-05-10 16:13:48 +02:00
Soblow "Opale" Xaselgio
eba44b8f78 [mastodon] Add unit tests for various statuses URL formats on various fediverse software (mastodon/akkoma/pleroma) 
Signed-off-by: Soblow "Opale" Xaselgio <113846014+Soblow@users.noreply.github.com>
 2025-05-09 12:03:57 +02:00
Soblow "Opale" Xaselgio
a94672bede [mastodon] Add support for Akkoma/Pleroma /notice/:status_id urls 
Signed-off-by: Soblow "Opale" Xaselgio <113846014+Soblow@users.noreply.github.com>
 2025-05-09 12:03:57 +02:00
Mike Fährmann
3799369248 [civitai] provide proper 'extension' for model files (#7432) 2025-05-04 17:34:06 +02:00
Mike Fährmann
25e293c16f release version 1.29.6 2025-05-04 09:18:03 +02:00
Mike Fährmann
b97dc456b0 [twitter] import 'transaction_id' only when needed 2025-05-04 07:42:44 +02:00
Mike Fährmann
6746e38670 [civitai] fix typo 2025-05-03 21:15:26 +02:00
Mike Fährmann
134c3854ee merge #7469: [weasyl] support '/view/' URLs 2025-05-03 20:45:58 +02:00
Mike Fährmann
bff02bc5f4 [weasyl] add '/view/' test 2025-05-03 20:44:34 +02:00
Mike Fährmann
7a67348fc2 [civitai] implement retrieving 'version' metadata (#7432) 2025-05-03 20:39:35 +02:00
Soblow "Opale" Xaselgio
07ac157acb Accept Weasyl URLs of format /view/12345 
Signed-off-by: Soblow "Opale" Xaselgio <113846014+Soblow@users.noreply.github.com>
 2025-05-03 19:35:37 +02:00
Mike Fährmann
edc67983ed [twitter] update 'x-csrf-token' header after ct init (#7467) 2025-05-03 12:55:31 +02:00
Mike Fährmann
771317b36c [twitter:ctid] cache client transaction keys (#7382) 
and 'ondemand.s.…a.js' responses
 2025-05-03 12:50:00 +02:00
Mike Fährmann
a1fd329783 [tumblr] improve error message for dashboard-only blogs (#7455) 2025-05-03 11:02:38 +02:00
Mike Fährmann
e26ae210bd [kemonoparty] restore using default creator posts endpoint 
(#7438 #7450 #7462)

- revert c3fb0f53ee
- add '"endpoint": "legacy+"' option
  to use '/posts-legacy' results and with full metadata
 2025-05-02 21:49:28 +02:00
Mike Fährmann
a6803afd36 [tests] implement testing for non-existence of a key 2025-05-02 21:10:05 +02:00
Mike Fährmann
e0913c95b2 [twitter] generate 'x-client-transaction-id' header values (#7382) 
TODO: cache ClientTransaction state on disk
 2025-05-02 12:10:05 +02:00
Mike Fährmann
c9123513af [twitter:ctid] integrate 'transaction-id' code (#7382) 2025-05-02 12:08:11 +02:00
Mike Fährmann
bbb1cc719d [chevereto] fix 'date' metadata (#7437) 2025-04-30 11:05:02 +02:00
Mike Fährmann
95ec8c7414 [manganelo:manga] extract more metadata 
- date_updated
- status
- tags
 2025-04-30 09:01:33 +02:00
Mike Fährmann
7b2bcf68a5 [manganelo] support 'nelomanga.net' and mirror domains (#7423) 
- natomanga.com
- nelomanga.net
- manganato.gg
- mangakakalot.gg
 2025-04-29 21:12:37 +02:00
prowlguru
5fa1e13866 [pixiv:novel] fix 'embeds' extraction with AJAX request (#7422 #7435) 
* [pixiv] fix novel embeds with AJAX request

All novel embeds seem to be broken since
`meta-preload-data` is no longer present in the
html of the php response after a site change.

* [pixiv] remove headers used for old novel embed request

* [pixiv] cleanup novel embed fix

* [pixiv] handle novel embed exceptions

* [pixiv] fix PEP formatting for novel embeds fix

* Fix error message

* [pixiv] include exception info in novel error message

---------

Co-authored-by: prowlguru <prowlguru@github.com>
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-04-28 17:14:41 +02:00
Mike Fährmann
16b5d41ce5 [kemonoparty] add 'endpoint' option (#7438) 2025-04-28 16:47:48 +02:00
Mike Fährmann
f6330bf16d [tests] implement explicit type checks for 'len:' 
for example 'len:dict:3' to check for a dict with 3 items
 2025-04-28 16:26:33 +02:00
Mike Fährmann
0725eab34f [deviantart] unescape \' in JSON data (#6653) 
https://github.com/mikf/gallery-dl/issues/6653#issuecomment-2833783238
 2025-04-28 09:42:16 +02:00
Mike Fährmann
9b43a3677e [pp:directory] add 'directory' post processor (#7432) 2025-04-28 09:08:35 +02:00
Mike Fährmann
7386cb78bc [chevereto] extract 'date' metadata (#7437) 2025-04-27 20:54:10 +02:00
Mike Fährmann
19fc4e0ba4 [job] do not reset skip count when 'skip-filter' fails (#7433) 2025-04-27 19:16:02 +02:00
Mike Fährmann
d7c365e08c [scrolller] fix exception for albums with missing media (#7428) 
fixes regression introduced in a44ea6b3d9
 2025-04-27 17:10:52 +02:00
Mike Fährmann
87f02acd9e release version 1.29.5 2025-04-26 17:32:07 +02:00
Mike Fährmann
a44ea6b3d9 [scrolller] support albums for subreddit results (#7339) 
update GraphQL queries
 2025-04-26 16:41:54 +02:00
Mike Fährmann
3ac33dce61 [pixiv:novel] handle exceptions during embeds extraction (#7422) 2025-04-26 15:11:15 +02:00
Mike Fährmann
3a8f98ad03 merge #7414: [pixiv] fix 'user_details' requests not being cached 2025-04-26 14:52:58 +02:00
prowlguru
6306b110fd [pixiv] fix 'user_details' requests not being cached 
Calls to user_detail would sometimes have an
integer user_id and sometimes string. Since this
is used as a key for caching this function, it
would cause an extra request to be made.

All calls to user_detail should be sending user_id
as a string, to avoid having multiple entries in
the cache, causing an extra request to be made.

Authored by: prowlguru
 2025-04-26 14:50:59 +02:00
Mike Fährmann
fd093c2ef8 [architizer] fix 'project' extractor (#7421) 2025-04-26 09:19:43 +02:00