github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,757 Commits 1 Branch 0 Tags
41191bb60aa73df4e95cf31ede90fba76bc2796f
Commit Graph

62 Commits

Author SHA1 Message Date
Mike Fährmann
41191bb60a 'match.group(N)' -> 'match[N]' (#7671) 
2.5x faster
 2025-06-18 13:05:58 +02:00
Mike Fährmann
e08ec7e083 update copyright notices 2025-06-13 00:03:41 +02:00
Mike Fährmann
87e3cb17e6 [oauth] improve wording when opening URL 2025-02-25 20:12:54 +01:00
Mike Fährmann
2e87a91e97 [oauth] prevent empty browser names 2024-10-01 08:22:50 +02:00
Mike Fährmann
60b4541199 improve a1bb3279, fix oauth:pixiv (#5757) 
Check 'input' option only when required.

This also fixes an exception in oauth:pixiv caused by using the same
'_input' name  as a method defined there.
 2024-06-18 16:50:04 +02:00
Mike Fährmann
ef8f02c3d9 [oauth] use Extractor.request() for HTTP requests (#5433) 
Enables using proxies and general network options.
 2024-04-04 20:51:54 +02:00
Mike Fährmann
89066844f4 add 'config_instance' method 
to allow for a more streamlined access to BaseExtractor instance options
 2024-01-18 03:20:36 +01:00
Mike Fährmann
57fc6fcf83 replace '24*3600' with '86400' 
and generalize cache maxage values
 2023-12-18 23:57:22 +01:00
Mike Fährmann
e256434c9e use custom HTTPBasicAuth class 
to support LazyPrompt as password
and to generate the Authorization header only once
instead of for every request
 2023-12-07 01:40:31 +01:00
Mike Fährmann
17e710c4bf [oauth] warn when cache is enabled but not writeable (#4771) 2023-11-13 16:24:54 +01:00
Mike Fährmann
a453335a9f remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
a383eca7f6 decouple extractor initialization 
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
 2023-07-25 22:16:16 +02:00
Mike Fährmann
076380e079 remove '*' indicating keyword-only arguments 
they are kind of unnecessary and
cause a non-insignificant function call overhead (~10%)
 2023-05-02 22:23:33 +02:00
Mike Fährmann
a05120412a [oauth] catch exception from 'webbrowser.get()' (#3947) 
It raises an exception instead of returning None
when no runnable browser is available.
 2023-04-23 15:00:09 +02:00
Mike Fährmann
b8a702929d [oauth] import extractor modules on demand 2023-03-02 18:39:54 +01:00
Mike Fährmann
dd88740ec7 replace remaining instances of base64 with binascii 2023-03-02 18:25:47 +01:00
Mike Fährmann
75570ad3f1 [oauth] remove stray 'exit()' (#3628) 
- bug from 70ce45d9
- broke oauth:tumblr, oauth:flickr, and oauth:smugmug
 2023-02-13 15:58:34 +01:00
Mike Fährmann
70ce45d965 [oauth] use default name for browsers without 'name' attribute 
(#3645)

Seem to only be an issue for MacOSXOSAScript before Python 3.11.
d12bec6993
 2023-02-11 14:48:59 +01:00
Mike Fährmann
568112dfbb [oauth] improve output 
- show which api key / client id gets used (#3518)
- show in which browser authorization URLs gets opened in
 2023-01-20 12:33:26 +01:00
Mike Fährmann
df91ebb945 [oauth] simplify OAuth 1.0a init 2023-01-19 18:34:54 +01:00
Martin Juhasz
9ea521bec5 [oauth] add host config option (#2806) 2022-08-14 17:08:01 +02:00
Mike Fährmann
688d6553b4 replace calls to print() with stdout_write() (#2529) 2022-05-19 17:09:24 +02:00
Mike Fährmann
e5f6af6e32 [oauth:pixiv] add note about 'code' expiring in 30 seconds (#2306) 2022-02-19 23:47:30 +01:00
Mike Fährmann
4f5f9ed1e5 [oauth] fix typo 
this has been here since February ...
(8974f036)
 2021-09-08 20:14:39 +02:00
Mike Fährmann
96fec14ef7 [deviantart] rename 'watch' option to 'auto-watch' 
(#1466, #1757)

Similar reason as in e05a96db.
'watch' is already used by the WatchExtractor class.
 2021-08-29 21:10:53 +02:00
Mike Fährmann
4718f9c5dd [oauth] use defaults when config values are set to None/null 
(fixes #1778)
 2021-08-21 22:46:01 +02:00
Mike Fährmann
04f4f9badb [oauth] prevent exceptions when reporting errors (#1603) 2021-06-05 14:28:26 +02:00
Mike Fährmann
fbfcbcbf57 Merge branch '1.17.0' 2021-02-15 02:32:33 +01:00
Mike Fährmann
8974f0361c [pixiv] update (#1304) 
- remove login with username & password
- require a refresh token
- add 'oauth:pixiv' functionality

See also:
- https://github.com/upbit/pixivpy/issues/158
- https://gist.github.com/ZipFile/c9ebedb224406f4f11845ab700124362
 2021-02-12 18:07:16 +01:00
Mike Fährmann
1d145a6186 [mastodon] use cache for OAuth tokens (#616) 2021-01-31 01:38:23 +01:00
Mike Fährmann
36bf76fa44 update 'oauth:mastodon:<instance>' code 2021-01-28 02:20:12 +01:00
Mike Fährmann
968d3e8465 remove '&' from URL patterns 
'/?&#' -> '/?#' and '?&#' -> '?#'

According to https://www.ietf.org/rfc/rfc3986.txt, URLs are
"organized hierarchically" by using "the slash ("/"), question
mark ("?"), and number sign ("#") characters to delimit components"
 2020-10-22 23:31:25 +02:00
Mike Fährmann
bdc6c8f074 improve message for 'oauth:deviantart' etc (closes #989) 2020-09-29 21:25:24 +02:00
Mike Fährmann
864f4220d9 update output of 'oauth:…' (#616) 2020-05-31 17:41:40 +02:00
Mike Fährmann
dfcf2a2c91 write OAuth token to cache by default (#616) 2020-05-25 22:35:45 +02:00
Mike Fährmann
c8787647ed add global WINDOWS bool 2020-05-19 22:32:53 +02:00
Mike Fährmann
5d7404ab58 [oauth] use the new name for 'DeviantartAPI' (fixes #670) 2020-04-04 20:34:47 +02:00
Mike Fährmann
913b8333cc write DeviantArt refresh-tokens to cache (#616) 
Writing the token is currently disabled by default and must be
enabled with 'extractor.oauth.cache'.

'extractor.deviantart.refresh-token' must be set to '"cache"'
to use the cached token.
 2020-02-25 22:55:11 +01:00
Mike Fährmann
da0d5f6092 [oauth] add 'port' option (#604) 2020-02-09 13:45:44 +01:00
Mike Fährmann
372ffe95ee [oauth] adjust Flickr redirect URI (fixes #503) 
Flickr now automatically forces https:// for all redirect URIs.
 2019-12-02 22:24:20 +01:00
Mike Fährmann
f5604492c3 update interface of config functions 2019-11-24 00:42:28 +01:00
Mike Fährmann
4409d00141 embed error messages in StopExtraction exceptions 2019-10-28 16:39:49 +01:00
Mike Fährmann
803d8f814e [oauth] update scope for reddit tokens (#428) 
'/user/<username>/...' requires the 'history' scope to be accessible
(https://www.reddit.com/dev/api/#GET_user_{username}_{where})
 2019-09-27 17:38:55 +02:00
Mike Fährmann
a2af2d2965 adjust cache maxage values 2019-03-14 22:21:49 +01:00
Mike Fährmann
4b1880fa5e propagate 'match' to base extractor constructor 2019-02-11 13:31:10 +01:00
Mike Fährmann
6284731107 simplify extractor constants 
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
 2019-02-08 13:45:40 +01:00
Mike Fährmann
b8fed34548 add generalized extractors for Mastodon instances (#144) 
Extractors for Mastodon instances can now be dynamically generated,
based on the instance names in the 'extractor.mastodon.*' config path.

Example:
{
    "extractor": {
        "mastodon": {
            "pawoo.net": { ... },
            "mastodon.xyz": { ... },
            "tabletop.social": { ... },
            ...
        }
    }
}

Each entry requires an 'access-token' value, which can be generated with
'gallery-dl oauth:mastodon:<instance URL>'.
An 'access-token' (as well as a 'client-id' and 'client-secret') for
pawoo.net is always available, but can be overwritten as necessary.
 2019-01-19 14:28:59 +01:00
Mike Fährmann
f3483a2b7c [smugmug] add OAuth support 2018-05-10 18:58:05 +02:00
Mike Fährmann
6a31ada9e3 re-implement OAuth1.0 code 
OAuth support for SmugMug needs some additional features
(auth-rebuild on redirect, query parameters in URL, ...)
and fixing this in the old code wouldn't work all that well.
 2018-05-10 18:47:05 +02:00
Mike Fährmann
0f1e07f627 [pinterest] scrap OAuth implementation; code improvements 
OAuth authentication isn't needed anymore and other tools
like Postman are better suited for this job anyway.
 2018-04-25 16:04:30 +02:00