github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,787 Commits 1 Branch 0 Tags
a60db454af2ad41549aa37218b44e137900269e2
Commit Graph

266 Commits

Author SHA1 Message Date
Mike Fährmann
a383eca7f6 decouple extractor initialization 
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
 2023-07-25 22:16:16 +02:00
Mike Fährmann
d97b8c2fba consistent cookie-related names 
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
 2023-07-22 01:20:50 +02:00
Mike Fährmann
22099422ca [deviantart] fix shortened URLs (#4316) 2023-07-18 17:55:13 +02:00
Mike Fährmann
98c9fdb414 [deviantart] revert e9353c63; retry downloads with private token 2023-04-23 21:10:16 +02:00
Mike Fährmann
e9353c63d6 [deviantart] keep using private access tokens 
for deviations returned from a private API call

also fixes a bug from 0a7eee3e where '_pagination()'
would never switch from unspecified (None) to private access token
 2023-04-13 14:46:06 +02:00
Mike Fährmann
0a7eee3ee0 [deviantart] add 'public' option 2023-04-08 23:04:34 +02:00
Mike Fährmann
34699fbf64 [deviantart:search] detect login redirects (#3860) 2023-04-03 19:37:12 +02:00
ClosedPort22
c489aecb3e [deviantart] add support for fxdeviantart.com URLs 
fxdeviantart.com is a service that fixes embeds on Discord, similar to
fxtwitter.com
 2023-03-07 16:04:17 +08:00
Mike Fährmann
6b03506655 [deviantart] allow searching when not logged in 2023-02-25 19:04:18 +01:00
Mike Fährmann
3fa456d989 [deviantart] remove mature scraps warning (#3691) 
warn about private deviations
when paginating over eclipse results
 2023-02-23 22:56:37 +01:00
Mike Fährmann
51301e0c31 replace remaining time.sleep() calls 
with Extractor.sleep() or request_interval
 2023-02-23 00:35:37 +01:00
Mike Fährmann
6ed4309aba [deviantart] add 'gallery-search' extractor (#1695) 2023-02-22 17:25:31 +01:00
Mike Fährmann
e1df7f73b1 [deviantart] add 'search' extractor 
(#538, #1264, #2954, #2970, #3577)

Requires login to fetch any results, since the API endpoint raises an
error for not logged in requests.

TODO: parse HTML search results
 2023-02-20 20:54:46 +01:00
Mike Fährmann
b4899c266f merge #3656: [deviantart] fix crash when handling deleted deviations in status updates 2023-02-18 13:11:47 +01:00
Mike Fährmann
725baedad3 [deviantart] use '/collections/all' endpoint for favorites 
(#3666 ,#3668)
 2023-02-17 23:11:35 +01:00
ClosedPort22
f4e211356d [deviantart] slight refactor 2023-02-16 21:23:55 +08:00
ClosedPort22
a74114ef7a [deviantart] fix crash when handling deleted deviations 
in status updates
 2023-02-15 00:17:11 +08:00
Mike Fährmann
7e2fd2e573 merge #3560: [deviantart] add support for /deviation/ and fav.me URLs 2023-01-23 14:16:18 +01:00
ClosedPort22
c90b4ea8d9 [deviantart] add support for fav.me URLs 2023-01-23 20:35:03 +08:00
ClosedPort22
674c719646 [deviantart] refactor base36 conversion 2023-01-23 17:59:55 +08:00
ClosedPort22
293abb8921 [deviantart] add support for /deviation/ URLs 2023-01-23 17:37:57 +08:00
ClosedPort22
597b89245e [deviantart] misc improvements to status extractor 
- relax regex pattern
- handle invalid 'items' field
- add a test for shared sta.sh item

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2023-01-21 13:04:27 +08:00
ClosedPort22
013733c9e9 [deviantart] fix index fields for embedded/shared images 2023-01-19 12:17:27 +08:00
ClosedPort22
c4aeca7a5a [deviantart] improve handling of statuses 
- recursively yield statuses
- ignore items with missing or unexpected field(s)
 2023-01-19 11:48:21 +08:00
ClosedPort22
3b32671fbd [deviantart] add extractor for status updates 
extract user status updates using the '/user/statuses/' endpoint
 2023-01-19 11:30:48 +08:00
Mike Fährmann
77df8d3116 [deviantart] implement username&password login for scraps (#1029) 
re-login when getting prematurely logged out by dA
is missing at the moment
 2023-01-03 18:42:28 +01:00
Mike Fährmann
8d7585534e Merge pull request #3367 from the-blank-x/deviantart-view 
[deviantart] add /view URL support
 2022-12-17 20:05:35 +01:00
blankie
6614d94b08 [deviantart] add /view URL support 2022-12-17 23:27:55 +07:00
ClosedPort22
bf3fd5951a Merge branch 'master' into da-extra-stash 2022-12-10 21:34:37 +08:00
Mike Fährmann
cd931e1139 update extractor test results 2022-12-08 18:58:29 +01:00
ClosedPort22
13d825731e [deviantart] fix test for sta.sh URL extraction 
Without the 'count' assertion, the test would be essentially useless.
 2022-12-07 00:32:03 +08:00
ClosedPort22
6356c9be96 [deviantart] extract sta.sh URLs from 'text_content' 2022-12-06 22:57:53 +08:00
Mike Fährmann
b0cb4a1b9c replace 'text.extract()' with 'text.extr()' where possible 2022-11-05 01:14:09 +01:00
Mike Fährmann
2a1cb403ee Revert "[Deviantart] [#1776] Remove the "you need session cookies to download mature scraps" warning (#1777)" 
This reverts commit 1f02878351.

Mature scraps do yet again require cookies.
 2022-11-01 21:23:05 +01:00
Mike Fährmann
14717f3fc9 [deviantart] add 'group' option (#3018) 
disabling this option allows to better download from deleted accounts
 2022-10-08 15:41:21 +02:00
Mike Fährmann
a12ce2bb41 [deviantart] fix 'deviation' extraction (#2981) 2022-10-05 21:30:08 +02:00
Mike Fährmann
6c76b5f90c [deviantart] fix extraction (#2981, #2983) 
send a 'csrf_token' with every Eclipse API request
 2022-09-29 10:28:39 +02:00
Mike Fährmann
43ec315a7f [deviantart] use public access token for journals (#2702) 
and retry with a private token if needed
 2022-07-29 16:18:09 +02:00
Mike Fährmann
c6a9bab019 update extractor test results 2022-07-12 15:49:22 +02:00
Mike Fährmann
4f7fe9b4be [deviantart] fix folder listings with 'pagination: manual' 
(#2488)
 2022-05-27 18:41:06 +02:00
Mike Fährmann
97e8a15295 [deviantart] implement 'pagination' option (#2488) 2022-04-18 18:08:01 +02:00
Mike Fährmann
1f9a0e2fd8 update extractor test results 2022-04-18 17:24:00 +02:00
Mike Fährmann
b5236656d5 [deviantart] notify users about registering an oauth application 
if they get repeated 429 errors and are using default API credentials
 2022-03-06 16:24:39 +01:00
Mike Fährmann
d441888bfb [deviantart] adjust API endpoints 
Start all endpoints with a forward slash '/'
to be consistent with other API interfaces.
 2021-12-21 00:18:06 +01:00
Mike Fährmann
8f0cf0bf71 [deviantart] use '/browse/newest' for most-recent searches 
(#2096)
 2021-12-20 22:40:03 +01:00
Mike Fährmann
8bea02c38c [deviantart] fix 'index' values for stashed deviations 2021-11-01 01:08:24 +01:00
Mike Fährmann
02a247f4e5 [deviantart] full resolution for non-downloadable images (#293) 
Many thanks to @Ironchest337 for discovering this method
and providing a well-documented implementation.
 2021-10-24 21:11:12 +02:00
Mike Fährmann
a7ddb5f5fa [deviantart] update 'search' argument handling (fixes #1911) 
- use 'alltime' by default
- support newer 'order' values (most-recent, this-week, etc)
 2021-10-23 21:48:02 +02:00
Mike Fährmann
a23f5d45d7 [deviantart] fix bug with fetching premium content (#1879) 
When a user has both 'watchers' and 'paid' folders and one of them is
inaccessible, the other one could get handled as inaccessible as well.
 2021-09-23 18:02:20 +02:00
Mike Fährmann
ada36c2044 [deviantart] update default archive_fmt for single deviations 
(#1874)
use the same as gallery downloads
 2021-09-23 18:02:19 +02:00