github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,804 Commits 1 Branch 0 Tags
01610a6e9e64c548188a630d4bbdbf9f47c19feb
Commit Graph

267 Commits

Author SHA1 Message Date
Mike Fährmann
ada36c2044 [deviantart] update default archive_fmt for single deviations 
(#1874)
use the same as gallery downloads
 2021-09-23 18:02:19 +02:00
Mike Fährmann
44f572c27f [deviantart] implement a 'auto-unwatch' option (#1466, #1757) 2021-09-08 00:27:28 +02:00
Mike Fährmann
47a780942c update extractor test results 2021-09-03 19:36:12 +02:00
Mike Fährmann
96fec14ef7 [deviantart] rename 'watch' option to 'auto-watch' 
(#1466, #1757)

Similar reason as in e05a96db.
'watch' is already used by the WatchExtractor class.
 2021-08-29 21:10:53 +02:00
Mike Fährmann
e75f2de9da [deviantart] add 'comments' option (#1800) 2021-08-29 20:56:34 +02:00
Mike Fährmann
6ce16c6d31 [deviantart] add 'tag' extractor (closes #1803) 2021-08-28 23:42:10 +02:00
Mike Fährmann
ecc8da4704 [deviantart] implement a 'watch' option (#1466, #1757) 2021-08-27 18:39:43 +02:00
Mike Fährmann
a4f249c22e [deviantart] prevent exception on empty videos (fixes #1796) 2021-08-25 20:14:50 +02:00
James C. Wise
1f02878351 [Deviantart] [#1776] Remove the "you need session cookies to download mature scraps" warning (#1777) 2021-08-16 22:50:53 +02:00
Mike Fährmann
bd08ee2859 remove most 'yield Message.Version' statements 
only leave them in oauth.py as noop results
 2021-08-16 03:10:48 +02:00
Mike Fährmann
220cfe244e [deviantart] get original files for GIF previews (#1731) 2021-07-30 00:20:53 +02:00
Mike Fährmann
fad4918208 [deviantart] use UUIDs in internal folder/collection URLs 2021-06-26 00:56:57 +02:00
Mike Fährmann
4e4ca3c330 [deviantart] pin API version (#1611) 
'/gallery/folders' in the newest version doesn't include subfolders.

It probably only needs the right query parameter to do so, but that
doesn't seem to be documented anywhere.
 2021-06-11 00:19:44 +02:00
Mike Fährmann
14f983eab6 [deviantart] use default ID when 'client-id' is None 2021-05-07 16:14:38 +02:00
Mike Fährmann
5a98bcec3a [deviantart] improve folder name matching (fixes #1451) 2021-04-11 20:39:40 +02:00
Mike Fährmann
457abf0e71 [deviantart] fix pagination for Eclipse results (fixes #1444) 
- don't crash on missing keys
- use fallback for invalid 'nextOffset' values
 2021-04-09 15:16:56 +02:00
Mike Fährmann
214ecf62ce [deviantart] fix arguments for search/popular results (#1408) 2021-03-27 18:26:10 +01:00
Mike Fährmann
139fb84108 [deviantart] fix username for 'watch' results (#794) 
before it'd use "/" as username
 2021-03-22 22:14:21 +01:00
Mike Fährmann
91c2e15da9 [deviantart] add support for posts from watched users (#794) 2021-03-22 19:25:04 +01:00
Mike Fährmann
03c20d8c8e [deviantart] update 'watch' URL pattern (#794) 2021-03-21 22:48:06 +01:00
Mike Fährmann
dc23cfd684 [deviantart] use fallback for /intermediary/ URLs 
instead of checking availability with HEAD requests
 2021-03-20 00:10:53 +01:00
Mike Fährmann
b0438c8f99 Revert "[deviantart] extend 'extra' option" 
This reverts commit
5ad2b9c82b,
5c32a7bf58, and
83f465faca.

(#1387, #1356)
 2021-03-19 16:24:23 +01:00
Mike Fährmann
69ca4e29f1 [deviantart] add 'watch' extractor (#794) 2021-03-17 22:50:02 +01:00
Mike Fährmann
83f465faca [deviantart] refactor 'extra' (#1356) 
- change its expected type to string
- let users specify a list of sources (stash, posts) or 'all'
 2021-03-07 22:23:52 +01:00
Mike Fährmann
5c32a7bf58 [deviantart] allow selecting source for 'extra' (#1356) 
Setting 'extra' to "stash" or "deviations" will only download embedded
sta.sh content or deviations. 'true' still downloads both.
 2021-03-06 21:31:28 +01:00
Mike Fährmann
5ad2b9c82b [deviantart] extend 'extra' option 
also download from embedded DeviantArt posts
 2021-02-21 19:27:04 +01:00
Mike Fährmann
23be48427c [deviantart] fix 'folders' option (closes #1302) 
don't assume parent folders are listed before their children
 2021-02-10 17:45:30 +01:00
Mike Fährmann
c6cc86d7d0 [deviantart] update parameters for '/browse/popular' 
- limit results to 50 when also querying metadata (fixes #1267)
- remove deprecated 'category_path' parameter
 2021-01-26 23:35:32 +01:00
Mike Fährmann
c26de0929d [deviantart] provide 'extension' for original file downloads 
(#1272)
 2021-01-25 19:11:00 +01:00
Mike Fährmann
193dca2ce1 update extractor test results 2021-01-21 21:35:42 +01:00
Mike Fährmann
e2d4ca4955 [deviantart] improve '--range' for favorites (closes #1226) 2021-01-08 22:57:35 +01:00
Mike Fährmann
91db8df1c7 [deviantart] add 'index_base36' metadata field (closes #1099) 
This is the same ID as found in 'filename' without the 'd' in front,
which is just 'index' encoded in base36.
 2020-11-07 18:39:50 +01:00
Mike Fährmann
968d3e8465 remove '&' from URL patterns 
'/?&#' -> '/?#' and '?&#' -> '?#'

According to https://www.ietf.org/rfc/rfc3986.txt, URLs are
"organized hierarchically" by using "the slash ("/"), question
mark ("?"), and number sign ("#") characters to delimit components"
 2020-10-22 23:31:25 +02:00
Mike Fährmann
3ebb174f2c add missing extractor info when spawning new ones (fixes #1051) 
Not having this information causes the blacklist/whitelist logic to
trigger and prevents things from functioning as intended when using
default settings.

Fixes issues for 8muses, deviantart, exhentai, and mangoxo.
 2020-10-08 14:34:53 +02:00
Mike Fährmann
136df52d1f [deviantart] support watchers-only/paid deviations (#995) 2020-09-15 16:03:46 +02:00
Mike Fährmann
f6fd449b59 reduce wait time growth rate from exponential to linear 
Waiting for 2**N seconds after each error grows too fast.
Simply waiting N seconds seems far more reasonable.
 2020-09-06 22:38:25 +02:00
Mike Fährmann
c6c06c41f6 [deviantart] don't add journal text to description (#712) 2020-06-05 21:56:12 +02:00
Mike Fährmann
41d03160ff [deviantart] also search journals for sta.sh links (#712) 
when 'extra' is enabled
 2020-06-04 14:47:08 +02:00
Mike Fährmann
dfcf2a2c91 write OAuth token to cache by default (#616) 2020-05-25 22:35:45 +02:00
Mike Fährmann
6294e2c540 add 'text.ensure_http_scheme()' 2020-05-19 22:32:53 +02:00
Mike Fährmann
65b1cb7acd [deviantart] use private access tokens for Journals (fixes #738) 2020-05-08 21:45:01 +02:00
Mike Fährmann
999efec5cc [deviantart] limit API wait times to 2**9=512 seconds (#721) 2020-04-30 21:16:09 +02:00
Mike Fährmann
6386ee54e1 [deviantart] add extractor info to 'following' results 2020-04-16 23:20:07 +02:00
Mike Fährmann
bae1e8ed12 [deviantart] fix JPEG quality replacement pattern 
'q_\d+' would sometimes also replace something in the 'token' query
parameter, invalidating the URL.
 2020-04-11 02:37:06 +02:00
Mike Fährmann
d02f7c1118 improve Extractor.wait() 
- allow 'until' to be a datetime object
- do "time calculations" with UTC timestamps
- set a default 'reason'
 2020-04-05 21:23:05 +02:00
Mike Fährmann
f9a590f92b [deviantart] apply HTTP request limits in more places 
"Request blocked" can also happen on sta.sh and for *any* HTTP
request directed at deviantart.com
 2020-04-03 21:21:59 +02:00
Mike Fährmann
ff7c0b7eff [deviantart] handle "Request blocked" errors (#655) 
- add a 2 second wait time between requests to deviantart.com
- catch 403 "Request blocked" errors and wait for 3 minutes until
  retrying
 2020-04-02 22:14:02 +02:00
Mike Fährmann
c874684f05 [deviantart] retrieve *all* download URLs through OAuth API 
'/extended_fetch' as well as Deviation webpages now again contain
Deviation UUIDs needed to grab Deviation info through the OAuth API,
meaning cookies are no longer necessary to grab original files.

The only instance were cookies are still needed are scraps marked as
"mature", since those entries are hidden for public users.

(#655, #657, #660)
 2020-04-02 22:10:33 +02:00
Mike Fährmann
5c27b25a8f [deviantart] improve sta.sh extraction 
Extract all sta.sh items in a single extractor run.
Don't spawn a new StashExtractor for each individual sta.sh item to
preserve the current requests.Session and its opened TCP connections.
 2020-04-01 03:17:25 +02:00
Mike Fährmann
e2fc4eaa6f [deviantart] detect stash folders (fixes #659) 2020-04-01 01:59:03 +02:00