github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,201 Commits 1 Branch 0 Tags
ec36df48514a6c7b88708ab22dd80c8ae66febde
Commit Graph

2201 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
ec36df4851 [deviantart] fix video extraction from 'extended_fetch' results 
DeviantArt is now serving videos from wixmp servers (1), instead of
the former film00.deviantart.com (2), even though those URLS are still
functional.

They seem to also have re-encoded those videos. The 10 MB 1080p video
from (2) is now only available in 720p at ~20 MB (with a higher
bitrate, but still …). Other videos are still available in 1080p, but
not this one for some reason.

(Changing the '720p' in (1) to '1080p' doesn't work.)

(1) https://wixmp-ed30a86b8c4ca887773594c2.wixmp.com/v/mp4/9feaa2c9-1baf-4fc2-84f7-f3384b34cefe/d5gxnb5-282a2e9a-b552-40ff-8542-b3c5eed823f5.720p.a837d7cec12c41be8ca2ee53152cea3a.mp4
(2) https://film00.deviantart.net/4c1d/v/mp4/2012/279/d/1/_video____brushes_i_use_in_paint_tool_sai_by_chi_u-d5gxnb5.mp4
 2020-01-30 18:02:21 +01:00
Mike Fährmann
48be2266ed [deviantart] better error message for 'extended_fetch' (#585) 2020-01-30 15:25:33 +01:00
Mike Fährmann
383795b550 prevent superfluous calls to Logger.makeRecord() 
… by setting an appropriate minimal logging level for the root Logger.
 2020-01-30 15:19:06 +01:00
Mike Fährmann
71851a6241 [pixiv] update URLs of followed users to the new format 2020-01-30 15:17:42 +01:00
Mike Fährmann
d086f30b42 [reddit] restore archive keys for i.redd.it images 2020-01-29 22:12:55 +01:00
Mike Fährmann
56f1c96168 implement 'parent-directory' option (#551) 2020-01-29 18:32:37 +01:00
Mike Fährmann
ae07f92f7e [reddit] rewrite extractor logic (closes #551) 
Handle images and videos hosted on Reddit "natively",
allowing them to use reddit-specific metadata to build directory
and file names.
 2020-01-29 17:57:25 +01:00
Mike Fährmann
2852691d78 [paheal] replace test URL 
searching for 'k-on' doesn't yield any results anymore
 2020-01-27 22:19:41 +01:00
Mike Fährmann
2a9be48511 improve util.load/save_cookiestxt() and add tests 
- take a file object as argument instead of an filename
- accept whitespace before comments ("   # comment")
- map expiration "0" to None and not the number 0
 2020-01-25 23:02:15 +01:00
Mike Fährmann
e35c2ea1a6 [weibo] use youtube-dl to download from m3u8 manifests 2020-01-24 23:39:34 +01:00
Mike Fährmann
6703b8a86b [blogger] implement video extraction (closes #587) 2020-01-24 23:37:23 +01:00
Mike Fährmann
b3b5754f2d update test_cookies.py 2020-01-21 22:46:58 +01:00
Mike Fährmann
c1a6862863 implement functions to load/save cookies.txt files (closes #586) 
The methods of the standard libraries' MozillaCookieJar have
several shortcomings (#HttpOnly_ cookies, 0 expiration timestamps, etc.)
and require construction of an ultimately pointless CookieJar object.
 2020-01-21 21:59:36 +01:00
Mike Fährmann
5d73b7f29c release version 1.12.3 2020-01-19 23:31:28 +01:00
Mike Fährmann
37247dbaff miscellaneous fixes 2020-01-19 22:53:06 +01:00
Mike Fährmann
0e9dc5c88e fix AttributeError when accessing 'temppath' 
[ci skip]
 2020-01-19 00:41:21 +01:00
Mike Fährmann
25d5ec4ff3 [twitter] add option to extract TwitPic embeds (#579) 2020-01-18 21:31:29 +01:00
Mike Fährmann
254f7c3999 implement a post-processor module to compare file versions 
(#530)
 2020-01-18 21:08:25 +01:00
Mike Fährmann
32d7195d08 [pinterest] improve detection of invalid pin.it links 2020-01-18 21:06:44 +01:00
Mike Fährmann
0b84068d84 remove temp files before downloading from fallback URLs 
otherwise the next call to download() with a fallback URL could see
the partially downloaded "remains" from the previous, failed download
attempt and "continue" it, writing the second half of a potentially
different version of that file.
 2020-01-18 00:47:17 +01:00
Mike Fährmann
760b9b4db4 add remove_file() and remove_directory() helpers 
these functions call os.unlink() or os.rmdir()
while catching and suppressing potential OSErrors
 2020-01-18 00:21:26 +01:00
Mike Fährmann
b2d542ad40 improve PathFormat._enum_file() 
open only one try-except block for the whole loop,
instead of one for each iteration in os.path.exists()
 2020-01-18 00:21:25 +01:00
Mike Fährmann
174117f827 allow multiple hashes for content tests 2020-01-18 00:19:35 +01:00
Alice
f498a9057f [twitter] Fix stop before real end (#573) 
* [twitter] Fix stop before real end

Fix for https://github.com/mikf/gallery-dl/issues/544. Makes sure that it really reached the end by checking that both "min_position" is null and "has_more_items" is false before stopping.

* [twitter] Fix stop before real end (update)
 2020-01-14 12:24:30 +01:00
Mike Fährmann
8bb32ee188 [hitomi] fix image URLs 2020-01-14 12:04:48 +01:00
Mike Fährmann
bd5ce9855c allow GalleryExtractors to set URL-independent extensions 2020-01-14 11:53:32 +01:00
Mike Fährmann
60a43f0264 fix downloader tests 2020-01-14 11:51:06 +01:00
Mike Fährmann
af42c75152 [mangadex] revert domain to 'mangadex.org' 2020-01-14 09:49:57 +01:00
Mike Fährmann
a6b98d3bec fix auto-generation of supportedsites.rst 2020-01-14 09:48:32 +01:00
Mike Fährmann
3c534d8c2c remove pyOpenSSL from README and setup.py 2020-01-13 22:11:39 +01:00
Mike Fährmann
200aea308a [downloader:common] enable 'job'/'extractor' for logging messages 
(#574)
 2020-01-12 21:41:16 +01:00
Mike Fährmann
e89413da22 update test results 2020-01-12 21:41:06 +01:00
Mike Fährmann
88ccfe984d update mentioned Python version for standalone executables 2020-01-12 20:06:02 +01:00
Mike Fährmann
33a6e0ac6e [hentaifoundry] extract more metadata (closes #565) 2020-01-11 23:22:50 +01:00
Mike Fährmann
5cac79c3d9 [erolord] remove extractor 2020-01-10 14:40:30 +01:00
Mike Fährmann
b9cbf932b4 [pixiv] update URL patterns (fixes #568) 
Pixiv now uses new URLs for
- user profiles and illustration listings:
  - https://www.pixiv.net/en/users/<ID>
  - https://www.pixiv.net/en/users/<ID>/artworks
- bookmarks:
  - https://www.pixiv.net/en/users/<ID>/bookmarks/artworks
 2020-01-10 13:38:09 +01:00
Mike Fährmann
9d369de592 release version 1.12.2 2020-01-05 21:47:13 +01:00
Mike Fährmann
988cc2ec23 [mangadex] change domain to mangadex.cc (closes #559) 2020-01-05 21:21:40 +01:00
Mike Fährmann
f8e137d6b4 [deviantart] show warning about private deviations only once 
… per call to '_pagination()'
 2020-01-05 17:43:05 +01:00
Mike Fährmann
939fec8ecd [deviantart] match new search/popular URLs (closes #538) 2020-01-05 17:32:06 +01:00
Mike Fährmann
09cc88b715 [deviantart] match '/favourites/all' URLs (closes #555) 2020-01-05 13:59:35 +01:00
Mike Fährmann
3811fd8a25 fix time formatting for Python 3.4 and 3.5 
'datetime.time.isoformat()' only has an optional 'timespec' argument
since Python 3.6.
 2020-01-05 00:47:10 +01:00
Mike Fährmann
43ab9572b4 [twitter] handle API rate limits (#526) 2020-01-04 23:46:29 +01:00
Mike Fährmann
569747a78d implement extractor.wait() 2020-01-04 23:42:07 +01:00
Mike Fährmann
5532e9c158 [twitter] handle quoted tweets (#526) 
… and categorize them as retweets
 2020-01-04 21:26:55 +01:00
Mike Fährmann
0b4cb8e57a [mangahere] send 'isAdult' cookie (fixes #556) 2020-01-04 21:25:35 +01:00
Mike Fährmann
025f6e3398 add fallback for missing WITHOUT ROWID support (#553) 2020-01-03 22:58:28 +01:00
Mike Fährmann
87c8b89ddd [postprocessor:metadata] add 'directory' option (#520) 2020-01-02 20:58:10 +01:00
Mike Fährmann
1afb91363c [imagefap] generalize URL patterns and add tests (#552) 2020-01-02 14:26:18 +01:00
Xope Totec
f701e9f33a Handle beta.imagefap.com URLs (#552) 2020-01-02 14:22:00 +01:00