github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,825 Commits 1 Branch 0 Tags
04442e262eedcbfd737bf7778a44cb66e00d2ddf
Commit Graph

93 Commits

Author SHA1 Message Date
Mike Fährmann
ec88ff1562 [flickr] relax unit test results 
Images are now randomly served from the 'live.staticflickr.com' domain
instead of the "old" 'farmN.staticflickr.com' one, making it impossible
to use static 'url' and 'keyword' hashes as results.

Image quality doesn't appear to be effected by which image-server is
used. Files from 'farmN' and 'live' are the same.
 2019-03-30 18:31:59 +01:00
Mike Fährmann
5530871b5a change results of text.nameext_from_url() 
Instead of getting a complete 'filename' from an URL and splitting that
into 'name' and 'extension', the new approach gets rid of the complete
version and renames 'name' to 'filename'. (Using anything other than
{extension} for a filename extension doesn't really work anyway)

Example: "https://example.org/path/filename.ext"

before:
- filename : filename.ext
- name     : filename
- extension: ext

now:
- filename : filename
- extension: ext
 2019-02-14 16:07:17 +01:00
Mike Fährmann
89ee8cd7e4 filter "private" kwdict entries 2019-02-13 13:22:11 +01:00
Mike Fährmann
61741d7333 provide type information for Queue messages 
Child extractors are now directly constructed with Extractor.from_url()
if the extractor class is known beforehand, instead of using
extractor.find() and searching through all possible extractor classes.
 2019-02-12 21:32:32 +01:00
Mike Fährmann
4b1880fa5e propagate 'match' to base extractor constructor 2019-02-11 13:31:10 +01:00
Mike Fährmann
6284731107 simplify extractor constants 
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
 2019-02-08 13:45:40 +01:00
Mike Fährmann
34bab080ae rewrite URL patterns to use only 1 per extractor 2019-02-08 12:03:10 +01:00
Mike Fährmann
9a98b6769d use extractor.request for API calls (#130) 
... at least for OAuth1.0 based APIs (flickr, smugmug, tumblr)
 2018-12-04 21:29:06 +01:00
Mike Fährmann
59bb434ba5 [flickr] add ability to download all albums of a user 
for example with 'https://www.flickr.com/photos/shona_s/albums'
 2018-11-23 09:09:37 +01:00
Mike Fährmann
8080071174 [flickr] improve album metadata (closes #109) 2018-09-29 16:21:55 +02:00
Mike Fährmann
26cbcb3a72 [flickr] improve error handling (#109) 2018-09-17 10:12:14 +02:00
Mike Fährmann
f3793660ef update tests 2018-08-02 14:57:28 +02:00
Mike Fährmann
212130b048 [deviantart] improve public-private token switching 
- rename option to `prefer-public`
- now also works for galleries with less than 24 items
 2018-07-25 12:52:36 +02:00
Mike Fährmann
1c1e086d01 use common base class for OAuth1.0 based API interfaces 2018-05-10 21:57:45 +02:00
Mike Fährmann
6a31ada9e3 re-implement OAuth1.0 code 
OAuth support for SmugMug needs some additional features
(auth-rebuild on redirect, query parameters in URL, ...)
and fixing this in the old code wouldn't work all that well.
 2018-05-10 18:47:05 +02:00
Mike Fährmann
d11fcf4804 smaller changes and fixes 
- fix the cloudflare challenge result if the last decimal places
  are zero (JS`s toFixed() removes trailing zeroes)
- fix downloading of kissmanga chapter-pages hosted on blogspot
  (accessing blogspot with "kissmanga.com" as referrer yields a 401)
- disable certificate validation for 'mangahere' tests
- update flickr test result
 2018-04-06 15:30:09 +02:00
Mike Fährmann
a112e3f2a0 [nijie] add doujin extractor 
adds support for "https://nijie.info/members_dojin.php?id=<artist_id>"
 2018-03-31 18:17:41 +02:00
Mike Fährmann
5008e105ee update archive IDs 
... to behave in a more straightforward way when dealing with
bookmarks/favourites/etc.

specific IDs are now grouped by their owner, album-id, ... to
allow for duplicates when it would be expected.
 2018-03-01 18:20:50 +01:00
Mike Fährmann
34873dbd90 set 'archive_fmt' values 
These are going to be used to create an unique id for each image.
 2018-02-01 15:30:49 +01:00
Mike Fährmann
19a6ae57b2 [sankaku] add pool extractor 2017-12-12 19:45:10 +01:00
Mike Fährmann
035ef655f1 [imagefap] update unit tests 
old gallery/image has been deleted
 2017-10-27 12:22:16 +02:00
Mike Fährmann
393755ee94 [tumblr] update tests 2017-10-09 00:10:37 +02:00
Mike Fährmann
54c0715135 allow users to set their own API access_tokens/client_ids 2017-09-09 17:50:19 +02:00
Mike Fährmann
f7cdfd4c25 add a simplified version of 'parse_qs' 
This version only returns a dict of plain string to string key-value
pairs and ignores multiple values for the same query variable.
 2017-08-24 20:55:58 +02:00
Mike Fährmann
8bcf88bff7 [flickr] fix extraction 
This issue was only noticeable with older Python versions, as these
don't exhibit a consistent ordering of dict keys.
 2017-08-12 21:41:10 +02:00
Mike Fährmann
852e7acd31 [twitter] ignore "Promoted Tweets" 2017-08-06 13:43:08 +02:00
Mike Fährmann
1dac76fd1c update extractor docstrings 2017-06-28 17:39:07 +02:00
Mike Fährmann
e1d82af5e0 small fixes 2017-06-22 18:46:42 +02:00
Mike Fährmann
719d45f89e [flickr] allow the use of Flickr's specifiers for format selection 
- renamed the 'width-max' option to 'size-max'
- filter by both width and height
 2017-06-20 16:09:25 +02:00
Mike Fährmann
e68af4febe [flickr] add 'width-max' option (#16) 
This option allows for simple format selection by
specifying a maximum image width.
 2017-06-18 22:20:15 +02:00
Mike Fährmann
2993206c4b smaller fixes and "security" measures 
- move the OAuthSession class into util.py
- block special extractors for reddit and recursive
- ignore 'only matching' tests for testresults script
 2017-06-16 21:01:40 +02:00
Mike Fährmann
8d5e92f641 resolve cyclic dependency between oauth and flickr 2017-06-14 16:11:18 +02:00
Mike Fährmann
9759fe8c6b allow 'only_matching' tests 2017-06-14 08:43:05 +02:00
Mike Fährmann
05ed95e5b0 [flickr] add search extractor 2017-06-13 08:01:32 +02:00
Mike Fährmann
5f55c854b9 [flickr] replace getPublic... API call with regular ones 2017-06-12 16:37:06 +02:00
Mike Fährmann
9a620784f9 [flickr] add support for user authentication (#16) 
Call '$ gallery-dl oauth:flickr' to get an access_token
and access_token_secret for your account.
 2017-06-12 09:36:14 +02:00
Mike Fährmann
e682e06518 [flickr] add group extractor (#16) 2017-06-06 16:22:30 +02:00
Mike Fährmann
8fd66ef0b3 [flickr] add gallery extractor (#16) 2017-06-06 14:22:36 +02:00
Mike Fährmann
4e80e0c884 [flickr] add user extractor (#16) 2017-06-02 17:15:05 +02:00
Mike Fährmann
b81d068a6d [flickr] add favorites extractor (#16) 2017-06-02 16:35:04 +02:00
Mike Fährmann
72f1c6f87a [flickr] add support for flic.kr/p/... URLs 
Example:
    https://flic.kr/p/FPVo9U
 2017-06-02 09:01:35 +02:00
Mike Fährmann
93e5d8cba3 [flickr] add album extractor 2017-05-31 17:31:51 +02:00
Mike Fährmann
659c65dbb0 [flickr] add image extractor 2017-05-30 17:43:02 +02:00