github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,413 Commits 1 Branch 0 Tags
c8787647ed403bbcf02c085dc3e221119535471e
Commit Graph

2413 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
5c27b25a8f [deviantart] improve sta.sh extraction 
Extract all sta.sh items in a single extractor run.
Don't spawn a new StashExtractor for each individual sta.sh item to
preserve the current requests.Session and its opened TCP connections.
 2020-04-01 03:17:25 +02:00
Mike Fährmann
e2fc4eaa6f [deviantart] detect stash folders (fixes #659) 2020-04-01 01:59:03 +02:00
Mike Fährmann
c034159701 [piczel] fix extraction for single images 2020-03-31 22:47:23 +02:00
Mike Fährmann
699036ea0c [weibo] accept status URLs with non-numeric IDs (#664) 2020-03-31 22:46:50 +02:00
Mike Fährmann
a0111edfd2 change Travis badge URL to .com 2020-03-31 22:23:53 +02:00
Mike Fährmann
fe96f99e4b [hentainexus] reduce line length (flake8) & update test 2020-03-31 22:08:43 +02:00
墨焓
6f81cac8fa Add metadata to hentainexus: circle, event, title_conventional. (#661) 2020-03-31 21:59:02 +02:00
Mike Fährmann
3ed72f82dc release version 1.13.3 2020-03-28 22:03:33 +01:00
Mike Fährmann
6f911aeb1c [deviantart] add error message for cloudFront blocks (#655) 2020-03-28 21:18:04 +01:00
Mike Fährmann
7499d71d02 [simplyhentai] ignore certificate errors in video test 2020-03-28 21:07:30 +01:00
Mike Fährmann
4203dc0bdc [mangapark] fix metadata extraction 2020-03-28 03:00:26 +01:00
Mike Fährmann
6ecb0a19cf handle sys.stdin being None when using '-' as input file (#653) 2020-03-25 22:33:39 +01:00
Mike Fährmann
c8db2a87e9 fix create_test_data.py script 2020-03-25 22:06:38 +01:00
Mike Fährmann
1b82d36ab2 [deviantart] handle decode errors for extended_fetch results (#655) 
This isn't going to solve the underlying problem, but it should at
least provide the server response when those errors happen.
 2020-03-24 20:56:41 +01:00
Mike Fährmann
88ebbadc58 remove dashes from subcategory names in supportedsites.rst 2020-03-24 02:49:00 +01:00
Mike Fährmann
09f2271528 [35photo] add 'tag' extractor 2020-03-24 02:49:00 +01:00
Mike Fährmann
77fda8190c [35photo] simplify/remove tests for the 'genre' extractor 
There is still a nice genre overview page (https://35photo.pro/genre/)
but the individual sub-pages don't list photos anymore
 2020-03-24 02:48:25 +01:00
Mike Fährmann
4bc161ca0f prevent crash when sys.stdout and co. are None (#653) 2020-03-23 23:38:55 +01:00
Mike Fährmann
d47d0f757c [travis] allow 'results' and 'snap' tests to fail 2020-03-23 23:06:21 +01:00
Mike Fährmann
ce73796eaa [travis] add flake8 job 2020-03-23 22:59:01 +01:00
Mike Fährmann
fb846c9ee5 [instagram] reduce line lengths and make flake8 happy 2020-03-23 22:56:43 +01:00
Mike Fährmann
ad2efa8509 [e621] derive from Danbooru extractors (#651) 
- use extractor implementations from 'danbooru'
- use "page": "b[ID]" to paginate over results instead of
  "tags": "id:<[ID]", avoiding infinite loops with certain
  post orders
- bump User-Agent version
 2020-03-22 21:08:45 +01:00
Mike Fährmann
9b39e1cd7e [e621] fix bug in API rate limiting (#651) 2020-03-22 14:01:23 +01:00
Mike Fährmann
b607d0ad7f [twitter] fix typo in 'x-twitter-auth-type' header (#625) 2020-03-21 23:11:39 +01:00
Mike Fährmann
9159cb8fb3 remove trailing dots and spaces from directory names (#647) 2020-03-19 21:12:18 +01:00
Mike Fährmann
2c3b9e1450 [nozomi] support multiple images per post (#646) 
This changes the default filename format as well as archive IDs,
since those assumed that each post would only have one image.
 2020-03-19 21:07:31 +01:00
Mike Fährmann
c606d0c854 [instagram] update pattern for user profile URLs 
Allow for query parameters and fragments,
for example https://www.instagram.com/instagram/?hl=en
 2020-03-18 22:24:20 +01:00
Mike Fährmann
2530db3f4d [mangadex] transform 'date' timestamps to datetime objects 2020-03-18 02:19:19 +01:00
Mike Fährmann
ae2a33243b [newgrounds] catch general Exceptions 2020-03-18 02:17:43 +01:00
Mike Fährmann
32e36d8f02 [sexcom] replace tests 2020-03-17 22:47:45 +01:00
Mike Fährmann
33b42dc847 [nozomi] sort search results (fixes #646) 2020-03-17 22:28:23 +01:00
Mike Fährmann
eaa60a438b [piczel] fix extraction 
- manually filter by folder_id
- extract data for single posts from embedded JSON, since the
  '/api/gallery/image/<id>' endpoint is no longer available
 2020-03-17 17:12:28 +01:00
Mike Fährmann
5bcc7184c9 [danbooru][e621] increase page limits 2020-03-17 15:53:28 +01:00
Mike Fährmann
90d15e3682 [instagram] use 'itertools.chain()' 2020-03-17 15:52:44 +01:00
Leonardo Taccari
160328d21c [instagram] Add support for user's saved medias (#644) 
* [instagram] Gracefully handle possible 'HttpErrorPage' in _extract_page()

`HttpErrorPage' is returned in shared_data at least  when not authenticated or
when trying to fetch other users saved medias
(i.e. `instagram.com/<user>/saved/').

Gracefully handle it by returning nothing.

* [instagram] Add support for user's saved medias

(Please note that this need the user to be authenticated and they can
only see their saved media (not other users ones).)

Close #643.

* [instagram] Bump copyright year
 2020-03-16 21:09:14 +01:00
Mike Fährmann
e0b0e8d62a release version 1.13.2 2020-03-14 20:19:36 +01:00
Mike Fährmann
5b676ea59d [e621] document username & password support (#640) 2020-03-14 01:15:22 +01:00
Mike Fährmann
1b3ba86110 improve lists in man pages 2020-03-14 01:08:31 +01:00
Mike Fährmann
d3482ace7f [furaffinity] extract more metadata 
- views
- favorites
- comments
- rating
- fa_category (since 'category' is already in use)
- theme
- species
- gender
- width
- height
 2020-03-13 23:56:55 +01:00
Mike Fährmann
f6c5edb76b pre-compile regex pattern for remove_html() and split_html() 2020-03-13 23:31:54 +01:00
Mike Fährmann
fdd2dd5136 [kabeuchi] add 'user' extractor (closes #561) 2020-03-13 16:45:42 +01:00
Mike Fährmann
59edcdc822 [hitomi] restore metadata fields from before f33b13a 
... and add a 'metadata' option to disable
visiting the gallery page and extracting data from it
if this is not needed.
 2020-03-12 23:43:41 +01:00
Mike Fährmann
2d5703c493 [twitter] use a simpler data structure to store cookies in cache 
Use a dict with name-value pairs instead of an entire
RequestsCookieJar object.
 2020-03-12 22:02:12 +01:00
Mike Fährmann
87d4f83597 [newgrounds] make post extraction nonfatal 2020-03-10 01:49:59 +01:00
Mike Fährmann
823fbeaae6 [newgrounds] add 'favorite' extractor (#394) 2020-03-10 01:07:09 +01:00
Mike Fährmann
a45fbc38ea [pixiv] implement 'avatar' option (#595, #623) 2020-03-09 21:18:16 +01:00
Mike Fährmann
a63a376ad2 [mangoxo] fix login 2020-03-08 23:01:51 +01:00
Mike Fährmann
ebc70e87ce [e621] update to new interface / API endpoints (closes #635) 2020-03-06 21:12:58 +01:00
Mike Fährmann
d1cf7ccdb3 [instagram] add 'post_shortcode' metadata field (#525) 2020-03-06 15:20:32 +01:00
Mike Fährmann
402025c3c3 fix some build issues 
- use 'os.name' to decide between Windows/Linux build
- don't check Windows executable version number,
  since Wine fails to run the executable and causes release.sh to stop
 2020-03-06 15:15:59 +01:00