github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,086 Commits 1 Branch 0 Tags
372ffe95eec44c85bd3e0edbdc8a4bda47fa6cab
Commit Graph

2086 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
372ffe95ee [oauth] adjust Flickr redirect URI (fixes #503) 
Flickr now automatically forces https:// for all redirect URIs.
 2019-12-02 22:24:20 +01:00
Mike Fährmann
004812258d [hentaifox] fix extraction 2019-12-02 22:21:45 +01:00
Mike Fährmann
e2710702d4 fix Cloudflare bypss 2019-12-01 01:07:24 +01:00
Mike Fährmann
8759403f37 [plurk] add delay between comment requests 2019-12-01 01:03:31 +01:00
Mike Fährmann
a28552fd19 update test results 
- hbrowse: one tag got removed
- mangoxo: gallery changed owner
- photobucket: ?, but photo still downloads
 2019-11-30 23:59:32 +01:00
Mike Fährmann
dcaa3d01bd [imagefap] adapt to new image URL format 2019-11-30 23:48:02 +01:00
Mike Fährmann
e62c209ca0 [nijie] fix 'date' parsing 2019-11-30 23:08:21 +01:00
Mike Fährmann
3bba763ab9 [twitter] improve 
- update metadata structure
  - combine all user… entries into their own dict
  - let 'user' always specify the Timeline owner
  - add 'author' entry that specifies the original Tweet author
- create directories per post (closes #491)
- fix username issues with /i/web/ URLs
 2019-11-30 22:30:37 +01:00
Mike Fährmann
26d2334550 [postprocessor:metadata] rename 'format' to 'content-format' 
Just to be consistent with the other 'extension-format' option name,
and only 'format' is also still accepted.
 2019-11-30 17:27:49 +01:00
Mike Fährmann
a412531451 [postprocessor:metadata] implement 'extension-format' option 
closes #477
 2019-11-30 17:26:17 +01:00
Mike Fährmann
0f1538af78 split filename formatting into its own function 2019-11-29 22:32:07 +01:00
Mike Fährmann
db35c3b581 [directlink] separate filenames from paths 
With this, all default filename formats specify an '{extension}'
and PathFormat.set_extension() reliably works for all files.
 2019-11-28 23:50:00 +01:00
Mike Fährmann
41a3169c67 [foolfuuka] use '{extension}' in default filename format 2019-11-28 23:12:48 +01:00
Mike Fährmann
e9aed62c91 [imgur] unescape image titles 2019-11-28 22:13:24 +01:00
Mike Fährmann
bca2222559 add '--exec-after' 2019-11-27 19:42:46 +01:00
Mike Fährmann
ed6592ea1a remove '--abort-on-skip' 2019-11-27 19:41:24 +01:00
Mike Fährmann
2c332edaad [plurk] fix comment pagination 2019-11-27 19:39:56 +01:00
Mike Fährmann
a3fa45bbb1 [behance] get images from 'media_collection' modules 2019-11-27 01:04:33 +01:00
Mike Fährmann
359c3bc1c5 [deviantart] revert to getting download URLs from OAuth API 
This commit (partially) reverts 27b5b24, 94eb7c6, and a437e78.

Download URLs from the 'extended_fetch' endpoint are now only
usable for logged in users, while those from the respective
OAuth API endpoint are working again. Everything except
scraps and direct deviation links should be fixed, and those
two categories will work with exported cookies. (#488)

TODO:
- "native" login with --username and --password
- better handling of internally stored cookies
 2019-11-26 23:29:46 +01:00
Mike Fährmann
42b9633c7e update test results 2019-11-26 23:27:15 +01:00
Mike Fährmann
d4d8c1c01f update links to standalone executables in README.rst 2019-11-26 12:05:31 +01:00
Mike Fährmann
b28bd1c73e [bobx] set generated session cookie (closes #482) 
This reverts commit 490831f and also restores original image downloads
by setting a randomly generated session cookie. No login required.
 2019-11-25 20:04:11 +01:00
Mike Fährmann
ae09f87602 improve SharedConfigMixin config lookups 2019-11-25 18:31:38 +01:00
Mike Fährmann
b5c964332b improve config.py test coverage 2019-11-25 17:20:00 +01:00
Mike Fährmann
f5604492c3 update interface of config functions 2019-11-24 00:42:28 +01:00
Mike Fährmann
4ca883c66f [smugmug] replace test for custom URLs 
The old one (http://www.creativedogportraits.com/) is empty and/or
no longer handled by SmugMug.
 2019-11-22 23:25:55 +01:00
Mike Fährmann
d45fabb79d match user profile handling on deviantart and newgrounds 2019-11-22 23:20:21 +01:00
Mike Fährmann
ea80dadd09 [deviantart] restore archive keys 
Commit 9fdc5e7 changed 'username' fields  to have consistent
capitalization, but that invalidated the archive keys of several
extractors where 'username' was usually lowercase.
 2019-11-21 17:00:08 +01:00
Mike Fährmann
3fc1e12949 [postprocessor:metadata] filter private entries 
i.e. keys starting with an underscore
 2019-11-21 16:58:44 +01:00
Mike Fährmann
ea094692c8 [vsco] fix collection extraction (#480) 2019-11-20 22:06:23 +01:00
Mike Fährmann
490831f84a [bobx] "fix" image download URLs 
Access to original images got restricted to (paid) members only.
All that's publicly accessible now are essentially preview pictures.
 2019-11-20 21:59:37 +01:00
Mike Fährmann
978cb03f81 update misc test results 
- Livedoor now uses https:// for its image URLs
- Instagram image URLs got simplified
 2019-11-20 21:45:48 +01:00
Mike Fährmann
fca87974fe [sexcom] fix video downloads by sending specific Referer headers 2019-11-19 23:52:34 +01:00
Mike Fährmann
bbbeff4c41 [downloader.http] implement file-specific HTTP headers 2019-11-19 23:50:54 +01:00
Mike Fährmann
977026c5ad update .travis.yml 
- remove any 'dist: xenial'
- add Python 3.7 and 3.8 to the default build list
- run "results" tests with Python 3.8
 2019-11-19 17:47:59 +01:00
Mike Fährmann
edc080468d [instagram] make 'video_url' fields optional (fixes #479) 
[ci skip]
 2019-11-19 11:18:43 +01:00
Mike Fährmann
9fdc5e74cb [deviantart] ensure consistent username capitalization (#455) 
The 'username' field was capitalized in a very inconsistent manner:
Either all lowercase, or as given by the input URL, or with the
"original" capitalization, depending on the extractor used among
other things.

Now usernames use their original capitalization for all extractors.
('UserName' instead of 'username' or 'uSeRnAmE')
 2019-11-18 22:09:58 +01:00
Mike Fährmann
b1f0609de5 [newgrounds] rewrite (#394) 
- restructure extractor hierarchy
- extract more metadata
- extract videos without youtube-dl
- be more resilient to errors

TODO:
- favorites
- games, but that might be near impossible for non-flash titles
 2019-11-18 21:13:33 +01:00
Mike Fährmann
3ece3976ae [newgrounds] implement login support (#394) 2019-11-16 23:45:32 +01:00
Mike Fährmann
3a07c06865 [newgrounds] update 
- create directory per post
- rename variables and methods
 2019-11-14 23:17:14 +01:00
Mike Fährmann
5513b66eb0 [vsco] fix user profile extraction 2019-11-12 23:36:48 +01:00
Mike Fährmann
abfcb356fc [flickr] support 3k, 4k, 5k, and 6k photo sizes (closes #472) 2019-11-10 17:52:51 +01:00
Mike Fährmann
521fcd2eb9 [imgbb] fix error in galleries without user info (closes #471) 2019-11-10 17:10:51 +01:00
Mike Fährmann
8061263d4c [imgbb] improve pagination logic 
- avoid unnecessary API calls for small or empty galleries
- combine duplicate code
 2019-11-10 17:07:27 +01:00
Mike Fährmann
da6789b2b0 disable unique archive id checks for some tests 
- same image twice in a livedoor blog post
- unreliable results for related pinterest items
 2019-11-10 17:04:51 +01:00
Mike Fährmann
67e54ed8ea release version 1.11.1 2019-11-09 00:52:06 +01:00
Mike Fährmann
ce98a86c0e fix data file inclusion in source distributions 2019-11-09 00:47:13 +01:00
Mike Fährmann
6c86fbfe2a release version 1.11.0 2019-11-08 22:34:56 +01:00
Mike Fährmann
ba083b30b2 fix snap build 
… hopefully
 2019-11-08 21:44:12 +01:00
Mike Fährmann
94a94f3b86 miscellaneous stuff 2019-11-08 20:58:53 +01:00