github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,411 Commits 1 Branch 0 Tags
19e00f13229caabe485c68cbe2b805df2cd2b064
Commit Graph

3411 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
19e00f1322 [dynastyscans] provide 'date' as proper datetime object (#2050) 2021-11-21 22:50:52 +01:00
Mike Fährmann
af6424f398 allow testing metadata in list elements 2021-11-21 22:46:34 +01:00
Mike Fährmann
c67756e187 [kemonoparty] add 'dms' option (#2008) 2021-11-20 23:36:16 +01:00
Mike Fährmann
3842cdcd8f [formatter] implement 'D' format specifier 
To be able to parse any string into a 'datetime' object
and format it as necessary.

Example:

{created_at:D%Y-%m-%dT%H:%M:%S%z}
->
"2010-01-01 00:00:00"

{created_at:D%Y-%m-%dT%H:%M:%S%z/%b %d %Y %I:%M %p}
->
"Jan 01 2010 12:00 AM"

with 'created_at' == "2010-01-01T01:00:00+0100"
 2021-11-20 23:04:34 +01:00
Mike Fährmann
3a7a19c7b9 [dynastyscans] add 'manga' extractor (closes #2035) 2021-11-19 22:51:26 +01:00
Mike Fährmann
9bc83af3a6 [kemonoparty] 'postfile' -> 'file' (#1991) 
to stay consistent with the existing file types for kemono
 2021-11-19 01:50:48 +01:00
Mike Fährmann
522782c09d [subscribestar] emit metadata for posts without media (#1569) 2021-11-18 23:42:17 +01:00
Mike Fährmann
1c8aaf9318 [subscribestar] add 'num' enumeration index (closes #2040) 2021-11-18 23:38:41 +01:00
Mike Fährmann
d433735750 [kemonoparty] skip duplicate files (#2032, #1991, #1899) 
Extract the SHA-256 file hash from URLs
and skip files with the same hash in the same post.

- provide a 'hash' metadata field (empty string if not available)
- remove 'patreon-skip-file' option
 2021-11-17 22:44:15 +01:00
Mike Fährmann
d4ec245554 [kemonoparty] implement a 'files' option (#1991) 
similar to 8d676151
 2021-11-17 22:43:41 +01:00
Mike Fährmann
ab8eea1a24 [twitter] fix extractor for direct image links (fixes #2030) 2021-11-16 22:57:46 +01:00
Mike Fährmann
2076d40681 [ytdl] improve error handling (#1680) 2021-11-15 22:56:42 +01:00
Mike Fährmann
8eaedb0bd3 [ytdl] fix some compatibility issues (#1680) 2021-11-15 22:37:03 +01:00
Mike Fährmann
2aaac3c997 [instagram] include user metadata for 'tagged' downloads (#2024) 
Adds
- tagged_owner_id
- tagged_full_name
- tagged_username
containing the values for the user profile the URL originated from,
e.g. 'instagram' for https://www.instagram.com/instagram/tagged/.
 2021-11-15 21:21:59 +01:00
Mike Fährmann
cfa4876848 [philomena] support furbooru.org (closes #1995) 2021-11-15 20:57:51 +01:00
Mike Fährmann
4377f1c284 [twitter] distinguish between fatal & nonfatal errors (#2020) 
only show a warning for nonfatal errors
and do not raise a StopExtraction exception
 2021-11-13 22:46:40 +01:00
Mike Fährmann
7c50f473e4 [ytdl] document 'cmdline-args' and 'config-file' options (#1680) 2021-11-12 18:54:04 +01:00
Mike Fährmann
19403a7fff [downloader:ytdl] prevent crash in '_progress_hook()' (#1680) 
'speed' is not guaranteed to be defined or convertible to 'int'
 2021-11-12 18:54:04 +01:00
Mike Fährmann
01b28f3674 [ytdl] fix syntax for Python 3.4 2021-11-12 18:54:04 +01:00
Kyle Anthony Williams
a14b72be21 [webtoons] Use swebtoon-phinf.pstatic.net instead of webtoon-phinf.pstatic.net (#2005) 
* [webtoons] Use swebtoon-phinf.pstatic.net instead of webtoon-phinf.pstatic.net

This trick to avoid having to set a Referer header comes from
Webtoon's RSS feeds. The two URLs below are equivalent in content:

https://webtoon-phinf.pstatic.net/20210929_153/1632867980912DmcGK_JPEG/16328679808882705182.jpg?type=q90
https://swebtoon-phinf.pstatic.net/20210929_153/1632867980912DmcGK_JPEG/16328679808882705182.jpg?type=q90

The URL with the domain "webtoon-phinf.pstatic.net" needs a Referer
header, and the domain "swebtoon-phinf.pstatic.net" does not. This
is because of the environment "swebtoon" images live in, one without
explicit network control: RSS feeds on sites such as Feedly. This change should
make it easier for gallery-dl developers to embed Webtoon comics without
worrying about headers.
 2021-11-11 20:03:34 +01:00
Mike Fährmann
6e3658ef52 [kemonoparty] provide 'date' metadata for gumroad (#2007) 
Not the 'published' or 'edited' values since they are 'null',
but still better then nothing at all.
 2021-11-11 19:38:10 +01:00
Mike Fährmann
f4d201f626 [ytdl] fix syntax for Python 3.4 2021-11-09 23:05:12 +01:00
Mike Fährmann
37c9dedee1 [seisoparty] remove module 2021-11-09 22:41:04 +01:00
Mike Fährmann
efa178cc91 [ytdl] implement parsing ytdl command-line options (#1680) 
- adds 'config-file' and 'cmdline-args' options
  for both ytdl downloader and extractor
- create 'ytdl' helper module, which combines YoutubeDL creation
  and option parsing.
- most likely a buggy mess due to incompatibilities between the
  original youtube-dl and yt-dlp.
 2021-11-07 02:44:11 +01:00
Mike Fährmann
a881305357 release version 1.19.2 2021-11-05 20:47:45 +01:00
Mike Fährmann
7cb303d745 [redgifs] improve URL extraction 
Fields inside 'urls' can be None, which would have caused an exception
with the old method.
 2021-11-05 20:02:43 +01:00
Mike Fährmann
2befed1a96 [redgifs] update search URL pattern (#1984) 2021-11-05 20:00:06 +01:00
Mike Fährmann
b315a0ecef [redgifs] update to API v2 (#1984) 2021-11-04 21:31:20 +01:00
Mike Fährmann
f0fc3b0ba1 [kemonoparty] add 'comments' option (#1980) 2021-11-03 23:02:13 +01:00
Mike Fährmann
1fac74b14d [reddit] prevent crash for galleries with no 'media_metadata' 
(fixes #2001)
 2021-11-03 17:55:40 +01:00
Mike Fährmann
2ab190ce08 add tests for special format strings 2021-11-01 23:26:18 +01:00
Mike Fährmann
211de95dd0 update extractor test results 2021-11-01 02:58:53 +01:00
Mike Fährmann
8bea02c38c [deviantart] fix 'index' values for stashed deviations 2021-11-01 01:08:24 +01:00
Mike Fährmann
dd88a7d980 {cyberdrop] restore video extraction (fixes #1993) 
fixes a regression introduced in f33c2ef7
 2021-10-31 04:34:01 +01:00
Mike Fährmann
fa5646eadc [mangoxo] fix login and extraction 2021-10-31 02:16:13 +01:00
Mike Fährmann
4c49174579 [mangakakalot] update domain and fix extraction 2021-10-31 02:16:13 +01:00
ImportTaste
69cd573809 add special type format strings to docs (#1987) 2021-10-30 21:46:26 +02:00
YongChan Cho
14852f7050 [hitomi] fix image path (#1988) 2021-10-30 21:45:01 +02:00
Mike Fährmann
46e17c5e61 support accessing the current local datetime in format strings 
{_now}, {_now:%Y-%m-%d}, etc
(#1968)
 2021-10-30 21:41:09 +02:00
Mike Fährmann
dad2875a3e fix calculating retry sleep times (fixes #1990) 2021-10-29 23:53:48 +02:00
Mike Fährmann
9156e90f1f [twitter] add 'pinned' option 2021-10-29 22:10:58 +02:00
Mike Fährmann
06b414c9a3 [redgifs] 'gfyId' -> 'id' (#1984) 2021-10-29 02:05:39 +02:00
Ryu juheon
d4614e5ba4 [hitomi] fix image URLs (#1982) 2021-10-28 19:29:48 +02:00
Mike Fährmann
6434ccf9e8 [redgifs] split from 'gfycat' (#1984) 
Update API endpoints and metadata names - mostly 'gfycat' -> 'gif' -
and remove some obsolete checks.
 2021-10-28 19:22:41 +02:00
Mike Fährmann
38193dba46 support accessing environment variables in format strings (#1968) 
{_env[HOME]} to get the value of $HOME
every other format string feature is supported as well
 2021-10-28 19:18:55 +02:00
Mike Fährmann
e4696b40ba [instagram] update query hashes 2021-10-27 21:37:31 +02:00
Alice
bfd7401b1e [skeb] add 'user' and 'post' extractors (#1031) (#1971) 
* Create skeb.py

* Update __init__.py

* Update supportedsites.py

* Update supportedsites.md

* Update supportedsites.py

* Update skeb.py
 2021-10-26 20:00:41 +02:00
Ryu juheon
6b6d92d51c [hitomi]: fix image URLs (#1975) 2021-10-26 19:35:01 +02:00
Mike Fährmann
dcb201ff19 [gfycat] show warning when there are no available formats 2021-10-26 19:26:50 +02:00
Mike Fährmann
e436a2607b [gfycat] consistent 'userName' values for 'user' downloads (#1962) 
by using the name from the input URL and not relying on possibly faulty
or incomplete API results.

'userData[username]', if available, will still have the original name.
 2021-10-26 19:15:30 +02:00