github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,546 Commits 1 Branch 0 Tags
ed317bfcf111354477b06f33e62276e35b9f2bca
Commit Graph

3280 Commits

Author SHA1 Message Date
Mike Fährmann
ed317bfcf1 warn about cookies expiring in less than 24 hours 
requires an expiration timestamp,
so this only works with cookies from a cookies.txt file
 2022-02-13 23:00:49 +01:00
David Hoppenbrouwers
b17e2dcf93 [wallpapercave] add extractor for images (#2205) 2022-02-11 23:44:51 +01:00
v-delta
c661737f36 [Imgbox] Fix ImgboxExtractor (#2281) 2022-02-11 22:17:02 +01:00
Thomas Jost
a7de819aca [lightroom] add Lightroom gallery extractor (#2263) 2022-02-11 21:30:59 +01:00
Mike Fährmann
563bd0ecf4 [danbooru] inherit from BaseExtractor 
- merge danbooru and e621 code
- support booru.allthefallen.moe (closes #2283)
- remove support for old e621 tag search URLs
 2022-02-11 21:01:51 +01:00
Mike Fährmann
bc0e853d30 combine KeyError & IndexError to common base class LookupError 2022-02-11 00:42:49 +01:00
Mike Fährmann
f1c853c6ef [furaffinity] add 'layout' option (#2277) 
to be able to force gallery-dl to parse according to a specific layout
in case its auto-detect fails
 2022-02-11 00:28:47 +01:00
Mike Fährmann
b4f8e15a1f allow BaseExtractors to use the domain pf the matched URL 2022-02-10 01:38:50 +01:00
Mike Fährmann
c0fddcefc5 [downloader:ytdl] make ImportErrors non-fatal (#2273) 2022-02-08 19:30:29 +01:00
Mike Fährmann
a57a44f510 [kemonoparty] handle files without 'name' (fixes #2276) 2022-02-08 18:27:05 +01:00
Mike Fährmann
4efe56f419 [furaffinity] improve new/old layout detection (fixes #2277) 2022-02-08 18:10:52 +01:00
Mike Fährmann
0f1e7ff319 [twitter] fix extraction (#2275) 2022-02-07 23:18:35 +01:00
Mike Fährmann
f351746483 release version 1.20.4 2022-02-06 22:02:36 +01:00
Mike Fährmann
dee0d22561 update extractor test results 2022-02-06 21:39:24 +01:00
Mike Fährmann
d7b8e04b50 [kemonoparty] use 'Accept-Encoding: identity' for all downloads 
(#2267)

fixes issues when data send with 'Content-Encoding: gzip' or other
encodings is larger than the actual file
 2022-02-05 18:06:58 +01:00
enormous-muscles
55326377d8 Add Kohlchan extractor (#2251) 2022-02-04 23:22:17 +01:00
Mike Fährmann
cc7dce5755 [sexcom] add 'pins' extractor (closes #2265) 2022-02-04 20:55:00 +01:00
Mike Fährmann
02e18f56be [e621] add 'favorite' extractor (closes #2250) 2022-02-04 20:54:48 +01:00
Mike Fährmann
70e6e1549e [twitter] provide fallback URLs for card images 
f2e8aedd74 (commitcomment-64057751)
 2022-02-03 23:43:18 +01:00
Mike Fährmann
86fa412b47 [hitomi] add 'format' option (#2260) 
default is 'webp' since downloading original files is no longer allowed
 2022-02-03 23:32:19 +01:00
Mike Fährmann
492436f936 [twitter] add 'warnings' option (#2258) 
disable reporting any non-fatal errors by default
 2022-02-02 18:37:19 +01:00
Mike Fährmann
a5163e4c70 [twitter] restore 'logout' functionality (#1719) 2022-02-02 18:21:15 +01:00
Mike Fährmann
f58364f6a8 update Firefox cipher list 2022-02-01 02:33:01 +01:00
Mike Fährmann
7e6981dda6 rename 'disabletls12' to 'tls12' 
and let config options override any default settings
 2022-02-01 01:37:03 +01:00
Mike Fährmann
bb3e182562 overhaul session initialization 
- share adapter & connection pool across sessions with the same
  ssl options, ssl ciphers, and source address
- simplify browser emulation to just a list of headers and ciphers
 2022-01-31 23:12:08 +01:00
Mike Fährmann
e670dc518e [weibo] update pagination code (fixes #2244) 
- send proper headers and query parameters
- use 'since_id' instead of page numbers
- set a 1-2 second delay between requests
 2022-01-31 19:16:01 +01:00
Robert Pendell
4c651f6252 [patreon] Disable TLS 1.2 by default (#2249) 
Disables TLS 1.2 on Patreon by default.
 2022-01-30 23:30:44 +01:00
Robert Pendell
392cf079f7 Add ability to disable TLS 1.2 (#2243) 
Fix for Patreon Cloudflare issues by having only TLS v1.3 or higher establish HTTPS connections

This now allows you to disable it on a per-host or global basis.  Add disabletls12 as a config option either under extractor.(host) or just under extractor.  Option is false by default.

Example:
        "patreon":
        {
            "disabletls12": true,
            "cookies": {
                "session_id": "X"
            }
        }
 2022-01-30 22:14:43 +01:00
Mike Fährmann
d33227fc38 [twitter] restore errors for protected timelines etc (fixes #2237) 2022-01-30 16:42:13 +01:00
Mike Fährmann
ebd3d5c1cc [bunkr] fix .mp4 downloads (closes #2239) 2022-01-28 23:21:16 +01:00
Mike Fährmann
e2be199124 [gelbooru] improve and fix pagination (#2230, #2232) 
Use 'id:<POSTID' as a tag instead of going through pages with 'pid'.

Something similar was already implemented in 93cef784,
but that got broken again in 3085aac4.
 2022-01-27 17:44:47 +01:00
Mike Fährmann
806badbeec release version 1.20.3 2022-01-26 01:10:44 +01:00
Mike Fährmann
8230f31800 [twitter] update query hashes 2022-01-26 00:49:46 +01:00
Mike Fährmann
c180806cec [twitter] fix deleted/invalid retweets (#2225) 2022-01-25 23:57:13 +01:00
Mike Fährmann
a2eecc6aa8 [kemonoparty] fix DMs extraction (#2008) 2022-01-25 23:16:13 +01:00
Mike Fährmann
2bf554a896 [twitter] fix several errors (#2212, #2216, #2225) 
- fix Tweets with deleted quotes
- fix suspended Tweets without 'legacy' entry
- fix unified_cards without 'type'
 2022-01-25 16:13:22 +01:00
Mike Fährmann
fbd17547f5 release version 1.20.2 2022-01-24 18:24:50 +01:00
Mike Fährmann
e5242b83bf [twitter] define directory format for events (#2109) 2022-01-24 17:44:17 +01:00
Mike Fährmann
efb3e65a6a [sexcom] extend URL pattern (fixes #2220) 2022-01-24 01:19:40 +01:00
vsyx
3f2b6335d7 [instagram] fix highlights extraction (#2197) 
* [instagram] fix highlights extraction

* [instagram] improve highlights extraction

- 'yield' individual reels instead of collecting them in a list
  and returning them all at once
- reduce 'chunk_size' to an even saver value
  (instagram.com also uses 5)
 2022-01-24 00:20:12 +01:00
Mike Fährmann
5ed26e1773 [twitter] fix pinned tweets (#2216) 
caused by the changes in dffa440ede
 2022-01-23 22:52:57 +01:00
Mike Fährmann
a9f78e6527 [twitter] improve error handling 
- handle accounts without 'rest_id'
- handle timelines with empty 'instructions'
 2022-01-23 18:01:05 +01:00
Mike Fährmann
729b07c1f5 [twitter] simplify 
- use dict with common GraphQL variables
- reduce 'variables' size with custom JSON encoder instance
- centralise TwitterAPI() creation
 2022-01-23 01:44:55 +01:00
Mike Fährmann
7cb29224f0 [philomena] fix search parameter escaping (#2215) 
The pluses from search terms in /tags/ URLs need to be
replaced with spaces to get accepted by Philomena.
 2022-01-23 01:03:37 +01:00
Mike Fährmann
9ca8bb2dc0 [twitter] improve error handling 2022-01-22 23:09:45 +01:00
Mike Fährmann
9a221494c3 [twitter] add 'event' extractor (closes #2109) 2022-01-22 20:55:50 +01:00
Mike Fährmann
14867dad6b [twitter] fix unified cards from search results 2022-01-22 20:25:10 +01:00
Mike Fährmann
dffa440ede [twitter] improve handling of deleted tweets (#2212) 2022-01-22 00:41:58 +01:00
Mike Fährmann
54ef874ba4 [twitter] fix retweet filter (#2212) 2022-01-21 23:53:59 +01:00
Mike Fährmann
cb43f7731b [twitter] update to GraphQL API (#2212) 
The old REST API endpoints, which were not used by Twitter since
summer 2021, are going to finally be phased out it seems, with
'/2/timeline/profile/USERID.json' being the first one.

Only Twitter's search doesn't have a GraphQL interface yet.
 2022-01-21 23:34:41 +01:00