396 KiB
396 KiB
Changelog
1.31.2 - 2026-01-02
Extractors
Additions
- [ahottie] add aupport (#8710)
- [discord] add
server-assetsextractor (#8589) - [imgpv] add
imageextractor (#8773) - [manganelo] add
bookmarkextractor (#8776) - [tiktok] add support for stories, liked posts, saved posts, reposts (#8035 #8466 #8715)
- [twitter] add
notificationsextractor (#7974) - [whyp] add support (#8725)
- [yourlesbians] add
albumextractor (#8713)
Fixes
- [batoto] replace k-subdomain image URLs with n-subdomain (#8791)
- [civitai] update
quality-videosdefault (#8787) - [deviantart:stash] fix
JSONDecodeEerrorfor folders (#8750) - [exhentai] fix possible exception in
finalize()(#8741) - [instagram:stories] extract correct
expiresdates (#8764) - [sankaku] fix re-authentication (#8779)
- [tapas] unescape HTML entities in image URLs (#8790)
- [tumblr:search] prevent
KeyErrorwhen usingoffsetpagination (#8720) - [xenforo] fix incomplete video URLs (#8786)
Improvements
- [bunkr] detect new maintenance video file (#8802)
- [fansly] disable
formatscheck by default (#8757) - [instagram] detect homepage redirects (#8714)
- [instagram] don't warn on minor image size differences (#8300)
- [pixiv] support
sketchinclude (#8789) - [subscribestar] support filterting
userposts bytag(#8737) - [tiktok] remove yt-dlp dependency (#7246 #8466 #8730 #8715)
- [webtoons] download episode background music (#8733)
- [xenforo] support
/#post-IDURLs
Metadata
- [pixiv] provide
countmetadata (#8794) - [tiktok] combine
…_idfields into a singlefile_idone (#8804) - [webtoons] extract
num_play&num_stopfor background music (#8733 #8755)
Options
- [facebook] add
loopoption (#8696) - [fansly] add
previewsoption (#8686) - [koofr] implement
zipoption (#6582 #8700) - [tiktok] add
order-postsoption (#8730 #8715)
Downloaders
- [http] add MIME type and signature for
.aacfiles - [ytdl] fix overwriting
mtimeof downloaded files (#8767) - [ytdl] expect all exception types when extracting
info_dict(#8343)
Miscellaneous
1.31.1 - 2025-12-20
Extractors
Additions
- [2chen] implement generic
2chenboard extractors- support
https://schan.help/(#8680)
- support
- [aryion] add
watchextractor (#8705) - [comedywildlifephoto] add
galleryextractor (#8690) - [koofr] add
sharedextractor (#8700) - [picazor] add
userextractor (#7083 #7504 #7795 #8717) - [weebdex] add support (#8722)
- [xenforo] support
allthefallen.moe/forum(#3249 #8268)
Fixes
- [aryion:favorite] fix extraction (#8705 #8723 #8728)
- [aryion] fix
descriptionmetadata - [boosty] include
Authorizationheader with file downloads (#8704) - [fanbox] make
_extract_post()non-fatal (#8711) - [furaffinity] fix
tagsmetadata (#8724) - [mastodon] fix
AttributeError: 'parse_datetime_iso'(#8709) - [tenor] fix
titlemetadata - [twitter] fix
avatar&backgrounddownloads with"expand": true(#8698)
Improvements
- [boosty] warn about expired
authcookie tokens (#8704) - [misskey] implement
order-postsoption (#8516) - [reddit] use
"videos": "dash"by default (#8657) - [pixiv] warn about invalid
PHPSESSIDcookie (#8689)
Downloaders
- [ytdl] fix
UnboundLocalError: 'tries'(#8707) - [ytdl] respect
--no-skip
Miscellaneous
- [path] implement dynamic length directories (#1350)
- [formatter] add
Iformat specifier - identity - [tests] add
pathtests
1.31.0 - 2025-12-12
Extractors
Additions
- [arena] add
channelextractor (#5847 #8509) - [aryion] add
searchextractor (#8567) - [audiochan] add support (#8602)
- [cfake] add support (#707 #6021 #8549 #8687 #8430)
- [cyberfile] add
sharedextractor (#8323) - [Danbooru] add
media-assetextractor (#8580) - [e621] add
artist&artist-searchextractors (#8448) - [eporner] add support (#8581)
- [fikfap] add support (#8673)
- [fitnakedgirls] add support (#8671)
- [myhentaigallery] add
tagextractor (#8537) - [nudostarforum] add support (#8664)
- [okporn] add support (#8575)
- [pornpics] add
category&listingextractors (#8662) - [pornstarstube] add support (#8576)
- [sexcom] add
feedextractor (#8519) - [shimmie2] support
soybooru.com(#8467) - [sxypix] add support (#4507 #8391 #8574)
- [xenforo] implement generic
XenForoforum extractors
Fixes
- [bellazon] fix errors when handling guest users (#8397)
- [belazon] fix starting from a specific page
- [cien] fix
creator&recentextractors (#8524) - [fanbox:redirect] disable cookie usage (#8565)
- [gofile] fix extraction (#8681 #8683)
- [imagebam] fix
filename&extensionfor names without ext (#8476) - [instagram] fix
AttributeError: 'videos_dash'(#8561) - [motherless] fix
gallery_titleextraction (#8605) - [paheal] fix
AttributeError - [pixiv] fix
KeyError: 'is_bookmarked'(#8398) - [postimg] fix extraction (#8505)
- [rawkuma] update extractors to new site layout (#8568)
- [realbooru] fix
tagsfor video posts (#8455) - [reddit] fix
KeyError: 'media_metadata'for embeds (#8551) - [sankaku][idolcomplex] fix download URLs (#8666)
- [schalenetwork] fix
tagscategories (#8625) - [silverpic] fix extraction & force
.netTLD - [simpcity] fix
contentfor first post of a thread - [simpcity] fix starting from a specific page (#8599)
- [twitter] fix
KeyErrorfortemporarily unavailableusers (#8423) - [twitter] fix
KeyError - 'source_id'with disabledtransform(#8429) - [twitter] fix
AttributeErrorforsearch-pagination": "max_id"(#8613) - [twitter] update & fix
pinnedTweet extraction (#8500) - [vsco] use
"browser": "firefox"by default (#8127) - [webtoons] fix
thumbnailextraction (#8413) - [xasiat] fix
IndexErrorfor albums without category (#8569)
Improvements
- [2ch] support
.orgTLD (#8629) - [bunkr] detect when an album is deleted mid-download (#8619)
- [cyberdrop] update domain to
cyberdrop.cr(#8496) - [cyberfile:folder] support subfolders (#8323)
- [deviantart:gallery] match URLs with query parameters (#8514)
- [discord] limit length of default filenames (#6582)
- [erome] improve error message for deleted & copyrighted albums (#8665)
- [fansly] handle posts without
accountId/contentId/attachments(#8572) - [flickr] extract public API key from website (#7564 #7649 #7700 #8553)
- [imagehost] improve
filename&extensionhandling - [imagetwist] detect deleted images (#8415)
- [imagevenue] improve error for deleted images (#8477)
- [imgbox] match direct links (#8474)
- [imhentai:search] support
/advanced-search/URLs (#8507) - [instagram] improve error for non-existent profiles (#8550)
- [jpgfish] update domain to
jpg7.cr(#8530) - [newgrounds] set error status when encountering inaccessible posts (#8654)
- [realbooru] extract video fallbacks (#8661)
- [reddit] use REST API by default (#8559)
- [reddit] support comment share link (#8434)
- [rule34us:tag] support URLs with empty
qquery parameter (#8546) - [pixhost] force
.toTLD (#8428) - [postimg] force
postimg.ccdomain (#8505) - [sankaku][idolcomplex] support URLs with locale code (#8667)
- [simpcity] implement login with username & password (#8418)
- [simpcity] extract attachment/inline files (#8560)
- [simpcity] extract
click to load mediaURLs (#8609) - [tiktok] ignore empty
musicentries (#8571) - [twitter] restore better user NotFoundError messages (#8621)
- [twitter] implement workarounds for empty
coredata (#8613) - [wikimedia] add
format=originaltofandom/wikiggfile URLs (#5512) - [wikimedia] implement config lookups for
fandom/wikiggsites (#7283)
Metadata
- [bellazon] add
num_internal&num_externalmetadata fields (#8415) - [bellazon] remove query parameters from attachment IDs (#8544)
- [bunkr] extract
album_…metadata for/f/URLs (#8405) - [chevereto] extract
album_id&album_slugmetadata (#8604) - [chevereto:album] extract
count&nummetadata (#8604) - [civitai] implement extracting
tagsmetadata (#8626) - [fanbox] return metadata of inaccessible posts (#8643)
- [hentaifoundry] extract
categoriesmetadata (#8656) - [imagehosts] provide
post_urlmetadata (#6582) - [instagram] extract
subscriptionmetadata for story/highlight items (#8459) - [instagram] extract correct
width&heightfor videos (#8399) - [kemono] improve
filename - [patreon] include full metadata with each URL (#4286 #8498)
- [pixhost] extract
directorymetadata (#6582) - [subscribestar] improve
filename(#8416) - [wikimedia] provide
langmetadata (#7283)
Options
- [bluesky] add
api-serveroption (#8668) - [civitai] add
sort&periodoptions (#8426) - [fanbox:creator] add
offsetoption (#6582) - [itaku] add
orderoption - [mangadex] add
data-saveroption (#8573) - [misskey] add
date-min&date-maxoptions (#8516) - [misskey] add
text-postsoption (#8516) - [patreon] add
order-postsoption (#7856 #8482) - [schalenetwork][hdoujin] re-implement
cbzoption (#8431) - [tiktok] add
coversoption (#8515) - [twitter] add general
limitoption (#8173) - [twitter] implement
retries-apioption (#8317) - [twitter] implement
search-resultsoption (#8613) - [twitter] implement using fallback values for
search-limit(#8173) - [weibo] add
textoption (#8422)
Removals
- [redbust] remove module (#6582)
Common
- allow general ISO 8601 values for
date-min&date-max - use
parentvalue asparent-metadatadefault (#8525 #8604)
Downloaders
- [http] fail downloads of empty files (#8661)
- [http] implement
_http_segmented(#8602) - [ytdl] implement
retryfunctionality (#1131 #8269) - [ytdl] improve error detection
- [ytdl] improve error message when importing default ytdl modules
- [ytdl] update
_extract_manifest() - [ytdl] forward
_ytdl_manifest_headersto formats - [ytdl] restructure code
Post Processors
Formatter
- add
Lbformat specifier -Lfor bytes - add
Xbformat specifier -Xfor bytes (#6582)
Job
- add
keywords-globaloption (#6582) - implement
post-filter&post-rangeoptions - implement
"archive-event": "after"(#8373) - use identity checks
- inline
dispatchloop
Path
- implement conditional
part-directory(#8329) - treat broken symlinks as existing files (#8490)
- improve
exists()performance
Miscellaneous
- [cookies] fix cookie count logging message order (#8414)
- [dt] move datetime utils into separate
dtmodule - [output] add
deferoption for logging files (#8523) - [output] add
Logger.traceback()helper - [scripts/init] fix error when running with default values (#8583)
- [util] support integer values for
…-rangeoptions (#8604) - [workflows:tests] include Python 3.14 & pypy3.11
1.30.10 - 2025-10-12
Extractors
Additions
Fixes
- [bellazon] fix video URL extraction (#8392)
- [bluesky] handle exceptions during file extraction
- [civitai] prevent downloading random posts from deleted users (#8299)
- [girlsreleased] update API endpoints (#8360)
- [instagram] restore
video_dash_manifestdownloads (#8364) - [kemono] prevent fatal exceptions when retrieving user profile data (#8382)
- [mangadex] fix
RuntimeErrorfor titles without adescription(#8389) - [naver-blog] fix video extraction (#8385)
- [poipiku] fix original file downloads (#8356)
- [weibo] fix retrieving followers-only content (#6447 #7939 #8063 #8354 #8357)
- [weibo] use
pageparameter forfeedresults (#7523 #8128 #8357) - [wikimedia] fix name & extension of files without an extension (#8344)
- [wikimedia] ignore missing files (#8388)
Improvements
- [bellazon] ignore links to other threads (#8392)
- [common] disable delay for
request_location() - [fansly] update format selection (#4401)
- [fansly] download user posts from all account walls (#4401)
- [instagram] support
/share/SHORTCODEURLs (#8340) - [weibo] ignore ongoing live streams (#8339)
- [zerochan] forward URL parameters to API requests (#8377)
Metadata
- [instagram] extract
subscriptionmetadata (#8349) - [webtoons] fix
episodemetadata extraction (#2591)
Removals
Post Processors
- [exec] support
{_temppath}replacement fields (#8329)
Miscellaneous
- [formatter] improve error messages (#8369)
- [path] implement conditional
base-directory - use
utf-8encoding when opening files in text mode (#8376)
1.30.9 - 2025-10-03
Extractors
Additions
- [mangafire] add support (#7064 #7701)
- [mangareader] add support (#6529 #6868)
- [patreon] add
collectionextractor (#8286) - [s3ndpics] add support (#8322)
Fixes
- [chevereto] fix
idfor links without file name (#8307) - [chevereto:album] fix video downloads (#8149 #8295)
- [hdoujin] fix
KeyError: 13by addingreclasstag type (#8290) - [misskey] include
withRenotesparameter in API requests (#8285) - [nozomi] percent-encode search tags (#8328)
- [simpcity] fix
KeyError: 'url'when thread author is deleted (#8323) - [twitter] fix
quote_idof individual Tweets (#8284) - [zerochan] prevent
HttpError: '503 Service Temporarily Unavailable'(#8288)
Improvements
- [chevereto] support URLs with
wwwsubdomain (#8149) - [imxto:gallery] support multiple pages (#8282)
- [instagram] add
warn-images&warn-videosoptions (#8283) - [instagram] use
reelsubcategory for/reel/SHORTCODEURLs (#8274) - [instagram] support
/reels/SHORTCODEURLs (#8318) - [paheal] normalize
No resultsoutput message (#8313) - [pixiv] implement searching past 5000 results (#1686 #7082 #8298)
- [thehentaiworld] support more
postURL formats (#8277) - [weibo] download
.m3u8manifests with ytdl (#8339) - [weibo] resolve
wblive-out.api.weibo.comURLs (#8339) - [weibo] use
replay_hdURLs as video fallback (#8339) - [wikimedia] add ability to download image revisions (#7283 #8330)
- [zerochan] normalize
No resultsoutput message (#8313)
Metadata
- [hdoujin] extract
sourcemetadata (#8280) - [instagram] provide
typemetadata (#8274) - [mangadex] extract more manga-related metadata (#8325)
Removals
- [chevereto] remove
img.kiwi
Downloaders
- [http] add MIME type and signature for m3u8 & mpd files (#8339)
Post Processors
- [python] restore
archivefunctionality
Miscellaneous
- [cookies] add support for
Orionbrowser (#8303) - [docker] include more optional Python dependencies (#8026)
- [docs] update
configuration.rstformatting
1.30.8 - 2025-09-23
Extractors
Additions
- [chevereto] support
imglike.com(#5179) - [chevereto] add
categoryextractor (#5179) - [Danbooru] add
randomextractor (#8270) - [hdoujin] add support (#6810)
- [imgpile] add support (#5044)
- [mangadex] add
coversextractor (#4994) - [mangataro] add support (#8237)
- [thehentaiworld] add support (#274 #8237)
Fixes
- [4archive] fix
TypeError(#8217) - [bellazon] fix video attachments (#8239)
- [bunkr] fix
JSONDecodeErrorfor files with URL slugs containing apostrophes'(#8150) - [instagram] ensure manifest data exists before attempting a DASH download (#8267)
- [schalenetwork] fix extraction (#6948 #7391 #7728)
- [twitter] fix quoted Tweets being marked as
deleted(#8225)
Improvements
- [2ch] update domain to
2ch.su, support2ch.lifeURLs (#8216) - [bellazon][simpcity][vipergirls] process threads in descending order (#8248)
- [bellazon] extract
inlineimages (##8247) - [bellazon] support video embeds (#8239)
- [bellazon] support
#comment-12345post links (#8239) - [lensdump] support new direct file URL pattern (#8251)
- [simpcity] extract URLs of
<iframe>embeds (#8214 #8256) - [simpcity] improve post content extraction (#8214)
Metadata
- [facebook] extract
biographymetadata (#8233) - [instagram:tagged] provide full
tagged_…metadata when usingid:…URLs (#8263) - [iwara] extract more metadata (#6582)
- [iwara] make
typeavailable for directories (#8245) - [reddit] provide
commentmetadata for all media files (#8228)
Options
- [bellazon] add
quotedoption (#8247) - [bellazon] implement
order-postsoption (#8248) - [kemono:discord] implement
order-postsoption (#8241) - [simpcity] implement
order-postsoption (#8248) - [vipergirls] implement
order-postsoption (#8248)
Downloaders
- [ytdl] fix errors caused by deprecated options removal
Post Processors
- [metadata] add
"mode": "print"(#2691) - [python] add
"mode": "eval" - close archive database connections (#8243)
Miscellaneous
- [util] define
__enter__&__exit__methods forNullResponseobjects (#8227) - [util] extend list of ISO 639 language codes
1.30.7 - 2025-09-14
Extractors
Additions
- [bellazon] add support (#7480)
- [cyberfile] add support (#5015)
- [fansly] add
creator-mediaextractor (#4401) - [simpcity] add support (#3127 #5145 #5879 #8187)
Fixes
- [aibooru] fix download URLs (#8212)
- [ao3] fix pagination (#8206)
- [boosty] fix extracting
accessTokenfrom cookies (#8203) - [comick] update
buildIdon404errors (#8157) - [facebook] fix
/photo/?fbid=…&set=…URLs being handled as a set (#8181) - [fansly] fix & improve format selection (#4401)
- [fansly] fix posts with more than 5 files (#4401)
- [imgbb] fix & update (#7936)
- [tiktok] fix
KeyError: 'author'(#8189)
Improvements
- [comick] handle redirects
- [fansly] provide fallback URL for manifest downloads (#4401)
- [fansly:creator] support custom wall IDs (#4401)
- [tungsten:user] support filtering results by tag (#8061)
- [twitter] continue searches on empty response (#8173)
- [twitter] implement various
search-…options (#8173)
Miscellaneous
- [formatter] exclude
<>\characters from!Rresults (#8180) - [formatter] support negative indicies
- [util] emit debug
Proxy Maplogging message (#8195)
1.30.6 - 2025-09-06
Extractors
Additions
- [chevereto] add
videoextractor (#8149) - [comick] add
coversextractor - [fansly] add support (#4401)
- [instagram] add
stories-trayextractor (#6582) - [shimmie2] support
co.llection.pics(#8166) - [tungsten] add support (#8061)
- [vk] add
wall-postextractor (#474 #6378 #8159)
Fixes
- [bunkr] fix downloading albums with more than 100 files (#8150 #8155 #8175)
- [chevereto:user] fix names starting with an
a(#8149) - [common] prevent exception when using empty
user-agent(#8116) - [deviantart:search] fix extraction (#8083)
- [hentaifoundry:story] fix
src&descriptionextraction (#8163) - [imagebam] update guard page bypass cookies (#8123)
- [kemono] fix
.binarchive files not being added to archives list (#8156) - [reddit] fix
TypeaErrorwhen processing comments (#8139) - [tumblr] fix pagination when using
date-max - [twitter] prevent exceptions in
_transform_community()(#8134) - [twitter] prevent
KeyError: 'name'in_transform_user()(#8154) - [twitter] fix
KeyError: 'core'when processing communities (#8141) - [zerochan] fix
500 Internal Server Errorduring login (#8097 #8114)
Improvements
- [comick] detect broken chapters (#8054)
- [erome] handle reposts on user profiles (#6582)
- [instagram] improve video quality warning regex (#8078)
- [jpgfish] update domain to
jpg6.su - [reddit] add
api&limitoptions (#7997 #8012 #8092) - [reddit] support video embeds (#8139)
- [tumblr:tagged] support
/archive/tagged/URLs (#8160)
Metadata
- [khinsider] extract
descriptionmetadata - [tumblr:tagged] provide
search_tagsmetadata (#8160) - [vk] parse
date&descriptionmetadata (#8029) - [vk:album] extract more metadata (#8029)
Downloaders
- [ytdl] implement
_ytdl_manifest_cookies
Miscellaneous
- [formatter] add
Rconversion - extract URLs (#8125) - [options] add
-aas short option for--user-agent - [scripts/init] implement
-s/--subcategory
1.30.5 - 2025-08-24
Extractors
Additions
- [shimmie2] support
noz.rip/booru(#8101) - [sizebooru] add support (#7667)
- [twitter] add
highlightsextractor (#7826) - [twitter] add
homeextractor (#7974)
Fixes
- [aryion] fix pagination (#8091)
- [rule34] support using
api-key&user-id(#8077 #8088 #8098) - [tumblr:search] fix
ValueError: not enough values to unpack(#8079) - [twitter] handle
KeyError: 'result'for retweets (#8072) - [zerochan] expect
500 Internal Server Errorresponses for HTML requests (#8097)
Improvements
- [civitai:search] add
tokenoption (#8093) - [instagram] warn about lower quality video downloads (#7921 #8078)
- [instagram] remove
candidateswarning (#7921 #7989 #8071) - [oauth] improve error messages (#8086)
- [pixiv] distinguish empty from deleted profiles (#8066)
- [twitter] update API endpoint query hashes & parameters
Metadata
- [batoto] extract more metadata (#7994)
- [instagram:highlights] extract
author&owner&usermetadata (#7846) - [newgrounds] extract
slugmetadata (#8064) - [twitter] extract
communitymetadata (#7424)
Removals
Downloaders
- [ytdl] improve playlist handling (#8085)
Scripts
- implement
rmhelper script - add
-g/--gitcommand-line options - [util] add
git()&lines()helper functions
Miscellaneous
- [config] add
confargument toconfig.load()(#8084)
1.30.4 - 2025-08-16
Extractors
Additions
- [civitai] add 'videos' extractor (#6644)
Fixes
- [civitai] fix posts not returning video files (#8053)
- [civitai] fix '403 Forbidden' errors for searches
- [kemono] use 'Accept: text/css' for API requests (#8047 #8057)
Improvements
- [newgrounds] add fallback for images with empty 'full_image_text' 'src'
Miscellaneous
- fix accessing methods through 'path-metadata' proxy (#6582)
1.30.3 - 2025-08-15
Extractors
Additions
- [booth] add support (#7920)
- [civitai] add
collection&user-collectionsextractors (#8005) - [facebook] add
infoextractor (#6582) - [facebook] add
albumsextractor (#7848) - [imgdrive] add
imageextractor (#7976) - [imgtaxi] add
imageextractor (#8019) - [imgwallet] add
imageextractor (#8021) - [picstate] add
imageextractor (#7946) - [silverpic] add
imageextractor (#8020) - [tumblr] add
following&followersextractors (#8018) - [xasiat] add support (#4161 #5929 #7934)
Fixes
- [blogger] fix video extraction (#7892)
- [comick] handle chapters without chapter data (#7972)
- [comick] handle volume-only chapters (#8043)
- [comick] fix exception when filtering by translation group (#8045)
- [deviantart:tiptap] fix
KeyError: 'attrs'(#7929) - [everia] fix image extraction (#7973 #7977)
- [facebook] fix
avatarextraction for empty profiles (#7962) - [facebook] handle profiles without photos or
set_id(#7962) - [fappic] rewrite thumbnail URLs (#8013)
- [idolcomplex] update to new domain and interface (#7559 #8009)
- [kemono][coomer] fix extraction (#8028 #8031)
- [kemono] update
/creatorsendpoint (#8039 #8040) - [kemono] don't set error status for posts without comments (#7961)
- [pixiv] fix
IndexErrorfor unviewable works (#7940) - [pixiv] fix artworks downloads when using expired cookies (#7987)
- [scrolller] fix NSFW subreddit pagination (#7945)
- [twitter] fix potential
UnboundLocalErrorwhenvideosare disabled (#7932) - [vsco] disable TLS 1.2 cipher suites by default (#7984 #7986)
- [wikimedia:wiki] fix
AttributeError: 'subcategories'(#7931)
Improvements
- [aibooru] support
general.aibooru.online&aibooru.download - [comick] add
langoption (#7938) - [hentaifoundry] add
descriptionsoption (#7952) - [facebook] raise
AuthRequiredfor profiles requiring cookies (#7962) - [instagram] warn about lower quality image downloads (#7921)
- [kemono] support
"endpoint": "posts+"for full metadata (#8028) - [misskey] support
misskey.art(#7923) - [motherless] detect
404/File not foundpages - [pixiv] detect suspended/deleted accounts (#7990)
- [pixiv] improve API error messages
- [pixiv] remove redundant cookies initialization code
- [scrolller] limit
titlelength in default filenames - [skeb] implement
includeoption (#6558 #7267) - [vk] update default
archive_fmt(#8030)
Metadata
- [cien] provide
author[id]metadata (#6582) - [dankefuerslesen] extract more metadata (#7915)
- [dankefuerslesen:manga] fix metadata being overwritten
- [facebook] ensure numeric
user_idvalues (#7953) - [facebook:set] fix/improve
user_idextraction (#7848) - [fappic] fix
filenamevalues
Common
- [common] implement
"user-agent": "@BROWSER"(#7947) - [common] improve error message for non-Netscape cookie files (#8014)
Downloaders
- [ytdl] don't overwrite existing
filenamedata (#7964)
Miscellaneous
- [docs/configuration] improve
client-id&api-keyinstructions - [docs/formatting] update and improve
- [job] apply
extension-maptoSimulationJobresults (#7954) - [job] improve URL
schemeextraction performance - [job] split collected DataJob results
- [path] implement
path-convertoption (#493 #6582) - [scripts] improve and extend
init,generate_test_result, andpyprint - extend
-A/--abort&"skip": "abort"functionality (#7891) - use more f-strings (#7671)
1.30.2 - 2025-07-27
Extractors
Additions
- [itaku] add
posts&bookmarksextractors (#7707)
Fixes
- [kemono] support new
kemono.crdomain (#7902 #7909 #7911 #7913 #7904) - [coomer] support new
coomer.stdomain (#7907 #7909 #7911 #7904)
Post Processors
- [exec] use
Falseasstart_new_sessiondefault to avoid aTypeError(#7899)
Miscellaneous
- [tests/postprocessor] fix
TypeErrorwhen logging an error (#6582)
1.30.1 - 2025-07-26
Extractors
Additions
- [civitai] add
generatedextractor (#7796) - [facebook] add
avatarextractor (#7848) - [imgadult] add
imageextractor (#7893) - [itaku] add
following&followersextractors (#7707) - [leakgallery] add support (#7872)
- [madokami] add
mangaextractor (#7828)
Changes
- [civitai] change default
userincludes to["user-images", "user-videos"](#7874)
Fixes
- [behance] fix
403 Forbiddenerrors by using"browser": "firefox"(#7803 #7877) - [civitai] fix
AttributeErrorwhen a file's post was deleted (#7860) - [pornhub] fix
galleryextractor (#7842) - [readcomiconline] force
One pagereading mode (#7890) - [sexcom] update
searchextractor (#7807) - [urlgalleries] fix extraction (#7858)
- [wikimedia] add missing
selfargument when callingprepare()(#7835)
Improvements
- [4chan] detect files containing only null bytes (#7883)
- [azurelanewiki] prevent Anubis challenge
- [bilibili] warn about blocked articles (#7880)
- [civitai] fix
extensionfor videos withoutnameandmimeType - [common] detect Cloudflare & DDoS-Guard challenge pages in
request_json()&request_xml()(#7833) - [facebook] add retries to profile page requests (#7725 #7834 #7852)
- [facebook] implement
includeoption (#7848) - [itaku] implement
includeoption (#7707) - [patreon] implement
cursorsupport (#7856) - [patreon] support
date-maxfor/homeURLs (#7856) - [pixiv] improve AJAX error messages (#7896)
Metadata
- [behance] provide
creator[name]metadata (#7885) - [civitai] ensure
file&postdata has adatevalue (#7548) - [inkbunny] enable
poolmetadata (#7850) - [nhentai] provide
gallery_idfor pagination results (#7868)
Downloaders
- [ytdl] add
deprecationsoption
Post Processors
- [exec] add
sessionoption (#6582)
Snap
- migrate base to
core22(#7841) - switch to
yt-dlp(#7865) - fix deprecated
CRAFT_ARCH_TRIPLETusage (#7866)
Formatter
Miscellaneous
- [actions] fix
parse_loggingimport (#7837) - [options] add
--sleep-429command-line option (#7871) - [scripts] ensure files use
utf-8encoding and\nnewlines (#7872) - [tests/extractor] improve example URL mismatch error message (#7872)
- [tests/results] fix
#logchecks for URLs raising exceptions - fix exit status for requests'
JSONDecodeError(#4380) - use walrus operators
:=inifstatements (#7671)
1.30.0 - 2025-07-15
Changes
- raise minimum supported Python version to 3.8 (#7671)
- update extractor names (#7746 #7612)
-
Old New kemonopartykemonocoomerpartycoomerkoharuschalenetworknavernaver-blogchzzknaver-chzzknaverwebtoonnaver-webtoonpixiv:novelpixiv-novel:novelpixiv:novel-userpixiv-novel:userpixiv:novel-seriespixiv-novel:seriespixiv:novel-bookmarkpixiv-novel:bookmark - config settings will automatically use the old values
- target directories using
{category}will use the new category names by default - use
--compator"category-map": "compat"to restore oldcategorynames
-
- include exit status bitmasks of
NotFoundError(8) andNoExtractorError(64) into generalHttpError(4) andInputError(32) respectively
Extractors
Additions
- [civitai] add
search-imagesandpostsextractors (#7609) - [comick] add support (#1825 #6782)
- [dankefuerslesen] add support (#7669)
- [dynastyscans] add
anthologyextractor (#7627) - [girlsreleased] add support (#6200)
- [girlswithmuscle] add support (#4493 #6016)
- [iwara] add support (#2652 #5840 #7785)
- [kemono] add
artistsextractor (#7582) - [misskey] add
avatar,background, andinfoextractors (#5347) - [motherless] add
groupextractor (#7774 #7787) - [naver-chzzk] add
commentandcommunityextractors (#7735 #7741) - [nudostar] add support (#5735 #6556)
- [rawkuma] add support (#4571)
- [redbust] add support (#6759 #6918 #7043)
Fixes
- [4archive] fix
threadextractor - [arcalive] fix download URLs (#7678)
- [arcalive] replace
ac-p.namusubdomains withac-o.namu(#7556) - [archivedmoe] fix redirection issue (#7652 #7653 #7664)
- [aryion] fix
favoriteextractor (#7775) - [batoto] fix downloading manga with alerts/notices (#7657)
- [behance] fix
403 Forbiddenerrors (#7710) - [bunkr] fix file downloads (#7747)
- [civitai] fix & update
searchextractor (#7609) - [danbooru] fix Ugoira conversions for posts without
ZIP:ZipFileName(#7630) - [deviantart:tag] fix
username(#7587) - [deviantart:tiptap] fix
TypeErrorwhentextAlignis null (#7639) - [directlink] fix config lookups by subcategory (#6582)
- [discord] support forwarded messages & handle missing threads (#7706 #7722)
- [furaffinity] fix
submissionsresults (#7759) - [hitomi] fix negative tag searches (#7694)
- [kemono] fix tagged creator posts
- [mangadex:list] fix config lookups for
list-feedsubcategory - [nijie] fix file extraction (#7624)
- [paheal] fix
404 Not Founderror for tags with URL encoded characters (#7642) - [patreon] send
Refererheader when downloading.m3u8videos (#7571) - [patreon] fix
campaign_idextraction from Next.js 13 creator pages (#7773) - [readcomiconline] fix extraction (#7606 #7789)
- [reddit] fix archive IDs of fallback files (#7760)
- [rule34] fix file downloads (#7697)
- [sankaku] fix extracting extended tag categories (#7744)
- [sexcom] prevent
.cssfile downloads (#7632) - [skeb] fix
KeyError - 'frame_rate'(#7798) - [tiktok] handle exceptions when extracting avatars (#7682)
- [vsco] fix JSON returned by VSCO (#7821)
- [warosu] HTML attribute fix (#7676 #7677 #7777)
Improvements
- [artstation] support downloading
.mviewfiles (#7812) - [civitai] support "My Reactions" results for videos (#7608)
- [e621] support
e621.cc/postsURLs (#6809) - [erome] restructure extractor hierarchy (#7804)
- [everia] prevent redirect when fetching post pages
- [exhentai] ensure file signature bytes aren`t all zero (#4902)
- [exhentai] implement
"source": "metadata"(#4902) - [fanbox] return
fileMapfiles in order (#2718) - [gelbooru] improve error message for
401 Unauthorizedresponses (#7674) - [imagevenue] detect
404image files (#7570) - [instagram] provide more descriptive URLs for
video_dash_manifestvideos (#7631) - [pinterest] support
pin.itredirects to board (#7805) - [pinterest] match board URLs with query strings (#7805)
- [rule34us] prioritize
video.rule34.usfor video downloads (#6582) - [rule34xyz] implement login with username & password (#7736)
- [sankaku] allow passing cookies (#7333)
- [sexcom] support
/pics/URLs (#7611) - [tiktok] detect
loginpage redirects (#7716) - [vk] detect
challengepage redirects (#7650) - [vk] prevent
404 Not Founderrors for file downloads - [vk] add continuation message (#7650)
- [warosu] detect missing images by checking hostname (#7698 #7699)
- [ytdl] set domain as subcategory when using
Genericextractor (#6582)
Metadata
- [civitai] always provide
file[…]metadata (#7548) - [everia] improve
filenameby unquoting URLs (#7620) - [fanbox] extract
archivesmetadata (#7454) - [gelbooru_v02] extract
total/search_countmetadata (#7689) - [instagram] provide
post_urlfor stories and highlights (#7810) - [kemono:discord] update server & channel metadata (#7569)
- [mangaread] fix
manga_altmetadata - [newgrounds] filter
<script>content intags(#7604) - [patreon] return metadata for paywalled posts (#6582)
- [pinterest] remove excess whitespace from
descriptionfields (#4335) - [pixiv] remove
/jump.phpfromcaptionlinks (#4327) - [tenor] extract more metadata
- [twitter] extract
source_idandsource_usermetadata (#7470 #7640) - [twitter] extract
sensitive_flagsmetadata (#2523) - [vk] fix
usermetadata extraction
Options
- [civitai] add option to retrieve
postmetadata (#7548) - [exhentai] add
limits-actionoption (#6504) - [fanbox] add
fee-maxoption (#7726) - [kemono] extend
duplicatesoption (#7696) - [mangadex] allow
ratingsto be a (comma-separated) string (#7799) - [misskey] add
includeoption (#5347) - [sankaku] remove
id-formatoption (#5073 #6808) - [webtoons] add
bannersandthumbnailsoptions (#6468 #7441)
Common
- update
browserUser-Agents and headers - allow using predefined Firefox/Chrome
headers&ciphers - allow overriding
user-agentwhenbrowseris used (#7647) - support using system certificates via
truststore(#6582) - fix URLs not getting written to
-e/--error-file(#7758) - raise ChallengeError for Cloudflare & DDoS-Guard challenge pages (#1945)
- prevent exceptions for for non-fatal requests (#7598)
- simplify
userextractors by usingDispatchmixin - allow
GalleryExtractorinstances to return additional asset files
Removals
- [mangasee] remove module
Downloaders
- support dynamic download
ratelimits (#7638) - [http] fail downloads with HTML content (#4798 #7697)
- [http] add MIME type and signature check for
.htmlfiles - [http] fix potential
FileExistsErrorwhen.partfile moved (#5385) - [http] implement
_http_signaturechecks (#4902) - [ytdl] fix
KeyError - 'filepath'when using legacyyoutube_dl(#6949 #7752 #7824) - [ytdl] fix postprocessing/merge errors (#7581)
- [ytdl] detect
yt-dlpindependent of module name (#7599) - [ytdl] support custom headers when fetching HLS/DASH manifests
Post Processors
- implement shortcuts for
modeandeventoptions (e.g.metadata/jsonl@post) - [exec] implement
commandsoption
Options
- add
category-mapandconfig-map(#7612) - add
signals-actions(#1861 #6582) - add
--compatcommand-line option - add
--Printcommand-line option - swap
--printand--Printsemantics
Cookies
- add native support for LibreWolf profiles (#4101 #7625)
- improve cookie-related logging messages
- update expired cookie messages (#7644)
Formatter
- add
Dconversion - ISO 8601 string todatetime - add
Lconversion - ISO 639-1 code to language name - change old
L/length conversion ton - implement
Mformat specifier
Optimizations
- replace
%-formatted and.format(…)strings withf-strings(#7671) - improve regular expression usage
- replace
match.group(N)withmatch[N](#7671) - update
match.lastindexusage - remove pre-3.8 workarounds (#7671)
- remove
@staticmethoddecorators - add
request_json()andrequest_xml()functions - slightly improve performance of
extractfunctions - slightly improve
filterperformance
Miscellaneous
- [actions] add
flagandraiseactions - [job] refactor parent-child config path building (#7527)
- [job:data] wrap exceptions in a dict (#7723)
- [path] support character sequences in
path-restrictreplacements (#1707) - [pyinstaller] exclude
pkg_resourcesmodule (#7592) - [scripts/init] add scripts to generate initial extractor code and test results
- [scripts/options] make output width independent of terminal size
- fix Last-Modified mtime overwriting post processor mtime (#7529)
- use
pipformake install(#7628)
1.29.7 - 2025-05-23
Extractors
Additions
Fixes
- [bluesky] handle posts without
recorddata (#7499) - [civitai] fix & improve video downloads (#7502)
- [civitai] fix exception for images without
modelVersionId(#7432) - [civitai] make metadata extraction non-fatal (#7562)
- [fanbox] use
"browser": "firefox"by default (#7490) - [idolcomplex] fix pagination logic (#7549)
- [idolcomplex] fix 429 error during login by adding a 10s delay
- [instagram:stories] fix
post_datemetadata (#7521) - [motherless] fix video gallery downloads (#7530)
- [pinterest] handle
story_pin_product_sticker_blockblocks (#7563) - [subscribestar] fix
contentandtitlemetadata (#7486 #7526)
Improvements
- [arcalive] allow overriding default
User-Agentheader (#7556) - [fanbox] update API headers (#7490)
- [flickr] add
infooption (#4720 #6817) - [flickr] add
profileoption - [instagram:stories] add
splitoption (#7521) - [mangadex] implement login with client credentials
- [mangadex] send
Authorizationheader only when necessary - [mastodon] support Akkoma/Pleroma
/notice/:IDURLs (#7496) - [mastodon] support Akkoma/Pleroma
/objects/:UUIDURLs (#7497) - [pixiv] Implement sanity handling for ugoira works (#4327 #6297 #7285 #7434)
- [twitter:ctid] reduce chance of generating the same ID
Metadata
- [civitai] provide proper
extensionfor model files (#7432) - [flickr] provide
license_namemetadata - [sankaku] support new
tagscategories (#7333 #7553) - [vipergirls] provide
numandcountmetadata (#7479) - [vipergirls] extract more metadata & rename fields (#7479)
Downloaders
Cookies
1.29.6 - 2025-05-04
Extractors
Additions
- [manganelo] support
nelomanga.netand mirror domains (#7423)
Fixes
- [deviantart] unescape
\'in JSON data (#6653) - [kemonoparty] revert to using default creator posts endpoint (#7438 #7450 #7462)
- [pixiv:novel] fix
embedsextraction by using AJAX API (#7422 #7435) - [scrolller] fix exception for albums with missing media (#7428)
- [twitter] fix
404 Not Found ()errors (#7382 #7386 #7426 #7430 #7431 #7445 #7459)
Improvements
- [kemonoparty] add
endpointoption (#7438 #7450 #7462) - [tumblr] improve error message for dashboard-only blogs (#7455)
- [weasyl] support
/view/URLs (#7469)
Metadata
- [chevereto] extract
datemetadata (#7437) - [civitai] implement retrieving
modelandversionmetadata (#7432) - [manganelo] extract more metadata
Post Processors
- [directory] add
directorypost processor (#7432)
Miscellaneous
- [job] do not reset skip count when
skip-filterfails (#7433)
1.29.5 - 2025-04-26
Extractors
Additions
- [bluesky] add
videoextractor (#4438) - [instagram] add
followersextractor (#7374) - [itaku] add
starsextractor (#7411) - [pictoa] add support (#6683 #7409)
- [twitter] add
followersextractor (#6331)
Fixes
- [architizer] fix
projectextractor (#7421) - [bluesky:likes] fix infinite loop (#7194 #7287)
- [deviantart] fix
401 Unauthorizederrors for for multi-image posts (#6653) - [everia] fix
titleextraction (#7379) - [fanbox] fix
commentsextraction - [fapello] stop pagination on empty results (#7385)
- [kemonoparty] fix
archivesoption (#7416 #7419) - [pixiv] fix
user_detailsrequests not being cached (#7414) - [pixiv:novel] handle exceptions during
embedsextraction (#7422) - [subscribestar] fix username & password login
- [wikifeet] support site redesign (#7286 #7396)
Improvements
- [bluesky:likes] use
repo.listRecordsendpoint (#7194 #7287) - [gelbooru] don't hardcode image server domains (#7392)
- [instagram] support
/share/URLs (#7241) - [kemonoparty] use
/posts-legacyendpoint (#6780 #6931 #7404) - [naver] support videos (#4682 #7395)
- [scrolller] support album posts (#7339)
- [subscribestar] add warning for missing login cookie
- [twitter] update API endpoint query hashes (#7382 #7386)
- [weasyl] use
gallery-dlUser-Agent header (#7412)
Metadata
- [deviantart:stash] extract more metadata (#7397)
- [moebooru:pool] replace underscores in pool names (#4646)
- [naver] fix recent
datebug (#4682)
Post Processors
- [ugoira] restore
keep-filesfunctionality (#7304) - [ugoira] support
"keep-files": true+ custom extension (#7304) - [ugoira] use
_ugoira_frame_indexto detect.zipfiles
Miscellaneous
- [util] auto-update Chrome version
- use internal version of
re.compile()for extractor patterns
1.29.4 - 2025-04-13
Extractors
Additions
Fixes
- [deviantart] fix
KeyError: 'has_subfolders'(#7272 #7337) - [discord] fix
parentkeyword inconsistency (#7341 #7353) - [E621:pool] fix
AttributeError(#7265 #7344) - [everia] fix/improve image extraction (#7270)
- [gelbooru] fix video URLs (#7345)
- [hentai2read] fix
AttributeErrorexception for chapters without artist (#7355) - [issuu] fix extractors (#7317)
- [kemonoparty] fix file paths with backslashes (#7321)
- [readcomiconline] fix
issueextractor (#7269 #7330) - [rule34xyz] update to API v2 (#7289)
- [zerochan] fix
KeyError: 'author'(#7282)
Improvements
- [instagram] use Chrome
User-Agentby default (#6379) - [pixiv] support
phixiv.netURLs (#7352) - [tumblr] support URLs without subdomain (#7358)
- [webtoons] download JPEG files in higher quality
- [webtoons] use a default 0.5-1.5s delay between requests (#7329)
- [zzup] support
w.zzup.comURLs (#7327)
Downloaders
- [ytdl] fix
KeyError: 'extractor'exception whenytdlreports an error (#7301)
Post Processors
- [metadata] add
metadata-pathoption (#6582) - [metadata] fix handling of empty directory paths (#7296)
- [ugoira] preserve
extensionwhen using"mode": "archive"(#7304)
Miscellaneous
- [formatter] add
iandfconversions (#6582)
1.29.3 - 2025-03-29
Extractors
Additions
Fixes
- [bunkr] fix
filenameextraction (#7237) - [deviantart:stash] fix legacy
sta.shlinks (#7181) - [hitomi] fix extractors (#7230)
- [mangapark] fix extractors (#4999 #5883 #6507 #6908 #7232)
- [nozomi] fix extractors (#7242)
- [patreon] include subdomains in
session_idcookie check (#7188) - [patreon] do not match
/messagesURLs as creator (#7187) - [pinterest] handle
story_pin_static_sticker_blockblocks (#7251) - [sexcom] fix
gifpin extraction (#7239) - [skeb] make exceptions when extracting posts non-fatal (#7250)
- [zerochan] parse
JSON-LDdata (#7178)
Improvements
- [arcalive] extend
gifsoption - [deviantart] support multiple images for single posts (#6653 #7261)
- [deviantart] add subfolder support (#4988 #7185 #7220)
- [deviantart] match
/gallery/recommended-for-youURLs (#7168 #7243) - [instagram] extract videos from
video_dash_manifestdata (#6379 #7006) - [mangapark] support mirror domains
- [mangapark] support v3 URLs (#2072)
- [mastodon] support
/statusesURLs (#7255) - [sexcom] support new-style
/gifsand/videosURLs (#7239) - [subscribestar] detect redirects to
/age_confirmation_warningpages - [tiktok] add retry mechanism to rehydration data extraction (#7191)
Metadata
- [bbc] extract more metadata (#6582)
- [kemonoparty] extract
archivesmetadata (#7195) - [kemonoparty] enable
username/user_profilemetadata by default - [kemonoparty:discord] always provide
channel_namemetadata (#7245) - [sexcom] extract
date_urlmetadata (#7239) - [subscribestar] extract
titlemetadata (#7219)
Downloaders
Miscellaneous
- [aes] simplify
block_countcalculation - [common] add
subdomainsargument tocookies_check()(#7188) - [config] fix using the same key multiple times with
apply(#7127) - [tests] implement expected failures
1.29.2 - 2025-03-15
Extractors
Additions
Fixes
- [civitai] fix/improve query parameter handling (#7138)
- [facebook] improve
dateextraction (#7151) - [sankaku] update API URLs (#7154 #7155 #7163)
- [twitter] prevent exception in
_extract_components()(#7139)
Improvements
- [batoto] add
domainoption (#7174) - [furaffinity] extract
scrapsmetadata (#7015) - [tiktok] implement audio extraction without
yt-dlp - [wikimedia] add
subcategoriesoption (#2340)
1.29.1 - 2025-03-08
Extractors
Additions
- [tenor] add support (#6075)
Fixes
- [bunkr] update API endpoint (#7097)
- [erome] fix
AttributeErrorfor albums without tags (#7076) - [furaffinity] fix
artistmetadata (#6582 #7115 #7123 #7130) - [jpgfish] decrypt file URLs (#7073 #7079 #7109)
- [sankaku] fix search tag limit check
- [vsco] fix
videoextractor (#7113) - [vsco] fix extracting videos from
/galleryresults (#7113)
Improvements
- [bunkr] add
endpointoption (#7097) - [danbooru:pool] download posts in pool order, add
order-postsoption (#7091) - [erome:search] recognize all URL query parameters (#7125)
- [reddit] add
selftextoption (#7111) - [redgifs:search] support
/search?query=...URLs (#7118) - [sankaku] increase wait time on 429 errors (#7129)
- [tiktok] improve
tiktok-rangeparsing (#7098)
Downloaders
- [http] detect Cloudflare/DDoS-Guard challenges (#7066 #7121)
- warn about invalid
subcategoryvalues (#7103 #7119)
1.29.0 - 2025-03-01
Changes
- build
gallery-dl.exeon Windows 10 / Python 3.13 (#6684) - provide Windows 7 / Python 3.8 builds as
gallery-dl_x86.exe
Extractors
Additions
- [bilibili] add
user-articles-favoriteextractor (#6725 #6781) - [boosty] add
direct-messagesextractor (#6768) - [discord] add support (#454 #6836 #7059 #7067)
- [furry34] add support (#1078 #7018)
- [hentaiera] add support (#3046 #6952 #7020)
- [hentairox] add support (#7003)
- [imgur] add support for personal posts (#6990)
- [imhentai] add support (#1660 #3046 #3824 #4338 #5936)
- [tiktok] add support (#3061 #4177 #5646 #6878 #6708)
- [vsco] support
/video/URLs (#4295 #6973)
Fixes
- [bunkr] decrypt file URLs (#7058 #7070 #7085 #7089 #7090)
- [chevereto/jpgfish] fix extraction (#7073 #7079)
- [generic] fix config lookups by subcategory
- [philomena] fix
datevalues without UTC offset (#6921) - [philomena] download
fullURLs to prevent potential 404 errors (#6922) - [pixiv] prevent exceptions during
commentsextraction (#6965) - [reddit] restrict subreddit search results (#7025)
- [sankaku] fix extraction (#7071 #7072)
- [subscribestar] fix
postextractor (#6582) - [twitter] revert generated CSRF token length to 32 characters (#6895)
- [vipergirls] change default
domaintoviper.click(#4166) - [weebcentral] fix extracting wrong number of chapter pages (#6966)
Improvements
- [b4k] update domain to
arch.b4k.dev(#6955 #6956) - [bunkr] update default archive ID format (#6935)
- [bunkr] provide fallback URLs for 403 download links (#6732 #6972)
- [bunkr] implement fast
--rangesupport (#6985) - [furaffinity] use a default delay of 1 second between requests (#7054)
- [itaku] support gallery section URLs (#6951)
- [patreon] support
/profile/creatorsURLs - [subscribestar] detect and handle redirects (#6916)
- [twibooru] match URLs with
wwwsubdomain (#6903) - [twitter] support
grokcards content (#7040) - [vsco] improve
m3u8handling - [weibo] add
moviesoption (#6988)
Metadata
- [bunkr] extract
id_urlmetadata (#6935) - [erome] extract
tagsmetadata (#7076) - [issuu] unescape HTML entities
- [newgrounds] provide
comment_htmlmetadata (#7038) - [patreon] extract
campaignmetadata (#6989)
Downloaders
- implement
downloaderoptions per extractor category - [http] add
sleep-429option (#6996) - [ytdl] support specifying
moduleas filesystem paths (#6991)
Archives
- [archive] implement support for PostgreSQL databases (#6152)
- [archive] add
archive-tableoption (#6152)
Miscellaneous
- [aes] handle errors during
cryptodomeimport (#6906) - [executables] fix loading
certifiSSL certificates (#6393) - improve
\fformat string handling for--print
1.28.5 - 2025-01-28
Extractors
Additions
- [nekohouse] add support (#5241, #6738)
- [turboimagehost] add support for galleries (#6855)
- [xfolio] add support (#5514, #6351, #6837)
Fixes
- [4archive] fix
TypeError - [adultempire] bypass age confirmation check
- [architizer] fix extraction
- [artstation] avoid Cloudflare challenges (#5817, #5658, #5564, #5554)
- [deviantart] prevent crash when accessing
premium_folderdata (#6873) - [fapachi] fix extraction (#6881)
- [issuu] fix
userextractor - [kemonoparty] fix
usernamemetadata and filtering bytagfor/postsURLs (#6833) - [mangafox] fix chapter extraction
- [mangahere] fix chapter extraction
- [pixiv] fix
sanity_levelworkaround (#4327) - [pornpics] fix pagination results from HTML pages
- [twitter] handle exceptions during file extraction (#6647)
- [vsco] fix
JSONDecodeError(#6887, #6891) - [weebcentral] fix extraction (#6860)
- [xhamster] fix
galleryextractor (#6818, #6876)
Improvements
- [batoto] use
chapter_idin default archive IDs (#6835) - [e621] support
e621.ccande621.anthro.frfrontend URLs (#6809) - [e621] prevent premature pagination end (#6886)
- [facebook] allow accessing all metadata in
directoryformat strings (#6874) - [hiperdex] update domain to
hiperdex.com - [kemonoparty] enable filtering creator posts by tag (#6833)
- [khinsider] add
coversoption (#6844) - [komikcast] update domain to
komikcast.la - [lofter] improve error handling (#6865)
- [pornpics] avoid redirect when retrieving a gallery page
- [urlgalleries] support new URL format
Metadata
- [bunkr] extract better
filenamemetadata (#6824) - [hiperdex] fix
descriptionmetadata - [khinsider] extract more
albummetadata (#6844) - [mangaread] fix manga metadata extraction
- [rule34xyz] fix
dateandtagsmetadata - [saint] fix metadata of
/d/URLs - [toyhouse] fix
date,artists, andcharactersmetadata - [webtoons] fix
usernameandauthor_namemetadata
Removals
- [cohost] remove module
- [fanleaks] remove module
- [shimmie2] remove
tentaclerape.net - [szurubooru] remove
booru.foalcon.com
Miscellaneous
- [docs] add
nixdocs to README (#6606) - [path] fix exception when using
--rename-to+--no-download(#6861) - [release] include
scripts/run_tests.pyin release tarball (#6856)
1.28.4 - 2025-01-12
Extractors
Additions
Fixes
- [bunkr] update to new site layout (#6798, #6805)
- [bunkr] fix
ValueErroron relative redirects (#6790) - [plurk] fix
userdata extraction and make it non-fatal (#6742)
Improvements
- [bunkr] support
/f/media URLs - [e621] accept
tagsearch URLs with empty tag (#6783) - [pixiv] provide fallback URLs (#6762)
- [wallhaven] extract
search[tags]andsearch[tag_id]metadata (#6772)
Miscellaneous
- [util] support not splitting
valueargument when callingcontains()(#6773)
1.28.3 - 2025-01-04
Extractors
Additions
Fixes
- [8muses] skip albums without valid
permalink(#6717) - [batoto] update domains (#6714)
- [deviantart:tiptap] fix deviation embeds without
token - [hitomi] fix searches (#6713)
- [instagram:reels] fix
pinnedvalues (#6719) - [kemonoparty] handle
discordfavorites (#6706) - [piczel] fix extraction (#6735)
- [poipiku] fix downloads when post has a warning (#6736)
- [sankaku] support alphanumeric book/pool IDs (#6757)
- [subscribestar] fix attachment downloads (#6721, #6724, #6758)
- [subscribestar] improve
contentmetadata extraction (#6761) - [tapas] fix
TypeErrorfor locked episodes (#6700)
Improvements
- [boosty] support
filepost attachments (#6760) - [deviantart:tiptap] support more content block types (#6686)
- [directlink] use domain as
subcategory(#6703) - [hitomi] provide
search_tagsmetadata fortagandsearchresults (#6756) - [subscribestar] support
audiofiles (#6758)
Miscellaneous
- [workflows:executables] build with Python 3.13
1.28.2 - 2024-12-20
Extractors
Additions
- [cyberdrop] add extractor for media URLs (#2496)
- [itaku] add
searchextractor (#6613) - [lofter] add initial support (#650, #2294, #4095, #4728, #5656, #6607)
- [yiffverse] add support (#6611)
Fixes
- [facebook] decode Unicode surrogate pairs in metadata values (#6599)
- [zerochan] parse API responses manually when receiving invalid JSON (#6632)
- [zerochan] fix
sourcemetadata extraction when not logged in
Improvements
- [bilibili] extract files from
module_topentries (#6687) - [bilibili] support
/upload/opusURLs (#6687) - [bluesky] default to
poststimeline whenrepostsorquotedis enabled (#6583) - [common] simplify HTTP error messages
- [common] detect
DDoS-Guardchallenge pages - [deviantart] improve
tiptapmarkup to HTML conversion (#6686)- fix
KeyError: 'attrs'for links withouthref - support
headingcontent blocks - support
striketext markers
- fix
- [instagram] extract
datemetadata for stories (#6677) - [kemonoparty:favorite] support new URL format (#6676)
- [saint] support
saint2.crURLs (#6692) - [tapas] improve extractor hierarchy (#6680)
Options
- [cohost] add
avatarandbackgroundoptions (#6656)
Miscellaneous
- support
*wildcards forparent>childcategories, for examplereddit>*(#6673) - use latest Firefox UA as default
user-agent - use random unused port for
"user-agent": "browser"requests
1.28.1 - 2024-12-07
Extractors
Additions
- [bluesky] add
infoextractor
Fixes
- [bluesky] fix exception when encountering non-quote embeds (#6577)
- [bluesky] unescape search queries (#6579)
- [common] restore using environment proxy settings by default (#6553, #6609)
- [common] improve handling of
user-agentsettings (#6594) - [e621] fix
TypeErrorwhenmetadatais enabled (#6587) - [gofile] fix website token extraction (#6596)
- [inkbunny] fix re-login loop (#6618)
- [instagram] handle empty
carousel_mediaentries (#6595) - [kemonoparty] fix
oquery parameter handling (#6597) - [nhentai] fix download URLs (#6620)
- [readcomiconline] fix
chapterextraction (#6070, #6335) - [realbooru] fix extraction (#6543)
- [rule34] fix
favoriteextraction (#6573) - [zerochan] download
.webpand.giffiles (#6576)
Improvements
- [hentaicosplays] update domains (#6578)
- [pixiv:ranking] implement filtering results by
content(#6574) - [pixiv] include user ID in failed AJAX request warnings (#6581)
Options
1.28.0 - 2024-11-30
Changes
- [common] disable using environment network settings by default (
HTTP_PROXY,.netrc, …) - [ytdl] change
forward-cookiesdefault value totrue(#6401, #6348)
Extractors
Additions
- [bilibili] add support for
opusarticles (#2824, #6443) - [bluesky] add
hashtagextractor (#4438) - [danbooru] add
artistandartist-searchextractors (#5348) - [everia] add support (#1067, #2472, #4091, #6227)
- [facebook] add support (#470, #2612, #5626, #6548)
- [hentaifoundry] add
tagextractor (#6465) - [hitomi] add
indexandsearchextractors (#2502, #6392, #3720) - [motherless] add support (#2074, #4413, #6221)
- [noop] add
noopextractor - [rule34vault] add support (#5708, #6240)
- [rule34xyz] add support (#1078, #4960)
- [saint] add support (#4405, #6324)
- [tumblr] add
searchextractor (#6394)
Fixes
- [8chan] avoid performing network requests within
_init()(#6387) - [bluesky] fix downloads from non-bsky PDSs (#6406)
- [bunkr] fix album names containing
<>&characters - [flickr] use
downloadURLs (#6360, #6464) - [hiperdex] update domain to
hipertoon.com(#6420) - [imagechest] fix extractors (#6475, #6491)
- [instagram] fix using numeric cursor values (#6414)
- [kemonoparty] update to new site layout (#6415, #6503, #6528, #6530, #6536, #6542, #6554)
- [koharu] update domain to
niyaniya.moe(#6430, #6432) - [mangadex] apply
langoption only to chapter results (#6372) - [newgrounds] fix metadata extraction (#6463, #6533)
- [nhentai] support
.webpfiles (#6442, #6479) - [patreon] use legacy mobile UA when no
session_idis set - [pinterest] update API headers (#6513)
- [pinterest] detect video/audio by block content (#6421)
- [scrolller] prevent exception for posts without
mediaSources(#5051) - [tumblrgallery] fix file downloads (#6391)
- [twitter] make
sourcemetadata extraction non-fatal (#6472) - [weibo] fix livephoto
filename&extension(#6471)
Improvements
- [bluesky] support
main.bsky.devURLs (#4438) - [bluesky] match common embed fixes (#6410, #6411)
- [boosty] update default video format list (#2387)
- [bunkr] support
bunkr.crURLs - [common] allow passing cookies to OAuth extractors
- [common] allow overriding more default
User-Agentheaders (#6496) - [philomena] switch default
ponyboorufilter (#6437) - [pinterest] support
story_pin_musicblocks (#6421) - [pixiv] get ugoira frame extension from
meta_single_pagevalues (#6056) - [reddit] support user profile share links (#6389)
- [steamgriddb] disable
adjust-extensionsforfake-pngfiles (#5274) - [twitter] remove cookies migration workaround
Metadata
- [bluesky] provide
author[instance]metadata (#4438) - [instagram] fix
extensionof apparent.webpfiles (#6541) - [pillowfort] provide
countmetadata (#6478) - [pixiv:ranking] add
rankmetadata field (#6531) - [poipiku] return
countas proper number (#6445) - [webtoons] extract
episode_nofor comic results (#6439)
Options
- [civitai] add
metadataoption - support fetchinggenerationdata (#6383) - [exhentai] implement
tagsoption (#2117) - [koharu] implement
tagsoption - [rule34xyz] add
formatoption (#1078)
Downloaders
- [ytdl] fix
AttributeErrorcaused bydecodeOption()removal (#6552)
Post Processors
- [classify] rewrite - fix skipping existing files (#5213)
- enable inheriting options from global
postprocessorobjects - allow
postprocessorsvalues to be a single post processor object
Cookies
- support Chromium table version 24 (#6162)
- fix GCM pad length calculation (#6162)
- try decryption with empty password as fallback
Documentation
Options
- add
-4/--force-ipv4and-6/--force-ipv6command-line options - fix passing negative numbers as arguments (#5262)
Miscellaneous
- [output] use default ANSI colors only when stream is a TTY
- [util] implement
defaultdictfilters-environment - [util] enable using multiple statements for all
filteroptions (#5262)
1.27.7 - 2024-10-25
Extractors
Additions
- [civitai] add extractors for global
modelsandimages(#6310) - [mangadex] add
authorextractor (#6372) - [scrolller] add support (#295, #3418, #5051)
Fixes
- [8chan] automatically detect
TOScookie name (#6318) - [bunkr] update to new site layout (#6344, #6352, #6368)
- [bunkr] send proper
Refererheaders for file downloads (#6319) - [civitai] add
uuidmetadata field & use it as default archive format (#6326) - [civitai] fix "My Reactions" results (#6263)
- [civitai] fix
modelfile download URLs for tRPC API - [lensdump] fix extraction (#6313)
- [pixiv] make retrieving ugoira metadata non-fatal (#6297)
- [pixiv] fix exception when processing deleted
sanity_levelworks (#6339) - [urlgalleries] fix extraction
- [wikimedia] fix non-English Fandom/wiki.gg articles (#6370)
Improvements
- [8chan] support
/last/thread URLs (#6318) - [bunkr] support
bunkr.phandbunkr.psURLs - [newgrounds] support page numbers in URLs (#6320)
- [patreon] support
/c/prefix in creator URLs (#6348) - [pinterest] support
storypins (#6188, #6078, #4229) - [pixiv] implement
sanity_levelworkaround for user artworks results (#4327, #5435, #6339)
Options
- [bluesky] add
quotedoption (#6323) - [pixiv] add
captionsoption (#4327) - [reddit] add
embedsoption (#6357) - [vk] add
offsetoption (#6328)
Downloaders
- [ytdl] implement explicit HLS/DASH handling
Post Processors
- add
errorevent
Miscellaneous
- [cookies] convert Chromium
expires_utcvalues to Unix timestamps - [util] add
stdobject to global eval namespace (#6330) - add
--printand--print-to-filecommand-line options (#6343) - use child extractor fallbacks only when a non-user error occurs (#6329)
1.27.6 - 2024-10-11
Extractors
Additions
- [ao3] add
subscriptionsextractor (#6247) - [boosty] add support (#2387)
- [civitai] add
postextractors (#6279) - [pixiv] support unlisted artworks (#5162)
Fixes
- [cohost] sanitize default filenames (#6262)
- limit
headlinelength - remove
plainTextBody
- limit
- [deviantart] fix & improve journal/literature extraction (#6254, #6207, #6196)
- extract journal HTML from webpage if possible
- support converting
tiptapmarkup to HTML
- [deviantart] fix
stashfolder extraction - [flickr] update default API credentials (#6300)
- [flickr] fix
ZeroDivisionError(#6252) - [imagefap] fix
{num}in single image default filenames - [myhentaigallery] fix
tagsextraction - [patreon] extract
attachments_mediafiles (#6241, #6268) - [pixiv] implement workaround for
limit_sanity_levelworks (#4327, #4747, #5054, #5435, #5651, #5655) - [pornhub] fix
KeyErrorwhen album images are missing (#6299) - [rule34us] fix extraction (#6289)
- [8chan] set TOS cookie for current and previous day
Improvements
- [bunkr] support
bunkr.pkURLs (#6272) - [civitai] use tRPC API by default (#6279)
- [civitai] improve default archive format (#6302)
- [komikcast] update domain to
komikcast.cz - [newgrounds] detect more comment embeds (#6253)
- [newgrounds] add more fallback URL formats for
art-imagesfiles - [oauth] prevent empty browser names
- [patreon] use mobile UA (#6241, #6239, #6140)
- [patreon] handle suspended accounts
- [pixiv] detect works requiring
My pixivaccess
Metadata
- [civitai] ensure image files have an
id(#6251) - [gelbooru_v02] unescape HTML entities in categorized tags
- [generic] ensure
pathmetadata is always defined - [pixiv] retrieve
captionfrom AJAX API when empty (#4327, #5191)
Options
Removals
- [blogger] remove
micmicidol.club - [chevereto] remove
deltaporno.com - [lolisafe] remove
xbunkr.com - [pururin] remove module
- [shimmie2] remove
loudbooru.com
Post Processors
- [ugoira] fix
BadZipFileexceptions (#6285) - [ugoira] catch all exceptions when extracting ZIP archives (#6285)
- [ugoira] forward frame data as
_ugoira_frame_data(#6154, #6285)
Miscellaneous
- [build] remove setuptools and requests version restrictions
- [docker] build from
python:3.12-alpine - [text] improve
parse_query()performance
1.27.5 - 2024-09-28
Extractors
Additions
- [ao3] add support (#6013)
- [civitai] add support (#3706, #3787, #4129, #5995, #6220)
- [cohost] add support (#4483, #6191)
Fixes
- [8chan] update
TOScookie name - [deviantart] work around OAuth API returning empty journal texts (#6196, #6207, #5916)
- [weasyl:favorite] fix pagination (#6113)
Improvements
- [bluesky] support video downloads (#6183)
- [deviantart] add
previewsoption (#3782, #6124) - [deviantart] warn about empty journal texts (#5916)
- [inkbunny:favorite] update default directory (#6115)
- [jpgfish] update domain to
jpg5.su(#6231) - [skeb] prevent 429 errors and need for
request_keycookie - [weasyl:favorite] support readable URL format (#6113)
- [wikimedia] automatically detect API endpoint when none is defined
- [zzup] support
up.zzup.comgalleries (#6181)
Post Processors
- [ugoira] implement storing "original" frames in ZIP archives (#6147)
- [ugoira] fix
KeyError: '_ugoira_frame_index'(#6154)
Formatter
- add
Lconversion - returns the length of a value - allow accessing
util.NONEvia global_nul
Miscellaneous
- [cookies] add
cookies-selectoption - [cookies:firefox] support using domain & container filters together
- [docker] prevent errors in Dockerfile build
- [tests] make
#categoryresult entries optional - allow filtering
--list-extractorsresults - implement alternatives for deprecated
utcdatetime functions
1.27.4 - 2024-09-06
Extractors
Additions
Fixes
- [bunkr] fix file downloads (#6037)
- [cyberdrop] fix extraction
- [deviantart] fix
"pagination": "manual"for cursor-based endpoints (#6079) - [deviantart] fix
"original": "images"(#6124) - [exhentai] fix
limitsoption (#6090) - [flickr] make
exifandcontextmetadata extraction non-fatal (#6002, #6077) - [flickr] make
albummetadata extraction non-fatal (#3441) - [furaffinity] fix
favoritepagination (#6151) - [gofile] fix
KeyError: 'childrenIds'(#5993) - [newgrounds] fix warning for age-restricted posts (#6005)
- [toyhouse] fix extraction of image URLs
- [tumblr] fix
401 Unauthorizedfor likes when using api-key (#5994) - [twitter] fix pinned Tweet extraction (#6102)
- [ytdl] fix processing playlists of playlists (#6127)
Improvements
- [bcbnsfw] use
*as query when retrieving all posts (#6135) - [bunkr] support
bunkr:URL prefix (#6017) - [e621] cache pool metadata API calls (#6001)
- [generic] better directory names (#6104)
- [koharu] improve format selection (#6088)
- [pixiv] implement downloading "original" ugoira frames (#6056)
- [pixiv] use mobile API for
series(#5983)
Metadata
- [batoto] improve chapter info regex (#5988, #5997)
- [batoto] extract
chapter_urlmetadata (#5562) - [batoto] improve
titleextraction (#5988) - [hitomi] extract
extension_originalmetadata (#6049) - [instagram] add
post_datemetadata field (#6081, #6091) - [sankaku] restore old
tagsformat (#6043) - [twitter] extract
typemetadata (#6111)
Options
- [bunkr] add
tldsoption to match URLs with all possible TLDs (#5875, #6017) - [instagram] add
max-postsoption (#6054) - [instagram] add
infoas a possibleincludevalue - [instagram] allow disabling
cursoroutput - [twitter] add
infoas a possibleincludevalue (#6114) - [twitter] allow disabling
cursoroutput (#5990)
Post Processors
- [hash] add
hashpost processor to compute file hash metadata (#6099) - [metadata] add
includeandexcludeoptions (#6058) - [metadata] fix using
..in directories on Windows (#5942, #6094) - [rename] add
renamepost processor to rename previously downloaded files (#5846, #6044) - [ugoira] support converting "original" frames (#6056)
- [ugoira] add
skipoption (#6056)
Miscellaneous
- [cookies:firefox] extract only cookies without container by default (#5957)
- [formatter] implement
Aformat specifier (#6036) - [tests] fix bug when running tests in a certain order
- [util] extend
CustomNonewith arithmetic operators (#6007, #6009) - add
--renameand--rename-tocommand-line options (#5846, #6044) - add
input-filesconfig option (#6059)
1.27.3 - 2024-08-10
Extractors
Additions
- [bunkr] support
bunkr.ciandbunkrrr.org(#5970) - [furaffinity] add
submissionsextractor (#5954) - [hentaicosplays] support
hentai-cosplay-xxx.com(#5959)
Fixes
- [behance] fix
KeyError: 'fields'(#5965) - [behance] fix video extraction (#5965)
- [cien] extract all files when authenticated (#5934)
- [deviantart] fix
KeyError - 'category'(#5960, #5961, #5969, #5971, #5976, #5978) - [fanbox] update pagination logic (#5949, #5951, #5956)
- [hotleak] fix AttributeError (#5950)
- [instagram] restore GraphQL API functionality (#5920)
- [twitter] update
x-csrf-tokenheader during login (#5945)
Improvements
- [bunkr] fail downloads for
maintenancefiles (#5952) - [zerochan] improve tag redirect handling, add
redirectsoption (#5891)
Post Processors
1.27.2 - 2024-08-03
Extractors
Additions
- [agnph] add
tagandpostextractors (#5284, #5890) - [aryion] add
favoriteextractor (#4511, #5870) - [cien] add support (#2885, #4103, #5240)
- [instagram] add
infoextractor (#5262) - [koharu] add
gallery,search, andfavoriteextractors (#5893, #4707) - [twitter] add
infoextractor (#3623)
Fixes
- [8chan] update
TOScookie name (#5868) - [behance] fix image extraction (#5873, #5926)
- [booru] prevent crash when file URL is empty (#5859)
- [deviantart] try to work around journal/status API changes (#5916)
- [hentainexus] fix error with spread pages (#5827)
- [hotleak] fix faulty image URLs (#5915)
- [inkbunny:following] fix potentially infinite loop
- [nijie] fix image URLs of single image posts (#5842)
- [readcomiconline] fix extraction (#5866)
- [toyhouse] fix Content Warning bypass (#5820)
- [tumblr] revert to
offsetpagination, implementpaginationoption (#5880) - [twitter] fix
username-altoption name (#5715) - [warosu] fix extraction
- [zerochan] handle
KeyError - 'items'(#5826) - [zerochan] fix error on tag redirections (#5891)
- [zerochan] fix
Invalid control charactererrors (#5892)
Improvements
- [bunkr] support
bunkr.fidomain (#5872) - [deviantart:following] use OAuth API endpoint (#2511)
- [directlink] extend recognized file extensions (#5924)
- [exhentai] improve error message when temporarily banned (#5845)
- [gelbooru_v02] use total number of posts as pagination end marker (#5830)
- [imagefap] add enumeration index to default filenames (#1746, #5887)
- [paheal] implement fast
--rangesupport (#5905) - [redgifs] support URLs with numeric IDs (#5898, #5899)
- [sankaku] match URLs with
wwwsubdomain (#5907) - [sankakucomplex] update domain to
news.sankakucomplex.com - [twitter] implement
cursorsupport (#5753) - [vipergirls] improve
threadURL pattern - [wallpapercave] support
albumlistings (#5925)
Metadata
- [dynastyscans] extract chapter
tags(#5904) - [erome] extract
datemetadata (#5796) - [furaffinity] extract
foldersandthumbnailmetadata (#1284, #5824) - [sankaku] implement
notesextraction (#5865) - [subscribestar] fix
dateparsing in updated posts (#5783) - [twitter] extract
bookmark_countandview_countmetadata (#5802) - [zerochan] fix
sourcemetadata - [zerochan] fix tag category extraction (#5874)
- [zerochan] delay fetching extended metadata (#5869)
Options
- [agnph] implement
tagsoption (#5284) - [booru] allow multiple
urlkeys (#5859) - [cien] add
filesoption (#2885) - [koharu] add
cbzandformatoptions (#5893) - [vsco] add
includeoption (#5911) - [zerochan] implement
tagsoption (#5874)
Removals
- [fallenangels] remove module
Post Processors
- [metadata] allow using format strings for
directory(#5728)
Options
- add
--print-trafficcommand-line option - add
-J/--resolve-jsoncommand-line option (#5864) - add
filters-environmentoption - implement
archive-eventoption (#5784)
Actions
- [actions] support multiple actions per pattern
- [actions] add
execaction (#5619) - [actions] add
abortandterminateactions (#5778) - [actions] allow setting a duration for
wait - [actions] emit logging messages before waiting/exiting/etc
Tests
- [tests] enable test results for external extractors (#5262)
- [tests] load results from
${GDL_TEST_RESULTS}(#5262)
Miscellaneous
- [cookies] add
thoriumsupport (#5781) - [job] add
resolveargument to DataJob (#5864) - [path] fix moving temporary files across drives on Windows (#5807)
- [ytdl] fix
--cookies-from-browseroption parsing (#5885) - make exceptions in filters/conditionals non-fatal
- update default User-Agent header to Firefox 128 ESR
- include
zstdin Accept-Encoding header when supported
1.27.1 - 2024-06-22
Extractors
Additions
- [hentainexus] restore module (#5275, #5712)
- [shimmie2] support
vidya.pics(#5632) - [tcbscans] support other domains (#5774)
Fixes
- [deviantart] fix watching module ID extraction (#5696, #5772)
- [fanbox] handle KeyError for no longer existing plans (#5759)
- [kemonoparty:favorite] fix exception when sorting
nullobjects (#5692. #5721) - [skeb] fix
429 Too Many Requestserrors (#5766) - [speakerdeck] fix extraction (#5730)
- [twitter] fix duplicate
ArkoseLogincheck
Improvements
- [nijie] support downloading videos (#5707, #5617)
- [philomena] support downloading
.svgfiles (#5643) - [szurubooru] support empty tag searches (#5711)
- [twitter] ignore
Unavailablemedia (#5736)
Metadata
Options
- [newgrounds] extend
formatoption (#5709) - [twitter] extend
ratelimitoption (#5532) - [twitter] add
username-altoption (#5715)
Removals
- [photobucket] remove module
- [nitter] remove instances
- [vichan] remove
wikieat.club
Downloaders
- [ytdl] fix exception due to missing
extin unavailable videos (#5675)
Formatter
Options
- add
--no-inputcommand-line andinputconfig option (#5733) - add
--config-opencommand-line option (#5713) - add
--config-statuscommand-line option (#5713)
Miscellaneous
- [actions] fix exception when
msgis not a string (#5683)
1.27.0 - 2024-06-01
Extractors
Additions
- [mastodon] add
favorite,list, andhashtagextractors (#5529) - [mastodon] add support for card images
- [pixeldrain] add support for single-file album downloads (#5641)
- [reddit] support comment embeds (#5366)
- [seiga] re-implement login with username & password
- [tapas] add
creatorextractor (#5306) - [vsco] add
avatarextractor (#5341) - [wikimedia] support
wiki.ggwikis
Fixes
- [4archive] fix extraction
- [8chan] fix file downloads by sending a
TOScookie (#5578) - [artstation] disable TLS 1.2 ciphers by default (#5564, #5658)
- [bluesky] filter reposts only for user timelines (#5528)
- [common] disable
check_hostnamefor custom SSLContexts (#3614, #4891, #5576) - [exhentai] fix Multi-Page Viewer detection (#4969)
- [exhentai] fix blank page detection
- [hiperdex] update domain to
hiperdex.top(#5635) - [hotleak] download files returning a 404 status code (#5395)
- [imgur] match URLs with title slugs (#5593)
- [kemonoparty] fix
KeyError - 'path'for posts without files (#5368, #5394, #5422, #5488) - [kemonoparty] fix crash on posts with missing datetime info (#5422)
- [mastodon] send canonical
true/falseboolean values (#5516) - [newgrounds] update and fix login procedure (#5109)
- [patreon] fix
bootstrapdata extraction (#5624) - [poipiku] fix downloading R-18 posts (#5567)
- [poipoku] avoid language-specific extraction (#5590, #5591)
- [realbooru] fix videos and provide fallback URLs (#2530)
- [slideshare] fix extraction
- [subscribestar] fix file URLs (#5631)
- [twitter] update domain to
x.com(#5597) - [twitter] transfer
twitter.comcookies tox.com(#5597) - [twitter] prevent crash when extracting
birdwatchmetadata (#5403) - [twitter] handle missing
expanded_urlfields (#5463, #5490) - [wikimedia] suppress exception for entries without
imageinfo(#5384) - [wikimedia] fix exception for files with empty
metadata
Improvements
- [exhentai] detect CAPTCHAs during login (#5492)
- [foolfuuka] improve
boardpattern & support pages (#5408) - [furaffinity] match
fxfuraffinity.net/fxraffinity.net/xfuraffinity.netURLs (#5511, #5568) - [gelbooru] improve pagination logic for meta tags (#5478)
- [kemonoparty:favorite] return artists/posts in native order and support
sortandorderquery parameters (#5375, #5620) - [oauth] use
Extractor.request()for HTTP requests to support proxy servers etc (#5433) - [pixiv] change
sanity_leveldebug message to a warning (#5180) - [twitter] improve username & password login procedure (#5445)
- [twitter] wait for rate limit reset before encountering a 429 error (#5532)
- [twitter] match
fixvx.comURLs (#5511) - [twitter] match Tweet URLs with query parameters (#5371, #5372)
- [twitter] match
/photo/and/video/Tweet URLs (#5443, #5601)
Options
- [common] add
sleep-429option (#5160) - [common] implement
skip-filteroption (#5255) - [common] implement
keywords-evaloption (#5621) - [kemonoparty] add
announcementsoption (#5262) - [pixiv:novel] add
coversoption (#5373) - [twitter] implement
reloginoption (#5445)
Downloaders
- [http] add MIME type and signature for
.m4vfiles (#5505)
Post Processors
- [mtime] do not overwrite
_mtimevalues withNone(#5439) - [ugoira] log errors for general exceptions
Archives
- [archive] move DownloadArchive code into its own module
- [archive] implement
DownloadArchiveMemoryclass (#5255) - [archive] add
archive-modeoption (#5255)
Cookies
- [cookies] use temporary file when saving cookies.txt files (#5461)
- [cookies] optimize
_find_most_recently_used_file()for exact profiles (#5538) - [cookies] set proper
expiresvalue for Chrome session cookies
Documentation
- [docs] update docs/configuration links (#5059, #5369, #5423)
- [docs] update link to "nightly" builds (#5618)
- [docs] replace AnchorJS with custom script
- [docs] update defaults of
sleep-request,browser,tls12 - [docs] complete Authentication info in docs/supportedsites
Formatter
- [formatter] allow dots in
'...'literals (#5539)
Output
- [output] enable colored output by default
- [output] extend
output.colors(#2566) - [output] support
NO_COLORenvironment variable - [output] add
--no-colorscommand-line option - [output] add
-w/--warningcommand-line option (#5474)
Tests
- [tests] select unused port number for local HTTP server
- [tests] allow filtering extractor result tests by URL or comment
- [tests] mark tests with missing auth as
only_matching
Update
- implement update-related command-line options (#5233)
-U/--updateupdates an executable file to the latest release--update-checkchecks if the local version is up to date--update-toallows switching to a different release channel (stableordev) as well as upgrading/downgrading to a specific tag.--update-to dev--update-to dev@2024.05.25--update-to v1.25.2
- (non-executable installations have only access to
-U/--update-checkfor version checks)
Miscellaneous
- add workaround for requests 2.32.3 issues (#5665)
- fix exit status of
--clear-cache/--list-extractors/--list-modules - restore
LD_LIBRARY_PATHfor executables built with PyInstaller (#5421) - store
matchandgroupsvalues in Extractor objects
1.26.9 - 2024-03-23
Extractors
Additions
- [artstation] support video clips (#2566, #3309, #3911)
- [artstation] support collections (#146)
- [deviantart] recognize
deviantart.com/stash/…URLs - [idolcomplex] support new pool URLs
- [lensdump] recognize direct image links (#5293)
- [skeb] add extractor for followed users (#5290)
- [twitter] add
quotesextractor (#5262) - [wikimedia] support
azurlane.koumakan.jp(#5256) - [xvideos] support
/channels/URLs (#5244)
Fixes
- [artstation] fix handling usernames with dashes in domain names (#5224)
- [bluesky] fix not spawning child extractors for followed users (#5246)
- [deviantart] handle CloudFront blocks (#5363)
- [deviantart:avatar] fix
indexfor URLs without?(#5276) - [deviantart:stash] fix
indexvalues (#5335) - [gofile] fix extraction
- [hiperdex] update URL patterns & fix
mangametadata (#5340) - [idolcomplex] fix metadata extraction
- [imagefap] fix folder extraction (#5333)
- [instagram] make accessing
like_countnon-fatal (#5218) - [mastodon] fix handling null
movedaccount field (#5321) - [naver] fix EUC-KR encoding issue in old image URLs (#5126)
- [nijie] increase default delay between requests (#5221)
- [nitter] ignore invalid Tweets (#5253)
- [pixiv:novel] fix text extraction (#5285, #5309)
- [skeb] retry 429 responses containing a
request_keycookie (#5210) - [warosu] fix crash for threads with deleted posts (#5289)
- [weibo] fix retweets (#2825, #3874, #5263)
- [weibo] fix
livephotofilename extensions (#5287) - [xvideos] fix galleries with more than 500 images (#5244)
Improvements
- [bluesky] improve API error messages
- [bluesky] handle posts with different
embedstructure - [deviantart:avatar] ignore default avatars (#5276)
- [fapello] download full-sized images (#5349)
- [gelbooru:favorite] automatically detect returned post order (#5220)
- [imgur] fail downloads when redirected to
removed.png(#5308) - [instagram] raise proper error for missing
reels_media(#5257) - [instagram] change
posts are privateexception to a warning (#5322) - [reddit] improve preview fallback formats (#5296, #5315)
- [steamgriddb] raise exception for deleted assets
- [twitter] handle "account is temporarily locked" errors (#5300)
- [weibo] rework pagination logic (#4168)
- [zerochan] fetch more posts by using the API (#3669)
Metadata
- [bluesky] add
instancemetadata field (#4438) - [gelbooru:favorite] add
date_favoritedmetadata field - [imagefap] extract
foldermetadata (#5270) - [instagram] default
likesto0(#5323) - [kemonoparty] add
revision_countmetadata field (#5334) - [naver] unescape post
titleanddescription - [pornhub:gif] extract
viewkeyandtimestampmetadata (#4463) - [redgifs] make
dateavailable for directories (#5262) - [subscribestar] fix
datemetadata - [twitter] add
birdwatchmetadata field (#5317) - [twitter] add
protectedmetadata field (#5327) - [warosu] fix
board_namemetadata
Options
- [bluesky] add
repostsoption (#4438, #5248) - [deviantart] add
comments-avatarsoption (#4995) - [deviantart] extend
metadataoption (#5175) - [flickr] add
contextsoption (#5324) - [gelbooru:favorite] add
order-postsoption (#5220) - [kemonoparty] add
order-revisionsoption (#5334) - [vipergirls] add
likeoption (#4166) - [vipergirls] add
domainoption (#4166)
Downloaders
- [http] add MIME type and signature for
.movfiles (#5287)
Docker
- build images from source instead of PyPI package
- build
linux/arm64images (#5227) - build images on every push to master
- tag images as
YYYY.MM.DD - tag the most recent build from master as
dev - tag the most recent release build as
latest
- tag images as
- reduce image size (#5097)
Miscellaneous
- [formatter] fix local DST datetime offsets for
:O - build Linux executable on Ubuntu 22.04 LTS (#4184)
- automatically create directories for logging files (#5249)
1.26.8 - 2024-02-17
Extractors
Additions
- [bluesky] add support (#4438, #4708, #4722, #5047)
- [bunkr] support new domains (#5114, #5130, #5134)
- [fanbox] add
homeandsupportingextractors (#5138) - [imagechest] add
userextractor (#5143) - [imagetwist] add
galleryextractor (#5190) - [kemonoparty] add
postsextractor (#5194, #5198) - [twitter] support communities (#4913)
- [vsco] support spaces (#5202)
- [weibo] add
gifsoption (#5183) - [wikimedia] support
www.pidgi.net(#5205) - [wikimedia] support
bulbapedia.bulbagarden.net(#5206)
Fixes
- [archivedmoe] fix
thebarchiveWebM URLs (#5116) - [batoto] fix crash when manga name or chapter contains a
-(#5200) - [bunkr] fix extraction (#5088, #5151, #5153)
- [gofile] update
website_tokenextraction - [idolcomplex] fix pagination for tags containing
:(#5184) - [kemonoparty] fix deleting file names when computing
revision_hash(#5103) - [luscious] fix IndexError for files without thumbnail (#5122, #5124, #5182)
- [naverwebtoon] fix
titlefor comics with empty tags (#5120) - [pinterest] fix section URLs for boards with
/,?, or#in their name (#5104) - [twitter] update query hashes
- [zerochan] fix skipping every other post
Improvements
- [deviantart] skip locked/blurred posts (#4567, #5193)
- [deviantart] implement downloading PNG versions of non-original images with
"quality": "png"(#4846) - [flickr] handle non-JSON errors (#5131)
- [idolcomplex] support alphanumeric post IDs (#5171)
- [kemonoparty] implement filtering duplicate revisions with
"revisions": "unique"(#5013) - [naverwebtoon] support
/webtoon/paths for all comics (#5123)
Metadata
- [idolcomplex] extract
id_alnummetadata (#5171) - [pornpics] support multiple values for
channel(#5195) - [sankaku] add
id-formatoption (#5073) - [skeb] add
numandcountmetadata fields (#5187)
Downloaders
Fixes
- [http] remove
pyopensslimport (#5156)
Miscellaneous
1.26.7 - 2024-01-21
Extractors
Additions
- [2ch] add support (#1009, #3540, #4444)
- [deviantart:avatar] add
formatsoption (#4995) - [hatenablog] add support (#5036, #5037)
- [mangadex] add
listextractor (#5025) - [steamgriddb] add support (#5033, #5041)
- [wikimedia] add support (#1443, #2906, #3660, #2340)
- [wikimedia] support
fandomwikis (#2677, #3378)
Fixes
- [blogger] fix
lh-*.googleusercontent.comURLs (#5091) - [bunkr] update domain (#5088)
- [deviantart] fix AttributeError for URLs without username (#5065)
- [deviantart] fix
KeyError: 'premium_folder_data'(#5063) - [deviantart:avatar] fix exception when
commentsare enabled (#4995) - [fuskator] make metadata extraction non-fatal (#5039)
- [gelbooru] only log "Incomplete API response" for favorites (#5045)
- [giantessbooru] update domain
- [issuu] fix extraction
- [nijie] fix download URLs of single image posts (#5049)
- [patreon] fix
KeyError: 'name'(#5048, #5069, #5093) - [pixiv] update API headers (#5029)
- [realbooru] fix download URLs of older posts
- [twitter] revert to using
mediatimeline by default (#4953) - [vk] transform image URLs to non-blurred versions (#5017)
Improvements
- [batoto] support more mirror domains (#5042)
- [batoto] improve v2 manga URL pattern
- [gelbooru] support
alltag and URLs with empty tags (#5076) - [patreon] download
m3u8manifests with ytdl - [sankaku] support post URLs with alphanumeric IDs (#5073)
Metadata
- [batoto] improve
manga_idextraction (#5042) - [erome] fix
countmetadata - [kemonoparty] add
revision_hashmetadata (#4706, #4727, #5013) - [paheal] fix
sourcemetadata - [webtoons] extract more metadata (#5061, #5094)
Removals
- [chevereto] remove
pixl.li - [hbrowse] remove module
- [nitter] remove
nitter.lacontrevoie.fr
1.26.6 - 2024-01-06
Extractors
Additions
- [batoto] add
chapterandmangaextractors (#1434, #2111, #4979) - [deviantart] add
avatarandbackgroundextractors (#4995) - [poringa] add support (#4675, #4962)
- [szurubooru] support
snootbooru.com(#5023) - [zzup] add
galleryextractor (#4517, #4604, #4659, #4863, #5016)
Fixes
- [gelbooru] fix
favoriteextractor (#4903) - [idolcomplex] fix extraction & update URL patterns (#5002)
- [imagechest] fix loading more than 10 images in a gallery (#4469)
- [jpgfish] update domain
- [komikcast] fix
mangaextractor (#5027) - [komikcast] update domain (#5027)
- [lynxchan] update
bbw-chandomain (#4970) - [manganelo] fix extraction & recognize
.toTLDs (#5005) - [paheal] restore
extensionmetadata (#4976) - [rule34us] add fallback for
video-cdn1videos (#4985) - [weibo] fix AttributeError in
userextractor (#5022)
Improvements
- [gelbooru] show error for invalid API responses (#4903)
- [rule34] recognize URLs with
wwwsubdomain (#4984) - [twitter] raise error for invalid
strategyvalues (#4953)
Metadata
- [fanbox] add
metadataoption (#4921) - [nijie] add
countmetadata (#146) - [pinterest] add
countmetadata (#4981)
Miscellaneous
- fix and update zsh completion (#4972)
- fix
--cookies-from-browsermacOS Firefox profile path
1.26.5 - 2023-12-23
Extractors
Additions
- [deviantart] add
intermediaryoption (#4955) - [inkbunny] add
unreadextractor (#4934) - [mastodon] support non-numeric status IDs (#4936)
- [myhentaigallery] recognize
/g/URLs (#4920) - [postmill] add support (#4917, #4919)
- {shimmie2[ support
rule34hentai.net(#861, #4789, #4945)
Fixes
- [deviantart] add workaround for integer
client-idvalues (#4924) - [exhentai] fix error for infinite
fallback-retries(#4911) - [inkbunny] stop pagination on empty results
- [patreon] fix bootstrap data extraction again (#4904)
- [tumblr] fix exception after waiting for rate limit (#4916)
Improvements
- [exhentai] output continuation URL when interrupted (#4782)
- [inkbunny] improve
/submissionsviewall.phppatterns (#4934) - [tumblr] support infinite
fallback-retries - [twitter] default to
tweetstimeline whenrepliesare enabled (#4953)
Metadata
- [danbooru] provide
tagsas list (#4942) - [deviantart] set
is_originalfor intermediary URLs tofalse - [twitter] remove
date_liked(#3850, #4108, #4657)
Docker
- add Docker instructions to README (#4850)
- fix auto-generation of
latesttags
1.26.4 - 2023-12-10
Extractors
Additions
- [exhentai] add
fallback-retriesoption (#4792) - [urlgalleries] add
galleryextractor (#919, #1184, #2905, #4886)
Fixes
- [nijie] fix image URLs of multi-image posts (#4876)
- [patreon] fix bootstrap data extraction (#4904, #4906)
- [twitter] fix
/mediatimelines (#4898, #4899) - [twitter] retry API requests when response contains incomplete results (#4811)
Improvements
- [exhentai] store more cookies when logging in with username & password (#4881)
- [twitter] generalize "Login Required" errors (#4734, #4324)
Options
- add
-e/--error-filecommand-line andoutput.errorfileconfig option (#4732)
Miscellaneous
- automatically build and push Docker images
- prompt for passwords on login when necessary
- fix
util.dump_response()to work withbytesheader values
1.26.3 - 2023-11-27
Extractors
Additions
- [behance] support
textmodules (#4799) - [behance] add
modulesoption (#4799) - [blogger] support
www.micmicidol.club(#4759) - [erome] add
countmetadata (#4812) - [exhentai] add
gpoption (#4576) - [fapello] support
.suTLD (#4840, #4841) - [pixeldrain] add
fileandalbumextractors (#4839) - [pixeldrain] add
api-keyoption (#4839) - [tmohentai] add
galleryextractor (#4808, #4832)
Fixes
- [cyberdrop] update to site layout changes
- [exhentai] handle
Downloading … requires GPerrors (#4576, #4763) - [exhentai] fix empty API URL with
"source": "hitomi"(#4829) - [hentaifoundry] check for and update expired sessions (#4694)
- [hiperdex] fix
mangametadata - [idolcomplex] update to site layout changes
- [imagefap] fix resolution of single images
- [instagram] fix exception on empty
video_versions(#4795) - [mangaread] fix extraction
- [mastodon] fix reblogs (#4580)
- [nitter] fix video extraction (#4853, #4855)
- [pornhub] fix
usermetadata for gifs - [tumblr] fix
dayextractor - [wallpapercave] fix extraction
- [warosu] fix file URLs
- [webtoons] fix pagination when receiving an HTTP redirect
- [xvideos] fix metadata extraction
- [zerochan] fix metadata extraction
Improvements
- [hentaicosplays] force
https://for download URLs - [oauth] warn when cache is enabled but not writeable (#4771)
- [sankaku] update URL patterns
- [twitter] ignore promoted Tweets (#3894, #4790)
- [weibo] detect redirects to login page (#4773)
Removals
- [foolslide] remove
powermanga.org
Downloaders
Changes
- [http] treat files not passing
filesize-min/-maxas skipped (#4821)
Options
Additions
- add
metadata-extractoroption (#4549) - support
metadata-*names for*-metadataoptions (for exampleurl-metadatais now also recognized asmetadata-url)
CLI
Additions
- implement
-I/--input-file-commentand-x/--input-file-deleteoptions (#4732) - add
--ugoiraas a general version of--ugoira-convand co. - add
--mtimeas a general version of--mtime-from-date - add
--cbz
Fixes
- allow
--mtime-from-dateto work with Weibo`s metadata structure
Miscellaneous
Additions
- add a simple Dockerfile (#4831)
1.26.2 - 2023-11-04
Extractors
Additions
- [4archive] add
threadandboardextractors (#1262, #2418, #4400, #4710, #4714) - [hitomi] recognize
imagesetgallery URLs (#4756) - [kemonoparty] add
revision_indexmetadata field (#4727) - [misskey] support
misskey.design(#4713) - [reddit] support Reddit Mobile share links (#4693)
- [sankaku] support
/posts/tag search URLs (#4740) - [twitter] recognize
fixupx.comURLs (#4755)
Fixes
- [exhentai] update to site layout changes (#4730, #4754)
- [exhentai] provide fallback URLs (#1021, #4745)
- [exhentai] disable
DHciphers to avoidDH_KEY_TOO_SMALLerrors (#1021, #4593) - [idolcomplex] disable sending Referer headers (#4726)
- [instagram] update API headers
- [kemonoparty] fix parsing of non-standard
datevalues (#4676) - [patreon] fix
campaign_idextraction (#4699, #4715, #4736, #4738) - [pixiv] load cookies for non-OAuth URLs (#4760)
- [twitter] fix avatars without
dateinformation (#4696) - [twitter] restore truncated retweet texts (#3430, #4690)
- [weibo] fix Sina Visitor requests
Improvements
- [behance] unescape embed URLs (#4742)
- [fantia] simplify
tagsto a list of strings (#4752) - [kemonoparty] limit
titlelength (#4741) - [nijie] set 1-2s delay between requests to avoid 429 errors
- [patreon] provide ways to manually specify a user's campaign_id
https://www.patreon.com/id:12345https://www.patreon.com/USER?c=12345https://www.patreon.com/USER?campaign_id=12345
- [twitter] cache
user_by_…results (#4719)
Post Processors
Fixes
- [metadata] ignore non-string tag values (#4764)
Miscellaneous
Fixes
- prevent crash when
stdout.line_bufferingis not defined (#642)
1.26.1 - 2023-10-21
Extractors
Additions
- [bunkr] add extractor for media URLs (#4684)
- [chevereto] add generic extractors for
cheveretosites (#4664) - [deviantart] implement
"group": "skip"(#4630) - [fantia] add
content_countandcontent_nummetadata fields (#4627) - [imgbb] add
displaynameanduser_idmetadata (#4626) - [kemonoparty] support post revisions; add
revisionsoption (#4498, #4597) - [kemonoparty] support searches (#3385, #4057)
- [kemonoparty] support discord URLs with channel IDs (#4662)
- [moebooru] add
metadataoption (#4646) - [newgrounds] support multi-image posts (#4642)
- [sankaku] support
/posts/URLs (#4688) - [twitter] add
sensitivemetadata field (#4619)
Fixes
- [4chanarchives] disable Referer headers by default (#4686)
- [bunkr] fix
/d/file URLs (#4685) - [deviantart] expand nested comment replies (#4653)
- [deviantart] disable
jwt(#4652) - [hentaifoundry] fix
.swffile downloads (#4641) - [imgbb] fix
usermetadata extraction (#4626) - [imgbb] update pagination end condition (#4626)
- [kemonoparty] update API endpoints (#4676, #4677)
- [patreon] update
campaign_idpath (#4639) - [reddit] fix wrong previews (#4649)
- [redgifs] fix
nichesextraction (#4666, #4667) - [twitter] fix crash due to missing
source(#4620) - [warosu] fix extraction (#4634)
Post Processors
Additions
- support
{_filename},{_directory}, and{_path}replacement fields for--exec(#4633)
Miscellaneous
Improvements
- avoid temporary copies with
--cookies-from-browserby opening cookie databases in read-only mode
1.26.0 - 2023-10-03
-
Extractors
Additions
- [behance] add
datemetadata field (#4417) - [danbooru] support
booru.borvar.art(#4096) - [danbooru] support
donmai.moe - [deviantart] add
is_originalmetadata field (#4559) - [e621] support
e6ai.net(#4320) - [exhentai] add
favoption (#4409) - [gelbooru_v02] support
xbooru.com(#4493) - [instagram] add
followingextractor (#1848) - [pillowfort] support
/tagged/URLs (#4570) - [pornhub] add
gifsupport (#4463) - [reddit] add
previewsoption (#4322) - [redgifs] add
nichesextractor (#4311, #4312) - [redgifs] support
orderparameter for user URLs (#4583) - [twitter] add
userextractor andincludeoption (#4275) - [twitter] add
tweet-endpointoption (#4307) - [twitter] add
date_originalmetadata for retweets (#4337, #4443) - [twitter] extract
sourcemetadata (#4459) - [twitter] support
x.comURLs (#4452)
Improvements
- include
Refererheader in all HTTP requests (#4490, #4518) (can be disabled withrefereroption) - [behance] show errors for mature content (#4417)
- [deviantart] re-add
qualityoption and/intermediary/transform - [fantia] improve metadata extraction (#4126)
- [instagram] better error messages for invalid users (#4606)
- [mangadex] support multiple values for
lang(#4093) - [mastodon] support
/@USER/followingURLs (#4608) - [moebooru] match search URLs with empty
tags(#4354) - [pillowfort] extract
b2_lg_urlmedia (#4570) - [reddit] improve comment metadata (#4482)
- [reddit] ignore
/message/composeURLs (#4482, #4581) - [redgifs] provide
collectionmetadata as separate field (#4508) - [redgifs] match
gfycatimage URLs (#4558) - [twitter] improve error messages for single Tweets (#4369)
Fixes
- [acidimg] fix extraction
- [architizer] fix extraction (#4537)
- [behance] fix and update
userextractor (#4417) - [behance] fix cookie usage (#4417)
- [behance] handle videos without
renditions(#4523) - [bunkr] fix media domain for
cdn9(#4386, #4412) - [bunkr] fix extracting
.wmvfiles (#4419) - [bunkr] fix media domain for
cdn-pizza.bunkr.ru(#4489) - [bunkr] fix extraction (#4514, #4532, #4529, #4540)
- [deviantart] fix full resolution URLs for non-downloadable images (#293, #4548, #4563)
- [deviantart] fix shortened URLs (#4316)
- [deviantart] fix search (#4384)
- [deviantart] update Eclipse API endpoints (#4553, #4615)
- [deviantart] use private tokens for
is_matureposts (#4563) - [flickr] update default API credentials (#4332)
- [giantessbooru] fix extraction (#4373)
- [hiperdex] fix crash for titles containing Unicode characters (#4325)
- [hiperdex] fix
mangametadata - [imagefap] fix pagination (#3013)
- [imagevenue] fix extraction (#4473)
- [instagram] fix private posts with long shortcodes (#4362)
- [instagram] fix video preview archive IDs (#2135, #4455)
- [instagram] handle exceptions due to missing media (#4555)
- [issuu] fix extraction (#4420)
- [jpgfish] update domain to
jpg1.su(#4494) - [kemonoparty] update
favoriteAPI endpoint (#4522) - [lensdump] fix extraction (#4352)
- [mangakakalot] update domain
- [reddit] fix
preview.redd.itURLs (#4470) - [patreon] fix extraction (#4547)
- [pixiv] handle errors for private novels (#4481)
- [pornhub] fix extraction (#4301)
- [pururin] fix extraction (#4375)
- [subscribestar] fix preview detection (#4468)
- [twitter] fix crash on private user (#4349)
- [twitter] fix
TweetWithVisibilityResults(#4369) - [twitter] fix crash when
sortIndexis undefined (#4499) - [zerochan] fix
tagsextraction (#4315, #4319)
Removals
- [gfycat] remove module
- [shimmie2] remove
meme.museum
- [behance] add
-
Post Processors
Changes
- update
finalizeevents- add
finalize-errorandfinalize-successevents that trigger depending on whether error(s) did or did not happen - change
finalizeto always trigger regardless of error status
- add
Additions
- add
pythonpost processor - add
prepare-afterevent (#4083) - [ugoira] add
"framerate": "uniform"(#4421)
Improvements
- [ugoira] extend
ffmpeg-output(#4421)
Fixes
- update
-
Downloaders
Fixes
- [http] close connection when file already exists (#4403)
-
Options
Additions
- support
parent>childcategories for child extractor options, for example animguralbum from aredditthread withreddit>imgur - implement
subconfigsoption (#4440) - add
"ascii+"as a specialpath-restrictvalue (#4371)
Removals
- remove
pyopenssloption
- support
-
Tests
Improvements
-
Miscellaneous
Fixes
1.25.8 - 2023-07-15
Changes
- update default User-Agent header to Firefox 115 ESR
Additions
- [gfycat] support
@meuser (#3770, #4271) - [gfycat] implement login support (#3770, #4271)
- [reddit] notify users about registering an OAuth application (#4292)
- [twitter] add
ratelimitoption (#4251) - [twitter] use
TweetResultByRestIdendpoint that allows accessing single Tweets without login (#4250)
Fixes
- [bunkr] use
.laTLD formedia-files12servers (#4147, #4276) - [erome] ignore duplicate album IDs
- [fantia] send
X-Requested-Withheader (#4273) - [gelbooru_v01] fix
sourcemetadata (#4302, #4303) - [gelbooru_v01] update
vidyartdomain - [jpgfish] update domain to
jpeg.pet - [mangaread] fix
tagsmetadata extraction - [naverwebtoon] fix
comicmetadata extraction - [newgrounds] extract & pass auth token during login (#4268)
- [paheal] fix extraction (#4262, #4293)
- [paheal] unescape
source - [philomena] fix
--range(#4288) - [philomena] handle
429 Too Many Requestserrors (#4288) - [pornhub] set
accessAgeDisclaimerPHcookie (#4301) - [reddit] use 0.6s delay between API requests (#4292)
- [seiga] set
skip_fetish_warningcookie (#4242) - [slideshare] fix extraction
- [twitter] fix
followingextractor not getting all users (#4287) - [twitter] use GraphQL search endpoint by default (#4264)
- [twitter] do not treat missing
TimelineAddEntriesinstruction as fatal (#4278) - [weibo] fix cursor based pagination
- [wikifeet] fix
tagextraction (#4289, #4291)
Removals
- [bcy] remove module
- [lineblog] remove module
1.25.7 - 2023-07-02
Additions
- [flickr] add 'exif' option
- [flickr] add 'metadata' option (#4227)
- [mangapark] add 'source' option (#3969)
- [twitter] extend 'conversations' option (#4211)
Fixes
- [furaffinity] improve 'description' HTML (#4224)
- [gelbooru_v01] fix '--range' (#4167)
- [hentaifox] fix titles containing '@' (#4201)
- [mangapark] update to v5 (#3969)
- [piczel] update API server address (#4244)
- [poipiku] improve error detection (#4206)
- [sankaku] improve warnings for unavailable posts
- [senmanga] ensure download URLs have a scheme (#4235)
1.25.6 - 2023-06-17
Additions
- [blogger] download files from
lh*.googleusercontent.com(#4070) - [fantia] extract
planmetadata (#2477) - [fantia] emit warning for non-visible content sections (#4128)
- [furaffinity] extract
favorite_idmetadata (#4133) - [jschan] add generic extractors for jschan image boards (#3447)
- [kemonoparty] support
.suTLDs (#4139) - [pixiv:novel] add
novel-bookmarkextractor (#4111) - [pixiv:novel] add
full-seriesoption (#4111) - [postimage] add gallery support, update image extractor (#3115, #4134)
- [redgifs] support galleries (#4021)
- [twitter] extract
conversation_idmetadata (#3839) - [vipergirls] add login support (#4166)
- [vipergirls] use API endpoints (#4166)
- [formatter] implement
Hconversion (#4164)
Fixes
- [acidimg] fix extraction (#4136)
- [bunkr] update domain to bunkrr.su (#4159, #4189)
- [bunkr] fix video downloads
- [fanbox] prevent exception due to missing embeds (#4088)
- [instagram] fix retrieving
/taggedposts (#4122) - [jpgfish] update domain to
jpg.pet(#4138) - [pixiv:novel] fix error with embeds extraction (#4175)
- [pornhub] improve redirect handling (#4188)
- [reddit] fix crash due to empty
crosspost_parent_lists(#4120, #4172) - [redgifs] update
searchURL pattern (#4115, #4185) - [senmanga] fix and update (#4160)
- [twitter] use GraphQL API search endpoint (#3942)
- [wallhaven] improve HTTP error handling (#4192)
- [weibo] prevent fatal exception due to missing video data (#4150)
- [weibo] fix
.jsonextension for some videos
1.25.5 - 2023-05-27
Additions
- [8muses] add
partsmetadata field (#3329) - [danbooru] add
datemetadata field (#4047) - [e621] add
datemetadata field (#4047) - [gofile] add basic password support (#4056)
- [imagechest] implement API support (#4065)
- [instagram] add
order-filesoption (#3993, #4017) - [instagram] add
order-postsoption (#3993, #4017) - [instagram] add
metadataoption (#3107) - [jpgfish] add
jpg.fishingextractors (#2657, #2719) - [lensdump] add
lensdump.comextractors (#2078, #4104) - [mangaread] add
mangaread.orgextractors (#2425, #2781) - [misskey] add
favoriteextractor (#3950) - [pixiv] add
novelsupport (#1241, #4044) - [reddit] support cross-posted media (#887, #3586, #3976)
- [postprocessor:exec] support tilde expansion for
command - [formatter] support slicing strings as bytes (#4087)
Fixes
- [8muses] fix value of
album[url](#3329) - [danbooru] refactor pagination logic (#4002)
- [fanbox] skip invalid posts (#4088)
- [gofile] automatically fetch
website-token - [kemonoparty] fix kemono and coomer logins sharing the same cache (#4098)
- [newgrounds] add default delay between requests (#4046)
- [nsfwalbum] detect placeholder images
- [poipiku] extract full
descriptions(#4066) - [tcbscans] update domain to
tcbscans.com(#4080) - [twitter] extract TwitPic URLs in text (#3792, #3796)
- [weibo] require numeric IDs to have length >= 10 (#4059)
- [ytdl] fix crash due to removed
no_colorattribute - [cookies] improve logging behavior (#4050)
1.25.4 - 2023-05-07
Additions
- [4chanarchives] add
threadandboardextractors (#4012) - [foolfuuka] add
archive.palanq.win - [imgur] add
favorite-folderextractor (#4016) - [mangadex] add
statusandtagsmetadata (#4031) - allow selecting a domain with
--cookies-from-browser - add
--cookies-exportcommand-line option - add
-Cas short option for--cookies - include exception type in config error messages
Fixes
- [exhentai] update sadpanda check
- [imagechest] load all images when a "Load More" button is present (#4028)
- [imgur] fix bug causing some images/albums from user profiles and favorites to be ignored
- [pinterest] update endpoint for related board pins
- [pinterest] fix
pin.itextractor - [ytdl] fix yt-dlp
--xff/--geo-bypasstests (#3989)
Removals
- [420chan] remove module
- [foolfuuka] remove
archive.alice.alandtokyochronos.net - [foolslide] remove
sensescans.com - [nana] remove module
1.25.3 - 2023-04-30
Additions
- [imagefap] extract
descriptionandcategoriesmetadata (#3905) - [imxto] add
galleryextractor (#1289) - [itchio] add
gameextractor (#3923) - [nitter] extract user IDs from encoded banner URLs
- [pixiv] allow sorting search results by popularity (#3970)
- [reddit] match
preview.redd.itURLs (#3935) - [sankaku] support post URLs with MD5 hashes (#3952)
- [shimmie2] add generic extractors for Shimmie2 sites (#3734, #943)
- [tumblr] add
dayextractor (#3951) - [twitter] support
profile-conversationentries (#3938) - [vipergirls] add
threadandpostextractors (#3812, #2720, #731) - [downloader:http] add
consume-contentoption (#3748)
Fixes
- [2chen] update domain to sturdychan.help
- [behance] fix extraction (#3980)
- [deviantart] retry downloads with private token (#3941)
- [imagefap] fix empty
tagsmetadata - [manganelo] support arbitrary minor version separators (#3972)
- [nozomi] fix file URLs (#3925)
- [oauth] catch exceptions from
webbrowser.get()(#3947) - [pixiv] fix
pixivisionextraction - [reddit] ignore
id-maxvalue"zik0zj"/2147483647(#3939, #3862, #3697, #3606, #3546, #3521, #3412) - [sankaku] sanitize
date:tags (#1790) - [tumblr] fix and update pagination logic (#2191)
- [twitter] fix
usermetadata when downloading quoted Tweets (#3922) - [ytdl] fix crash due to
--geo-bypassdeprecation (#3975) - [postprocessor:metadata] support putting keys in quotes
- include more optional dependencies in executables (#3907)
1.25.2 - 2023-04-15
Additions
- [deviantart] add
publicoption - [nitter] extract videos from
sourceelements (#3912) - [twitter] add
date_likedanddate_bookmarkedmetadata for liked and bookmarked Tweets (#3816) - [urlshortener] add support for bit.ly & t.co (#3841)
- [downloader:http] add MIME type and signature for
.heicfiles (#3915)
Fixes
- [blogger] update regex to get the highest resolution URLs (#3863, #3870)
- [bunkr] update domain to
bunkr.la(#3813, #3877) - [deviantart] keep using private access tokens when requesting download URLs (#3845, #3857, #3896)
- [hentaifoundry] fix content filters (#3887)
- [hotleak] fix downloading of creators whose name starts with a category name (#3871)
- [imagechest] fix extraction (#3914)
- [realbooru] fix extraction (#2530)
- [sexcom] fix pagination (#3906)
- [sexcom] fix HD video extraction
- [shopify] fix
collectionextractor (#3866, #3868) - [twitter] update to bookmark timeline v2 (#3859, #3854)
- [twitter] warn about "withheld" Tweets and users (#3864)
Improvements
- [danbooru] reduce number of API requests when fetching extended
metadata - [deviantart:search] detect login redirects (#3860)
- [generic] write regular expressions without
xflags - [mastodon] try to get account IDs without access token
- [twitter] calculate
datefrom Tweet IDs
1.25.1 - 2023-03-25
Additions
- [nitter] support nitter.it (#3819)
- [twitter] add
hashtagextractor (#3783) - [twitter] support Tweet content with >280 characters
- [formatter] support loading f-strings from template files (#3800)
- [formatter] support filesystem paths for
\fMmodules (#3399) - [formatter] support putting keys in quotes (e.g.
user['name']) (#2559) - [postprocessor:metadata] add
skipoption (#3786)
Fixes
- [output] set
errors=replacefor output streams (#3765) - [gelbooru] extract favorites without needing cookies (#3704)
- [gelbooru] fix and improve
--rangefor pools - [hiperdex] fix extraction (#3768)
- [naverwebtoon] fix extraction (#3729)
- [nitter] fix extraction for instances without user banners
- [twitter] update API query hashes and parameters
- [weibo] support
mix_media_infoentries (#3793) - fix circular reference detection for
-K
Changes
- update
globalsinstead of overwriting the default (#3773)
1.25.0 - 2023-03-11
Changes
- [e621] split
e621extractors fromdanboorumodule (#3425) - [deviantart] remove mature scraps warning (#3691)
- [deviantart] use
/collections/allendpoint for favorites (#3666, #3668) - [newgrounds] update default image and audio archive IDs to prevent ID overlap (#3681)
- rename
--ignore-configto--config-ignore
Extractors
- [catbox] add
fileextractor (#3570) - [deviantart] add
searchextractor (#538, #1264, #2954, #2970, #3577) - [deviantart] add
gallery-searchextractor (#1695) - [deviantart] support
fxdeviantart.comURLs (##3740) - [e621] implement
notesandpoolsmetadata extraction (#3425) - [gelbooru] add
favoriteextractor (#3704) - [imagetwist] support
phun.imagetwist.comandimagehaha.comdomains (#3622) - [instagram] add
usermetadata field (#3107) - [manganelo] update and fix metadata extraction
- [manganelo] support mobile-only chapters
- [mangasee] extract
authorandgenremetadata (#3703) - [misskey] add
misskeyextractors (#3717) - [pornpics] add
galleryandsearchextractors (#263, #3544, #3654) - [redgifs] support v3 URLs (#3588. #3589)
- [redgifs] add
collectionextractors (#3427, #3662) - [shopify] support ohpolly.com (#440, #3596)
- [szurubooru] add
tagandpostextractors (#3583, #3713) - [twitter] add
transformoption
Options
- [postprocessor:metadata] add
sortandseparatorsoptions - [postprocessor:exec] implement archive options (#3584)
- add
--config-createcommand-line option (#2333) - add
--config-tomlcommand-line option to load config files in TOML format - add
output.stdout,output.stdin, andoutput.stderroptions (#1621, #2152, #2529) - add
hash_md5andhash_sha1functions (#3679) - implement
globalsoption to enable defining custom functions forevalstatements - implement
archive-pragmaoption to use SQLite PRAGMA statements - implement
actionsto trigger events on logging messages (#3338, #3630) - implement ability to load external extractor classes
-X/--extractorscommand-line optionsextractor.modules-sourcesconfig option
Fixes
- [bunkr] fix extraction (#3636, #3655)
- [danbooru] send gallery-dl User-Agent (#3665)
- [deviantart] fix crash when handling deleted deviations in status updates (#3656)
- [fanbox] fix crash with missing images (#3673)
- [imagefap] update
galleryURLs (#3595) - [imagefap] fix infinite pagination loop (#3594)
- [imagefap] fix metadata extraction
- [oauth] use default name for browsers without
nameattribute - [pinterest] unescape search terms (#3621)
- [pixiv] fix
--write-tagsfor"tags": "original"(#3675) - [poipiku] warn about incorrect passwords (#3646)
- [reddit] update
videosoption (#3712) - [soundgasm] rewrite (#3578)
- [telegraph] fix extraction when images are not in
<figure>elements (#3590) - [tumblr] raise more detailed errors for dashboard-only blogs (#3628)
- [twitter] fix some
originalretweets not downloading (#3744) - [ytdl] fix
--parse-metadata(#3663) - [downloader:ytdl] prevent exception on empty results
Improvements
- [downloader:http] use
time.monotonic() - [downloader:http] update
_http_retryto accept a Python function (#3569) - [postprocessor:metadata] speed up JSON encoding
- replace
json.loads/dumpswith direct calls toJSONDecoder.decode/JSONEncoder.encode - improve
option.Formatterperformance
Removals
- [nitter] remove
nitter.pussthecat.org
1.24.5 - 2023-01-28
Additions
- [booru] add
urloption - [danbooru] extend
metadataoption (#3505) - [deviantart] add extractor for status updates (#3539, #3541)
- [deviantart] add support for
/deviation/andfav.meURLs (#3558, #3560) - [kemonoparty] extract
hashmetadata for discord files (#3531) - [lexica] add
searchextractor (#3567) - [mastodon] add
numandcountmetadata fields (#3517) - [nudecollect] add
imageandalbumextractors (#2430, #2818, #3575) - [wikifeet] add
galleryextractor (#519, #3537) - [downloader:http] add signature checks for
.blend,.obj, and.clipfiles (#3535) - add
extractor.retry-codesoption - add
-O/--postprocessor-optioncommand-line option (#3565) - improve
write-pagesoutput
Fixes
- [bunkr] fix downloading
.mkvand.tsfiles (#3571) - [fantia] send
X-CSRF-Tokenheaders (#3576) - [generic] fix regex for non-src image URLs (#3555)
- [hiperdex] update domain (#3572)
- [hotleak] fix video URLs (#3516, #3525, #3563, #3581)
- [instagram] always show
cursorvalue after errors (#3440) - [instagram] update API domain, headers, and csrf token handling
- [oauth] show
client-id/api-keyvalues (#3518) - [philomena] match URLs with www subdomain
- [sankaku] update URL pattern (#3523)
- [twitter] refresh guest tokens (#3445, #3458)
- [twitter] fix search pagination (#3536, #3534, #3549)
- [twitter] use
"browser": "firefox"by default (#3522)
1.24.4 - 2023-01-11
Additions
- [downloader:http] add
validateoption
Fixes
1.24.3 - 2023-01-10
Additions
- [danbooru] extract
uploadermetadata (#3457) - [deviantart] initial implementation of username & password login for
scraps(#1029) - [fanleaks] add
postandmodelextractors (#3468, #3474) - [imagefap] add
folderextractor (#3504) - [lynxchan] support
bbw-chan.nl(#3456, #3463) - [pinterest] support
All Pinsboards (#2855, #3484) - [pinterest] add
domainoption (#3484) - [pixiv] implement
metadata-bookmarkoption (#3417) - [tcbscans] add
chapterandmangaextractors (#3189) - [twitter] implement
syndication=extended(#3483) - implement slice notation for
rangeoptions (#918, #2865) - allow
filteroptions to be a list of expressions
Fixes
- [behance] use delay between requests (#2507)
- [bunkr] fix URLs returned by API (#3481)
- [fanbox] return
imageMapfiles in order (#2718) - [imagefap] use delay between requests (#1140)
- [imagefap] warn about redirects to
/human-verification(#1140) - [kemonoparty] reject invalid/empty files (#3510)
- [myhentaigallery] handle whitespace before title tag (#3503)
- [poipiku] fix extraction for a different warning button style (#3493, #3460)
- [poipiku] warn about login requirements
- [telegraph] fix file URLs (#3506)
- [twitter] fix crash when using
expandandsyndication(#3473) - [twitter] apply tweet type checks before uniqueness check (#3439, #3455)
- [twitter] force
https://for TwitPic URLs (#3449) - [ytdl] adapt to yt-dlp changes
- update and improve documentation (#3453, #3462, #3496)
1.24.2 - 2022-12-18
Additions
- [2chen] support
.clubURLs (#3406) - [deviantart] extract sta.sh URLs from
text_content(#3366) - [deviantart] add
/viewURL support (#3367) - [e621] implement
thresholdoption to control pagination (#3413) - [fapello] add
post,userandpathextractors (#3065, #3360, #3415) - [imgur] add support for imgur.io URLs (#3419)
- [lynxchan] add generic extractors for lynxchan imageboards (#3389, #3394)
- [mangafox] extract more metadata (#3167)
- [pixiv] extract
date_urlmetadata (#3405) - [soundgasm] add
audioanduserextractors (#3384, #3388) - [webmshare] add
videoextractor (#2410) - support Firefox containers for
--cookies-from-browser(#3346)
Fixes
- [2chen] fix file URLs
- [bunkr] update domain (#3391)
- [exhentai] fix pagination
- [imagetwist] fix extraction
- [imgth] rewrite
- [instagram] prevent post
dateoverwriting filedate(#3392) - [khinsider] fix metadata extraction
- [komikcast] update domain and fix extraction
- [reddit] increase
id-maxdefault value (#3397) - [seiga] raise error when redirected to login page (#3401)
- [sexcom] fix video URLs (#3408, #3414)
- [twitter] update
searchpagination (#544) - [warosu] fix and update
- [zerochan] update for layout v3
- restore paths for archived files (#3362, #3377)
- use
util.NONEaskeyword-defaultdefault value (#3334)
Removals
- [foolslide] remove
kireicake - [kissgoddess] remove module
1.24.1 - 2022-12-04
Additions
- [artstation] add
pro-firstoption (#3273) - [artstation] add
max-postsoption (#3270) - [fapachi] add
postanduserextractors (#3339, #3347) - [inkbunny] provide additional metadata (#3274)
- [nitter] add
retweetsoption (#3278) - [nitter] add
videosoption (#3279) - [nitter] support
/i/web/and/i/user/URLs (#3310) - [pixhost] add
gallerysupport (#3336, #3353) - [weibo] add
countmetadata field (#3305) - [downloader:http] add
retry-codesoption (#3313) - [formatter] implement
Sformat specifier to sort lists (#3266) - implement
version-metadataoption (#3201)
Fixes
- [2chen] fix extraction (#3354, #3356)
- [bcy] fix JSONDecodeError (#3321)
- [bunkr] fix video downloads (#3326, #3335)
- [bunkr] use
media-filesservers for more file types - [itaku] remove
Extremerating (#3285, #3287) - [hitomi] apply format check for every image (#3280)
- [hotleak] fix UnboundLocalError (#3288, #3293)
- [nitter] sanitize filenames (#3294)
- [nitter] retry downloads on 404 (#3313)
- [nitter] set
hlsPlaybackcookie - [patreon] fix
403 Forbiddenerrors (#3341) - [patreon] improve
campaign_idextraction (#3235) - [patreon] update API query parameters
- [pixiv] preserve
tagsorder (#3266) - [reddit] use
dash_urlfor videos (#3258, #3306) - [twitter] fix error when using user IDs for suspended accounts
- [weibo] fix bug with empty
playback_list(#3301) - [downloader:http] fix potential
ZeroDivisionError(#3328)
Removals
- [lolisafe] remove
zz.ht
1.24.0 - 2022-11-20
Additions
- [exhentai] add metadata to search results (#3181)
- [gelbooru_v02] implement
notesextraction - [instagram] add
guideextractor (#3192) - [lolisafe] add support for xbunkr (#3153, #3156)
- [mastodon] add
instance_remotemetadata field (#3119) - [nitter] add extractors for Nitter instances (#2415, #2696)
- [pixiv] add support for new daily AI rankings category (#3214, #3221)
- [twitter] add
avatarandbackgroundextractors (#349, #3023) - [uploadir] add support for
uploadir.com(#3162) - [wallhaven] add
userextractor (#3212, #3213, #3226) - [downloader:http] add
chunk-sizeoption (#3143) - [downloader:http] add file signature check for
.mp4files - [downloader:http] add file signature check and MIME type for
.aviffiles - [postprocessor] implement
post-afterevent (#3117) - [postprocessor:metadata] implement
"mode": "jsonl" - [postprocessor:metadata] add
open,encoding, andprivateoptions - add
--chunk-sizecommand-line option (#3143) - add
--user-agentcommand-line option - implement
http-metadataoption - implement
"user-agent": "browser"(#2636)
Changes
- [deviantart] restore cookies warning for mature scraps (#3129)
- [instagram] use REST API for unauthenticated users by default
- [downloader:http] increase default
chunk-sizeto 32768 bytes (#3143) - build Windows executables using py2exe's new
freeze()API - build executables on GitHub Actions with Python 3.11
- reword error text for unsupported URLs
Fixes
- [exhentai] fix pagination (#3181)
- [khinsider] fix extraction (#3215, #3219)
- [realbooru] fix download URLs (#2530)
- [realbooru] fix
tagsextraction (#2530) - [tumblr] fall back to
gifvwhen possible (#3095, #3159) - [twitter] fix login (#3220)
- [twitter] update URL for syndication API (#3160)
- [weibo] send
Refererheaders (#3188) - [ytdl] update
parse_byteslocation (#3256)
Improvements
- [imxto] extract additional metadata (#3118, #3175)
- [instagram] allow downloading avatars for private profiles (#3255)
- [pixiv] raise error for invalid search/ranking parameters (#3214)
- [twitter] update
bookmarkspagination (#3172) - [downloader:http] refactor file signature checks
- [downloader:http] improve
-r/--limit-rateaccuracy (#3143) - add loaded config files to debug output
- improve
-Koutput for lists
Removals
- [instagram] remove login support (#3139, #3141, #3191)
- [instagram] remove
channelextractor - [ngomik] remove module
1.23.5 - 2022-10-30
Fixes
- [instagram] fix AttributeError on user stories extraction (#3123)
1.23.4 - 2022-10-29
Additions
- [aibooru] add support for aibooru.online (#3075)
- [instagram] add 'avatar' extractor (#929, #1097, #2992)
- [instagram] support 'instagram.com/s/' highlight URLs (#3076)
- [instagram] extract 'coauthors' metadata (#3107)
- [mangasee] add support for 'mangalife' (#3086)
- [mastodon] add 'bookmark' extractor (#3109)
- [mastodon] support cross-instance user references and '/web/' URLs (#3109)
- [moebooru] implement 'notes' extraction (#3094)
- [pixiv] extend 'metadata' option (#3057)
- [reactor] match 'best', 'new', 'all' URLs (#3073)
- [smugloli] add 'smugloli' extractors (#3060)
- [tumblr] add 'fallback-delay' and 'fallback-retries' options (#2957)
- [vichan] add generic extractors for vichan imageboards
Fixes
- [bcy] fix extraction (#3103)
- [gelbooru] support alternate parameter order in post URLs (#2821)
- [hentai2read] support minor versions in chapter URLs (#3089)
- [hentaihere] support minor versions in chapter URLs
- [kemonoparty] fix 'dms' extraction (#3106)
- [kemonoparty] update pagination offset
- [manganelo] update domain to 'chapmanganato.com' (#3097)
- [pixiv] use 'exact_match_for_tags' as default search mode (#3092)
- [redgifs] fix 'token' extraction (#3080, #3081)
- [skeb] fix extraction (#3112)
- improve compatibility of DownloadArchive (#3078)
1.23.3 - 2022-10-15
Additions
- [2chen] Add
2chen.moeextractor (#2707) - [8chan] add
threadandboardextractors (#2938) - [deviantart] add
groupoption (#3018) - [fanbox] add
contentmetadata field (#3020) - [instagram] restore
cursorfunctionality (#2991) - [instagram] restore warnings for private profiles (#3004, #3045)
- [nana] add
nanaextractors (#2967) - [nijie] add
feedandfollowedextractors (#3048) - [tumblr] support
https://www.tumblr.com/BLOGNAMEURLs (#3034) - [tumblr] add
offsetoption - [vk] add
taggedextractor (#2997) - add
path-extendedoption (#3021) - emit debug logging messages before calling time.sleep() (#2982)
Changes
- [postprocessor:metadata] assume
"mode": "custom"whenformatis given
Fixes
- [artstation] skip missing projects (#3016)
- [danbooru] fix ugoira metadata extraction (#3056)
- [deviantart] fix
deviationextraction (#2981) - [hitomi] fall back to
webpwhen selected format is not available (#3030) - [imagefap] fix and improve folder extraction and gallery pagination (#3013)
- [instagram] fix login (#3011, #3015)
- [nozomi] fix extraction (#3051)
- [redgifs] fix extraction (#3037)
- [tumblr] sleep between fallback retries (#2957)
- [vk] unescape error messages
- fix duplicated metadata bug with
-j(#3033) - fix bug when processing input file comments (#2808)
1.23.2 - 2022-10-01
Additions
- [artstation] support search filters (#2970)
- [blogger] add
labelandquerymetadata fields (#2930) - [exhentai] add a slash to the end of gallery URLs (#2947)
- [instagram] add
countmetadata field (#2979) - [instagram] add
apioption - [kemonoparty] add
countmetadata field (#2952) - [mastodon] warn about moved accounts (#2939)
- [newgrounds] add
gamesextractor (#2955) - [newgrounds] extract
typemetadata - [pixiv] add
seriesextractor (#2964) - [sankaku] implement
refreshoption (#2958) - [skeb] add
searchextractor andfiltersoption (#2945)
Fixes
- [deviantart] fix extraction (#2981, #2983)
- [fappic] fix extraction
- [instagram] extract higher-resolution photos (#2666)
- [instagram] fix
usernameandfullnamemetadata for saved posts (#2911) - [instagram] update API headers
- [kemonoparty] send
Refererheaders (#2989, #2990) - [kemonoparty] restore
favoritesAPI endpoints (#2994) - [myportfolio] use fallback when no images are found (#2959)
- [plurk] fix extraction (#2977)
- [sankaku] detect expired links (#2958)
- [tumblr] retry extraction of failed higher-resolution images (#2957)
1.23.1 - 2022-09-18
Additions
- [flickr] add support for
secure.flickr.comURLs (#2910) - [hotleak] add hotleak extractors (#2890, #2909)
- [instagram] add
highlight_titleanddatemetadata for highlight downloads (#2879) - [paheal] add support for videos (#2892)
- [tumblr] fetch high-quality inline images (#2877)
- [tumblr] implement
ratelimitoption (#2919) - [twitter] add general support for unified cards (#2875)
- [twitter] implement
cards-blacklistoption (#2875) - [zerochan] add
metadataoption (#2861) - [postprocessor:zip] implement
filesoption (#2872)
Fixes
- [bunkr] fix extraction (#2903)
- [bunkr] use
media-filesservers form4vandmovdownloads (#2925) - [exhentai] improve 509.gif detection (#2901)
- [exhentai] guess extension for original files (#2842)
- [poipiku] use
img-org.poipiku.comas image domain (#2796) - [reddit] prevent exception with empty submission URLs (#2913)
- [redgifs] fix download URLs (#2884)
- [smugmug] update default API credentials (#2881)
- [twitter] provide proper
datefor syndication results (#2920) - [twitter] fix new-style
/card_img/URLs - remove all whitespace before comments after input file URLs (#2808)
1.23.0 - 2022-08-28
Changes
- [twitter] update
userandauthormetdata fields- for URLs with a single username or ID like
https://twitter.com/USERor a search with a singlefrom:statement,userwill now always refer to the user referenced in the URL. - for all other URLs like
https://twitter.com/i/bookmarks,userandauthorrefer to the same user authorwill always refer to the original Tweet author
- for URLs with a single username or ID like
- [twitter] update
quote_idandquote_bymetadata fieldsquote_idis now non-zero for quoted Tweets and contains the Tweet ID of the quotng Tweet (was the other way round before)quote_byis only defined for quoted Tweets like before, but now contains the screen name of the user quoting this Tweet
- [skeb] improve archive IDs for thumbnails and article images
Additions
- [artstation] add
numandcountmetadata fields (#2764) - [catbox] add
albumextractor (#2410) - [blogger] emit metadata for posts without files (#2789)
- [foolfuuka] update supported domains
- [gelbooru] add support for
api_keyanduser_id(#2767) - [gelbooru] implement pagination for
poolresults (#2853) - [instagram] add support for a user's saved collections (#2769)
- [instagram] provide
datefor directory format strings (#2830) - [kemonoparty] add
favoritesoption (#2826, #2831) - [oauth] add
hostconfig option (#2806) - [rule34] implement pagination for
poolresults (#2853) - [skeb] add option to download
articleimages (#1031) - [tumblr] download higher-quality images (#2761)
- [tumblr] add
countmetadata field (#2804) - [wallhaven] implement
metadataoption (#2803) - [zerochan] add
tagandimageextractors (#1434) - [zerochan] implement login with username & password (#1434)
- [postprocessor:metadata] implement
mode: modifyandmode: delete(#2640) - [formatter] add
gconversion for slugifying a string (#2410) - [formatter] apply
:Jonly to lists (#2833) - implement
path-metadataoption (#2734) - allow comments after input file URLs (#2808)
- add global
warningsoption to controlurllib3warning behavior (#2762)
Fixes
- [bunkr] fix extraction (#2788)
- [deviantart] use public access token for journals (#2702)
- [e621] fix extraction of
popularposts - [fanbox] download cover images in original size (#2784)
- [mastodon] allow downloading without access token (#2782)
- [hitomi] update cache expiry time (#2863)
- [hitomi] fix error when number of tag results is a multiple of 25 (#2870)
- [mangahere] fix
page-reverseoption (#2795) - [poipiku] fix posts with more than one image (#2796)
- [poipiku] update filter for static images (#2796)
- [slideshare] fix metadata extraction
- [twitter] unescape
+in search queries (#2226) - [twitter] fall back to unfiltered search (#2766)
- [twitter] ignore invalid user entries (#2850)
- [vk] prevent exceptions for broken/invalid photos (#2774)
- [vsco] fix
collectionextraction - [weibo] prevent exception for missing
playback_list(#2792) - [weibo] prevent errors when paginating over album entries (#2817)
1.22.4 - 2022-07-15
Additions
- [instagram] add
pinnedmetadata field (#2752) - [itaku] categorize sections by group (#1842)
- [khinsider] extract
platformmetadata - [tumblr] support
/blog/viewURLs (#2760) - [twitter] implement
strategyoption (#2712) - [twitter] add
countmetadata field (#2741) - [formatter] implement
Oformat specifier (#2736) - [postprocessor:mtime] add
valueoption (#2739) - add
--no-postprocessorscommand-line option (#2725) - implement
format-separatoroption (#2737)
Changes
- [pinterest] handle section pins with separate extractors (#2684)
- [postprocessor:ugoira] enable
mtimeby default (#2714)
Fixes
- [bunkr] fix extraction (#2732)
- [hentaifoundry] fix metadata extraction
- [itaku] fix user caching (#1842)
- [itaku] fix
dateparsing - [kemonoparty] ensure all files have an
extension(#2740) - [komikcast] update domain
- [mangakakalot] update domain
- [newgrounds] only attempt to login if necessary (#2715)
- [newgrounds] prevent exception on empty results (#2727)
- [nozomi] reduce memory consumption during searches (#2754)
- [pixiv] fix default
backgroundfilenames - [sankaku] rewrite file URLs to s.sankakucomplex.com (#2746)
- [slideshare] fix
descriptionextraction - [twitter] ignore previously seen Tweets (#2712)
- [twitter] unescape HTML entities in
content(#2757) - [weibo] handle invalid or broken status objects
- [postprocessor:zip] ensure target directory exists (#2758)
- make
brotlian optional dependency (#2716) - limit path length for
--write-pagesoutput on Windows (#2733)
Removals
- [foolfuuka] remove archive.wakarimasen.moe
1.22.3 - 2022-06-28
Changes
1.22.2 - 2022-06-27
Additions
- [cyberdrop] add fallback URLs (#2668)
- [horne] add support for horne.red (#2700)
- [itaku] add
galleryandimageextractors (#1842) - [poipiku] add
userandpostextractors (#1602) - [skeb] add
followingextractor (#2698) - [twitter] implement
expandoption (#2665) - [twitter] implement
csrfoption (#2676) - [unsplash] add
collection_titleandcollection_idmetadata fields (#2670) - [weibo] support
tabtype=videolistings (#2601) - [formatter] implement slice operator as format specifier
- support cygwin/BSD/etc for
--cookies-from-browser
Fixes
- [instagram] improve metadata generated by
_parse_post_api()(#2695, #2660) - [instagram} fix
tagextractor (#2659) - [instagram] automatically invalidate expired login sessions
- [twitter] fix pagination for conversion tweets
- [twitter] improve
"replies": "self"(#2665) - [twitter] improve strategy for user URLs (#2665)
- [vk] take URLs from
*_srcentries (#2535) - [weibo] fix URLs generated by
userextractor (#2601) - [weibo] fix retweets (#2601)
- [downloader:ytdl] update
_set_outtmpl()(#2692) - [formatter] fix
!jconversion for non-serializable types (#2624) - [snap] Fix missing libslang dependency (#2655)
1.22.1 - 2022-06-04
Additions
- [gfycat] add support for collections (#2629)
- [instagram] support specifying users by ID
- [paheal] extract more metadata (#2641)
- [reddit] add
homeextractor (#2614) - [weibo] support usernames in URLs (#1662)
- [weibo] support
livephotoandgiffiles (#2146) - [weibo] add support for several different
tabtypelistings (#686, #2601) - [postprocessor:metadata] write to stdout by setting filename to "-" (#2624)
- implement
output.ansioption (#2628) - support user-defined
output.modesettings (#2529)
Changes
- [readcomiconline] remove default
browsersetting (#2625) - [weibo] switch to desktop API (#2601)
- fix command-line argument name of
--cookies-from-browser(#1606, #2630)
Fixes
- [bunkr] change domain to
app.bunkr.is(#2634) - [deviantart] fix folder listings with
"pagination": "manual"(#2488) - [gofile] fix 401 Unauthorized errors (#2632)
- [hypnohub] move to gelbooru_v02 instances (#2631)
- [instagram] fix and update extractors (#2644)
- [nozomi] remove slashes from search terms (#2653)
- [pixiv] include
.gifin background fallback URLs (#2495) - [sankaku] extend URL patterns (#2647)
- [subscribestar] fix
datemetadata (#2642)
1.22.0 - 2022-05-25
Additions
- [gelbooru_v01] add
favoriteextractor (#2546) - [Instagram] add
tagged_usersto keywords for stories (#2582, #2584) - [lolisafe] implement
domainoption (#2575) - [naverwebtoon] support (best)challenge comics (#2542)
- [nijie] support /history_nuita.php listings (#2541)
- [pixiv] provide more data when
metadatais enabled (#2594) - [shopify] support several more sites by default (#2089)
- [twitter] extract alt texts as
description(#2617) - [twitter] recognize vxtwitter URLs (#2621)
- [weasyl] implement
metadataoption (#2610) - implement
--cookies-from-browser(#1606) - implement
output.colorsoptions (#2532) - implement string literals in replacement fields
- support using extended format strings for archive keys
Changes
- [foolfuuka] match 4chan filenames (#2577)
- [pixiv] implement
includeoption- provide
avatar/backgrounddownloads as separate extractors (#2495)
- provide
- [twitter] use a better strategy for user URLs
- [twitter] disable
cardsby default - delay directory creation (#2461, #2474)
- flush writes to stdout/stderr (#2529)
- build executables on GitHub Actions with Python 3.10
Fixes
- [artstation] use
"browser": "firefox"by default (#2527) - [imgur] prevent exception with empty albums (#2557)
- [instagram] report redirects to captcha challenges (#2543)
- [khinsider] fix metadata extraction (#2611)
- [mangafox] send Referer headers (#2592)
- [mangahere] send Referer headers (#2592)
- [mangasee] use randomly generated PHPSESSID cookie (#2560)
- [pixiv] make retrieving ugoira metadata non-fatal (#2562)
- [readcomiconline] update deobfuscation code (#2481)
- [realbooru] fix extraction (#2530)
- [vk] handle photos without width/height info (#2535)
- [vk] fix user ID extraction (#2535)
- [webtoons] extract real episode numbers (#2591)
- create missing directories for archive files (#2597)
- detect circular references with
-K(#2609) - replace "\f" in
--filenamearguments with a form feed character (#2396)
Removals
- [gelbooru_v01] remove tlb.booru.org from supported domains
1.21.2 - 2022-04-27
Additions
- [deviantart] implement
paginationoption (#2488) - [pixiv] implement
backgroundoption (#623, #1124, #2495) - [postprocessor:ugoira] report ffmpeg/mkvmerge errors (#2487)
Fixes
- [cyberdrop] match cyberdrop.to URLs (#2496)
- [e621] fix 403 errors (#2533)
- [issuu] fix extraction (#2483)
- [mangadex] download from available chapters despite
externalUrl(#2503) - [photovogue] update domain and api endpoint (#2494)
- [sexcom] add fallback for empty files (#2485)
- [twitter] improve syndication video selection (#2354)
- [twitter] fix various syndication issues (#2499, #2354)
- [vk] fix extraction (#2512)
- [weibo] fix infinite retries for deleted accounts (#2521)
- [postprocessor:ugoira] use compatible paths with mkvmerge (#2487)
- [postprocessor:ugoira] do not auto-select the
image2demuxer (#2492)
1.21.1 - 2022-04-08
Additions
- [gofile] add gofile.io extractor (#2364)
- [instagram] add
previewsoption (#2135) - [kemonoparty] add
duplicatesoption (#2440) - [pinterest] add extractor for created pins (#2452)
- [pinterest] support multiple files per pin (#1619, #2452)
- [telegraph] Add telegra.ph extractor (#2312)
- [twitter] add
syndicationoption (#2354) - [twitter] accept fxtwitter.com URLs (#2484)
- [downloader:http] support using an arbitrary method and sending POST data (#2433)
- [postprocessor:metadata] implement archive options (#2421)
- [postprocessor:ugoira] add
mtimeoption (#2307) - [postprocessor:ugoira] support setting timecodes with
mkvmerge(#1550) - [formatter] support evaluating f-string literals
- add
--ugoira-conv-copycommand-line option (#1550) - implement a
contains()function for filter statements (#2446)
Fixes
- [aryion] provide correct
datemetadata independent of DST - [furaffinity] fix search result pagination (#2402)
- [hitomi] update and fix metadata extraction (#2444)
- [kissgoddess] extract all images (#2473)
- [mangasee] unescape manga names (#2454)
- [newgrounds] update and fix pagination (#2456)
- [newgrounds] warn about age-restricted posts (#2456)
- [pinterest] do not force
m3u8_nativefor video downloads (#2436) - [twibooru] fix posts without
name(#2434) - [unsplash] replace dash with space in search API queries (#2429)
- [postprocessor:mtime] fix timestamps from datetime objects (#2307)
- fix yet another bug in
_check_cookies()(#2372) - fix loading/storing cookies without domain
1.21.0 - 2022-03-14
Additions
- [fantia] add
numenumeration index (#2377) - [fantia] support "Blog Post" content (#2381)
- [imagebam] add support for /view/ paths (#2378)
- [kemonoparty] match beta.kemono.party URLs (#2348)
- [kissgoddess] add
galleryandmodelextractors (#1052, #2304) - [mememuseum] add
tagandpostextractors (#2264) - [newgrounds] add
post_urlmetadata field (#2328) - [patreon] add
image_largefile type (#2257) - [toyhouse] support
artlistings (#1546, #2331) - [twibooru] add extractors for searches, galleries, and posts (#2219)
- [postprocessor:metadata] implement
mtimeoption (#2307) - [postprocessor:mtime] add
eventoption (#2307) - add fish shell completion (#2363)
- add
timedeltaclass to global namespace in filter expressions
Changes
- [seiga] require authentication with
user_sessioncookie (#2372)- remove username & password login due to 2FA
- refactor proxy support (#2357)
- allow gallery-dl proxy settings to overwrite environment proxies
- allow specifying different proxies for data extraction and download
Fixes
- [bunkr] fix mp4 downloads (#2239)
- [fanbox] fetch data for each individual post (#2388)
- [hentaicosplays] send
Refererheader (#2317) - [imagebam] set
nsfw_intercookie (#2334) - [kemonoparty] limit default filename length (#2373)
- [mangadex] fix chapters without
translatedLanguage(#2352) - [newgrounds] fix video descriptions (#2328)
- [skeb] add
sent-requestsoption (#2322, #2330) - [slideshare] fix extraction
- [subscribestar] unescape attachment URLs (#2370)
- [twitter] fix handling of 429 Too Many Requests responses (#2339)
- [twitter] warn about age-restricted Tweets (#2354)
- [twitter] handle Tweets with "softIntervention" entries
- [twitter] update query hashes
- fix another bug in
_check_cookies()(#2160)
1.20.5 - 2022-02-14
Additions
- [furaffinity] add
layoutoption (#2277) - [lightroom] add Lightroom gallery extractor (#2263)
- [reddit] support standalone submissions on personal user pages (#2301)
- [redgifs] support i.redgifs.com URLs (#2300)
- [wallpapercave] add extractor for images and search results (#2205)
- add
signals-ignoreoption (#2296)
Changes
- [danbooru] merge
danbooruande621extractors- support
atfbooru(#2283) - remove support for old e621 tag search URLs
- support
Fixes
- [furaffinity] improve new/old layout detection (#2277)
- [imgbox] fix ImgboxExtractor (#2281)
- [inkbunny] rename search parameters to their API equivalents
- [kemonoparty] handle files without names (#2276)
- [twitter] fix extraction (#2275, #2295)
- [vk] fix infinite pagination loops (#2297)
- [downloader:ytdl] make
ImportErrors non-fatal (#2273)
1.20.4 - 2022-02-06
Additions
- [e621] add
favoriteextractor (#2250) - [hitomi] add
formatoption (#2260) - [kohlchan] add Kohlchan extractors (#2251)
- [sexcom] add
pinsextractor (#2265) - [twitter] add
warningsoption (#2258) - add ability to disable TLS 1.2 (#2243)
- add examples for custom gelbooru instances (#2262)
Fixes
- [bunkr] fix mp4 downloads (#2239)
- [gelbooru] improve and fix pagination (#2230, #2232)
- [hitomi] "fix" 403 errors (#2260)
- [kemonoparty] fix downloading smaller text files (#2267)
- [patreon] disable TLS 1.2 by default (#2249)
- [twitter] restore errors for protected timelines etc (#2237)
- [twitter] restore
logoutfunctionality (#1719) - [twitter] provide fallback URLs for card images
- [weibo] update pagination code (#2244)
1.20.3 - 2022-01-26
Fixes
- [kemonoparty] fix DMs extraction (#2008)
- [twitter] fix crash on Tweets with deleted quotes (#2225)
- [twitter] fix crash on suspended Tweets without
legacyentry (#2216) - [twitter] fix crash on unified cards without
type - [twitter] prevent crash on invalid/deleted Retweets (#2225)
- [twitter] update query hashes
1.20.2 - 2022-01-24
Additions
- [twitter] add
eventextractor (closes #2109) - [twitter] support image_carousel_website unified cards
- add
--source-addresscommand-line option (#2206) - add environment variable syntax to formatting.md (#2065)
Changes
- [twitter] changes to
cardsoption- enable
cardsby default - require
cardsto be set to"ytdl"to invoke youtube-dl/yt-dlp on unsupported cards
- enable
Fixes
- [blogger] support new image domain (#2204)
- [gelbooru] improve video file detection (#2188)
- [hitomi] fix
tagextraction (#2189) - [instagram] fix highlights extraction (#2197)
- [mangadex] re-enable warning for external chapters (#2193)
- [newgrounds] set suitabilities filter before starting a search (#2173)
- [philomena] fix search parameter escaping (#2215)
- [reddit] allow downloading from quarantined subreddits (#2180)
- [sexcom] extend URL pattern (#2220)
- [twitter] update to GraphQL API (#2212)
1.20.1 - 2022-01-08
Additions
- [newgrounds] add
searchextractor (#2161)
Changes
- restore
-d/--destfunctionality from before 1.20.0 (#2148) - change short option for
--directoryto-D
Fixes
- [gelbooru] handle changed API response format (#2157)
- [hitomi] fix image URLs (#2153)
- [mangadex] fix extraction (#2177)
- [rule34] use
https://api.rule34.xxxfor API requests - fix cookie checks for patreon, fanbox, fantia
- improve UNC path handling (#2126)
1.20.0 - 2021-12-29
Additions
- [500px] add
favoriteextractor (#1927) - [exhentai] add
sourceoption - [fanbox] support pixiv redirects (#2122)
- [inkbunny] add
searchextractor (#2094) - [kemonoparty] support coomer.party (#2100)
- [lolisafe] add generic album extractor for lolisafe/chibisafe instances (#2038, #2105)
- [rule34us] add
tagandpostextractors (#1527) - add a generic extractor (#735, #683)
- add
-d/--directoryand-f/--filenamecommand-line options - add
--sleep-requestand--sleep-extractorcommand-line options - allow specifying
sleep-*options as string
Changes
- [cyberdrop] include file ID in default filenames
- [hitomi] disable
metadataby default - [kemonoparty] use
serviceas subcategory (#2147) - [kemonoparty] change default
filesorder toattachments,file,inline(#1991) - [output] write download progress indicator to stderr
- [ytdl] prefer yt-dlp over youtube-dl (#1850, #2028)
- rename
--write-infojsonto--write-info-json
Fixes
- [500px] create directories per photo
- [artstation] create directories per asset (#2136)
- [deviantart] use
/browse/newestfor most-recent searches (#2096) - [hitomi] fix image URLs
- [instagram] fix error when PostPage data is not in GraphQL format (#2037)
- [instagran] match post URLs with usernames (#2085)
- [instagram] allow downloading specific stories (#2088)
- [furaffinity] warn when no session cookies were found
- [pixiv] respect date ranges in search URLs (#2133)
- [sexcom] fix and improve embed extraction (#2145)
- [tumblrgallery] fix extraction (#2112)
- [tumblrgallery] improve
idextraction (#2115) - [tumblrgallery] improve search pagination (#2132)
- [twitter] include
4096x4096as a default image fallback (#1881, #2107) - [ytdl] update argument parsing to latest yt-dlp changes (#2124)
- handle UNC paths (#2113)
1.19.3 - 2021-11-27
Additions
- [dynastyscans] add
mangaextractor (#2035) - [instagram] include user metadata for
taggeddownloads (#2024) - [kemonoparty] implement
filesoption (#1991) - [kemonoparty] add
dmsoption (#2008) - [mangadex] always provide
artist,author, andgroupmetadata fields (#2049) - [philomena] support furbooru.org (#1995)
- [reactor] support thatpervert.com (#2029)
- [shopify] support loungeunderwear.com (#2053)
- [skeb] add
thumbnailsoption (#2047, #2051) - [subscribestar] add
numenumeration index (#2040) - [subscribestar] emit metadata for posts without media (#1569)
- [ytdl] implement
cmdline-argsandconfig-fileoptions to allow parsing ytdl command-line options (#1680) - [formatter] implement
Dformat specifier - extend
blacklist/whitelistsyntax (#2025)
Fixes
- [dynastyscans] provide
dateas datetime object (#2050) - [exhentai] fix extraction for disowned galleries (#2055)
- [gelbooru] apply workaround for pagination limits
- [kemonoparty] skip duplicate files (#2032, #1991, #1899)
- [kemonoparty] provide
datemetadata for gumroad (#2007) - [mangoxo] fix metadata extraction
- [twitter] distinguish between fatal & nonfatal errors (#2020)
- [twitter] fix extractor for direct image links (#2030)
- [webtoons] use download URLs that do not require a
Refererheader (#2005) - [ytdl] improve error handling (#1680)
- [downloader:ytdl] prevent crash in
_progress_hook()(#1680)
Removals
- [seisoparty] remove module
1.19.2 - 2021-11-05
Additions
- [kemonoparty] add
commentsoption (#1980) - [skeb] add
userandpostextractors (#1031, #1971) - [twitter] add
pinnedoption - support accessing environment variables and the current local datetime in format strings (#1968)
- add special type format strings to docs (#1987)
Fixes
- [cyberdrop] fix video extraction (#1993)
- [deviantart] fix
indexvalues for stashed deviations - [gfycat] provide consistent
userNamevalues foruserdownloads (#1962) - [gfycat] show warning when there are no available formats
- [hitomi] fix image URLs (#1975, #1982, #1988)
- [instagram] update query hashes
- [mangakakalot] update domain and fix extraction
- [mangoxo] fix login and extraction
- [reddit] prevent crash for galleries with no
media_metadata(#2001) - [redgifs] update to API v2 (#1984)
- fix calculating retry sleep times (#1990)
1.19.1 - 2021-10-24
Additions
- [inkbunny] add
followingextractor (#515) - [inkbunny] add
poolextractor (#1937) - [kemonoparty] add
discordextractor (#1827, #1940) - [nhentai] add
tagextractor (#1950, #1955) - [patreon] add
filesoption (#1935) - [picarto] add
galleryextractor (#1931) - [pixiv] add
sketchextractor (#1497) - [seisoparty] add
favoriteextractor (#1906) - [twitter] add
sizeoption (#1881) - [vk] add
albumextractor (#474, #1952) - [postprocessor:compare] add
equaloption (#1592)
Fixes
- [cyberdrop] extract direct download URLs (#1943)
- [deviantart] update
searchargument handling (#1911) - [deviantart] full resolution for non-downloadable images (#293)
- [furaffinity] unquote search queries (#1958)
- [inkbunny] match "long" URLs for pools and favorites (#1937)
- [kemonoparty] improve inline extraction (#1899)
- [mangadex] update parameter handling for API requests (#1908)
- [patreon] better filenames for
contentimages (#1954) - [redgifs][gfycat] provide fallback URLs (#1962)
- [downloader:ytdl] prevent crash in
_progress_hook() - restore SOCKS support for Windows executables
1.19.0 - 2021-10-01
Additions
- [aryion] add
tagextractor (#1849) - [desktopography] implement desktopography extractors (#1740)
- [deviantart] implement
auto-unwatchoption (#1466, #1757) - [fantia] add
datemetadata field (#1853) - [fappic] add
imageextractor (#1898) - [gelbooru_v02] add
favoriteextractor (#1834) - [kemonoparty] add
favoriteextractor (#1824) - [kemonoparty] implement login with username & password (#1824)
- [mastodon] add
followingextractor (#1891) - [mastodon] support specifying accounts by ID
- [twitter] support
/with_repliesURLs (#1833) - [twitter] add
quote_bymetadata field (#1481) - [postprocessor:compare] extend
actionoption (#1592) - implement a download progress indicator (#1519)
- implement a
page-reverseoption (#1854) - implement a way to specify extended format strings
- allow specifying a minimum/maximum for
sleep-*options (#1835) - add a
--write-infojsoncommand-line option
Changes
- [cyberdrop] change directory name format (#1871)
- [instagram] update default delay to 6-12 seconds (#1835)
- [reddit] extend subcategory depending on input URL (#1836)
- move util.Formatter and util.PathFormat into their own modules
Fixes
- [artstation] use
/album/allview for user portfolios (#1826) - [aryion] update/improve pagination (#1849)
- [deviantart] fix bug with fetching premium content (#1879)
- [deviantart] update default archive_fmt for single deviations (#1874)
- [erome] send Referer header for file downloads (#1829)
- [hiperdex] fix extraction
- [kemonoparty] update file download URLs (#1902, #1903)
- [mangadex] fix extraction (#1852)
- [mangadex] fix retrieving chapters from "pornographic" titles (#1908)
- [nozomi] preserve case of search tags (#1860)
- [redgifs][gfycat] remove webtoken code (#1907)
- [twitter] ensure card entries have a
url(#1868) - implement a way to correctly shorten displayed filenames containing east-asian characters (#1377)
1.18.4 - 2021-09-04
Additions
- [420chan] add
threadandboardextractors (#1773) - [deviantart] add
tagextractor (#1803) - [deviantart] add
commentsoption (#1800) - [deviantart] implement a
auto-watchoption (#1466, #1757) - [foolfuuka] add
galleryextractor (#1785) - [furaffinity] expand URL pattern for searches (#1780)
- [kemonoparty] automatically generate required DDoS-GUARD cookies (#1779)
- [nhentai] add
favoriteextractor (#1814) - [shopify] support windsorstore.com (#1793)
- [twitter] add
urlto user objects (#1787, #1532) - [twitter] expand t.co links in user descriptions (#1787, #1532)
- show a warning if an extractor doesn`t yield any results (#1428, #1759)
- add a
jformat string conversion - implement a
fallbackoption (#1770) - implement a
path-stripoption
Changes
- [shopify] use API for product listings (#1793)
- update default User-Agent headers
Fixes
- [deviantart] prevent exceptions for "empty" videos (#1796)
- [exhentai] improve image limits check (#1808)
- [inkbunny] fix extraction (#1816)
- [mangadex] prevent exceptions for manga without English title (#1815)
- [oauth] use defaults when config values are set to
null(#1778) - [pixiv] fix pixivision title extraction
- [reddit] delay RedditAPI initialization (#1813)
- [twitter] improve error reporting (#1759)
- [twitter] fix issue when filtering quote tweets (#1792)
- [twitter] fix
logoutoption (#1719)
Removals
- [deviantart] remove the "you need session cookies to download mature scraps" warning (#1777, #1776)
- [foolslide] remove entry for kobato.hologfx.com
1.18.3 - 2021-08-13
Additions
- [bbc] add
widthoption (#1706) - [danbooru] add
externaloption (#1747) - [furaffinity] add
externaloption (#1492) - [luscious] add
gifoption (#1701) - [newgrounds] add
formatoption (#1729) - [reactor] add
gifoption (#1701) - [twitter] warn about suspended accounts (#1759)
- [twitter] extend
repliesoption (#1254) - [twitter] add option to log out and retry when blocked (#1719)
- [wikieat] add
threadandboardextractors (#1699, #1607)
Changes
- [instagram] increase default delay between HTTP requests from 5s to 8s (#1732)
Fixes
- [bbc] improve image dimensions (#1706)
- [bbc] support multi-page gallery listings (#1730)
- [behance] fix
collectionextraction - [deviantart] get original files for GIF previews (#1731)
- [furaffinity] fix errors when using
category-transfer(#1274) - [hitomi] fix image URLs (#1765)
- [instagram] use custom User-Agent header for video downloads (#1682, #1623, #1580)
- [kemonoparty] fix username extraction (#1750)
- [kemonoparty] update file server domain (#1764)
- [newgrounds] fix errors when using
category-transfer(#1274) - [nsfwalbum] retry backend requests when extracting image URLs (#1733, #1271)
- [vk] prevent exception for empty/private profiles (#1742)
1.18.2 - 2021-07-23
Additions
- [bbc] add
galleryandprogrammeextractors (#1706) - [comicvine] add extractor (#1712)
- [kemonoparty] add
max-postsoption (#1674) - [kemonoparty] parse
oquery parameters (#1674) - [mastodon] add
reblogsandrepliesoptions (#1669) - [pixiv] add extractor for
pixivisionarticles (#1672) - [ytdl] add experimental extractor for sites supported by youtube-dl (#1680, #878)
- extend
parent-metadatafunctionality (#1687, #1651, #1364) - add
archive-prefixoption (#1711) - add
url-metadataoption (#1659, #1073)
Changes
- [kemonoparty] skip duplicated patreon files (#1689, #1667)
- [mangadex] use custom User-Agent header (#1535)
Fixes
- [hitomi] fix image URLs (#1679)
- [imagevenue] fix extraction (#1677)
- [instagram] fix extraction of
/explore/tags/posts (#1666) - [moebooru] fix
tagsending with a+when logged in (#1702) - [naverwebtoon] fix comic extraction
- [pururin] update domain and fix extraction
- [vk] improve metadata extraction and URL pattern (#1691)
- [downloader:ytdl] fix
outtmplsetting for yt-dlp (#1680)
1.18.1 - 2021-07-04
Additions
- [mangafox] add
mangaextractor (#1633) - [mangasee] add
chapterandmangaextractors - [mastodon] implement
text-postsoption (#1569, #1669) - [seisoparty] add
userandpostextractors (#1635) - implement conditional directories (#1394)
- add
Tformat string conversion (#1646) - document format string syntax
Changes
- [twitter] set
retweet_idfor original retweets (#1481)
Fixes
- [directlink] manually encode Referer URLs (#1647)
- [hiperdex] use domain from input URL
- [kemonoparty] fix
usernameextraction (#1652) - [kemonoparty] warn about missing DDoS-GUARD cookies
- [twitter] ensure guest tokens are returned as string (#1665)
- [webtoons] match arbitrary language codes (#1643)
- fix depth counter in UrlJob when specifying
-gmultiple times
1.18.0 - 2021-06-19
Additions
- [foolfuuka] support
archive.wakarimasen.moe(#1595) - [mangadex] implement login with username & password (#1535)
- [mangadex] add extractor for a user's followed feed (#1535)
- [pixiv] support fetching privately followed users (#1628)
- implement conditional filenames (#1394)
- implement
filteroption for post processors (#1460) - add
-T/--terminatecommand-line option (#1399) - add
-P/--postprocessorcommand-line option (#1583)
Changes
- [kemonoparty] update default filenames and archive IDs (#1514)
- [twitter] update default settings
- change
retweetsandquotedoptions fromtruetofalse - change directory format for search results to the same as other extractors
- change
- require an argument for
--clear-cache
Fixes
- [500px] update GraphQL queries
- [furaffinity] improve metadata extraction (#1630)
- [hitomi] update image URL generation (#1637)
- [idolcomplex] improve and fix pagination (#1594, #1601)
- [instagram] fix login (#1631)
- [instagram] update query hashes
- [mangadex] update to API v5 (#1535)
- [mangafox] improve URL pattern (#1608)
- [oauth] prevent exceptions when reporting errors (#1603)
- [philomena] fix tag escapes handling (#1629)
- [redgifs] update API server address (#1632)
- [sankaku] handle empty tags (#1617)
- [subscribestar] improve attachment filenames (#1609)
- [unsplash] update collections URL pattern (#1627)
- [postprocessor:metadata] handle dicts in
mode:tags(#1598)
1.17.5 - 2021-05-30
Additions
- [kemonoparty] add
metadataoption (#1548) - [kemonoparty] add
typemetadata field (#1556) - [mangapark] recognize v2.mangapark URLs (#1578)
- [patreon] extract user-defined
tags(#1539, #1540) - [pillowfort] implement login with username & password (#846)
- [pillowfort] add
inlineandexternaloptions (#846) - [pixiv] implement
max-postsoption (#1558) - [pixiv] add
metadataoption (#1551) - [twitter] add
text-tweetsoption (#570) - [weibo] extend
retweetsoption (#1542) - [postprocessor:ugoira] support using the
image2demuxer (#1550) - [postprocessor:ugoira] add
repeat-last-frameoption (#1550) - support
XDG_CONFIG_HOME(#1545) - implement
parent-skipand"skip": "terminate"options (#1399)
Changes
- [twitter] resolve
t.coURLs incontent(#1532)
Fixes
- [500px] update query hashes (#1573)
- [aryion] find text posts in
recursive=falsemode (#1568) - [imagebam] fix extraction of NSFW images (#1534)
- [imgur] update URL patterns (#1561)
- [manganelo] update domain to
manganato.com - [reactor] skip deleted/empty posts
- [twitter] add missing retweet media entities (#1555)
- fix ISO 639-1 code for Japanese (
jp->ja)
1.17.4 - 2021-05-07
Additions
- [gelbooru] add extractor for
/redirect.phpURLs (#1530) - [inkbunny] add
favoriteextractor (#1521) - add
output.skipoption - add an optional argument to
--clear-cacheto select which cache entries to remove (#1230)
Changes
- [pixiv] update
translated-tagsoption (#1507)- rename to
tags - accept
"japanese","translated", and"original"as values
- rename to
Fixes
- [500px] update query hashes
- [kemonoparty] fix download URLs (#1514)
- [imagebam] fix extraction
- [instagram] update query hashes
- [nozomi] update default archive-fmt for
tagandsearchextractors (#1529) - [pixiv] remove duplicate translated tags (#1507)
- [readcomiconline] change domain to
readcomiconline.li(#1517) - [sankaku] update invalid-token detection (#1515)
- fix crash when using
--no-downloadwith--ugoira-conv(#1507)
1.17.3 - 2021-04-25
Additions
- [danbooru] add option for extended metadata extraction (#1458)
- [fanbox] add extractors (#1459)
- [fantia] add extractors (#1459)
- [gelbooru] add an option to extract notes (#1457)
- [hentaicosplays] add extractor (#907, #1473, #1483)
- [instagram] add extractor for
taggedposts (#1439) - [naverwebtoon] ignore non-comic images
- [pixiv] also save untranslated tags when
translated-tagsis enabled (#1501) - [shopify] support omgmiamiswimwear.com (#1280)
- implement
output.fallbackoption - add archive format to InfoJob output (#875)
- build executables with SOCKS proxy support (#1424)
Fixes
- [500px] update query hashes
- [8muses] fix JSON deobfuscation
- [artstation] download
/4k/images (#1422) - [deviantart] fix pagination for Eclipse results (#1444)
- [deviantart] improve folder name matching (#1451)
- [erome] skip deleted albums (#1447)
- [exhentai] fix image limit detection (#1437)
- [exhentai] restore
limitsoption (#1487) - [gelbooru] fix tag category extraction (#1455)
- [instagram] update query hashes
- [komikcast] fix extraction
- [simplyhentai] fix extraction
- [slideshare] fix extraction
- [webtoons] update agegate/GDPR cookies (#1431)
- fix
category-transferoption
Removals
- [yuki] remove module for yuki.la
1.17.2 - 2021-04-02
Additions
- [deviantart] add support for posts from watched users (#794)
- [manganelo] add
chapterandmangaextractors (#1415) - [pinterest] add
searchextractor (#1411) - [sankaku] add
tag_stringmetadata field (#1388) - [sankaku] add enumeration index for books (#1388)
- [tapas] add
seriesandepisodeextractors (#692) - [tapas] implement login with username & password (#692)
- [twitter] allow specifying a custom format for user results (#1337)
- [twitter] add extractor for direct image links (#1417)
- [vk] add support for albums (#474)
Fixes
- [aryion] unescape paths (#1414)
- [bcy] improve pagination
- [deviantart] update
watchURL pattern (#794) - [deviantart] fix arguments for search/popular results (#1408)
- [deviantart] use fallback for
/intermediary/URLs - [exhentai] improve and simplify image limit checks
- [komikcast] fix extraction
- [pixiv] fix
favoriteURL pattern (#1405) - [sankaku] simplify
pooltags (#1388) - [twitter] improve error message when trying to log in with 2FA (#1409)
- [twitter] don't use youtube-dl for cards when videos are disabled (#1416)
1.17.1 - 2021-03-19
Additions
- [architizer] add
projectandfirmextractors (#1369) - [deviantart] add
watchextractor (#794) - [exhentai] support
/tag/URLs (#1363) - [gelbooru_v01] support
drawfriends.booru.org,vidyart.booru.org, andtlb.booru.orgby default - [nozomi] support
/index-N.htmlURLs (#1365) - [philomena] add generalized extractors for philomena sites (#1379)
- [philomena] support post URLs without
/images/ - [twitter] implement
usersoption (#1337) - implement
parent-metadataoption (#1364)
Changes
Fixes
- [exhentai] improve favorites count extraction (#1360)
- [gelbooru] update domain for video downloads (#1368)
- [hentaifox] improve image and metadata extraction (#1366, #1378)
- [imgur] fix and improve rate limit handling (#1386)
- [weasyl] improve favorites URL pattern (#1374)
- use type check before applying
browseroption (#1358) - ensure
-s/--simulatealways prints filenames (#1360)
Removals
- [hentaicafe] remove module
- [hentainexus] remove module
- [mangareader] remove module
- [mangastream] remove module
1.17.0 - 2021-03-05
Additions
- [cyberdrop] add support for
https://cyberdrop.me/(#1328) - [exhentai] add
metadataoption; extract more metadata from gallery pages (#1325) - [hentaicafe] add
searchandtagextractors (#1345) - [hentainexus] add
originaloption (#1322) - [instagram] support
/user/reels/URLs (#1329) - [naverwebtoon] add support for
https://comic.naver.com/(#1331) - [pixiv] add
translated-tagsoption (#1354) - [tbib] add support for
https://tbib.org/(#473, #1082) - [tumblrgallery] add support for
https://tumblrgallery.xyz/(#1298) - [twitter] add extractor for followed users (#1337)
- [twitter] add option to download all media from conversations (#1319)
- [wallhaven] add
collectionsextractor (#1351) - [snap] allow access to user's .netrc for site authentication (#1352)
- add extractors for Gelbooru v0.1 sites (#234, #426, #473, #767, #1238)
- add
-E/--extractor-infocommand-line option (#875) - add GitHub Actions workflow for building standalone executables (#1312)
- add
browserandheadersoptions (#1117) - add option to use different youtube-dl forks (#1330)
- support using multiple input files at once (#1353)
Changes
- [deviantart] extend
extraoption to also download embedded DeviantArt posts. - [exhentai] rename metadata fields to match API results (#1325)
- [mangadex] use
api.mangadex.orgas default API server - [mastodon] cache OAuth tokens (#616)
- replace
wait-minandwait-maxwithsleep-request
Fixes
- [500px] skip unavailable photos (#1335)
- [komikcast] fix extraction
- [readcomiconline] download high quality image versions (#1347)
- [twitter] update GraphQL endpoints
- fix crash when
base-directoryis an empty string (#1339)
Removals
- remove support for formerly deprecated options
- remove
cloudflaremodule
1.16.5 - 2021-02-14
Additions
- [behance] support
videomodules (#1282) - [erome] add
album,user, andsearchextractors (#409) - [hentaifox] support searching by group (#1294)
- [imgclick] add
imageextractor (#1307) - [kemonoparty] extract inline images (#1286)
- [kemonoparty] support URLs with non-numeric user and post IDs (#1303)
- [pillowfort] add
userandpostextractors (#846)
Changes
- [kemonoparty] include
servicein directories and archive keys - [pixiv] require a
refresh-tokento login (#1304) - [snap] use
core18as base
Fixes
- [500px] update query hashes
- [deviantart] update parameters for
/browse/popular(#1267) - [deviantart] provide filename extension for original file downloads (#1272)
- [deviantart] fix
foldersoption (#1302) - [inkbunny] add
sidparameter to private file downloads (#1281) - [kemonoparty] fix absolute file URLs
- [mangadex] revert to
https://mangadex.org/api/and addapi-serveroption (#1310) - [nsfwalbum] use fallback for deleted content (#1259)
- [sankaku] update
invalid tokendetection (#1309) - [slideshare] fix extraction
- [postprocessor:metadata] fix crash with
extension-format(#1285)
1.16.4 - 2021-01-23
Additions
- [furaffinity] add
descriptionsoption (#1231) - [kemonoparty] add
userandpostextractors (#1216) - [nozomi] add
numenumeration index (#1239) - [photovogue] added portfolio extractor (#1253)
- [twitter] match
/i/user/IDURLs - [unsplash] add extractors (#1197)
- [vipr] add image extractor (#1258)
Changes
- [derpibooru] use "Everything" filter by default (#862)
Fixes
- [derpibooru] update
dateparsing - [foolfuuka] stop search when results are exhausted (#1174)
- [instagram] fix regex for
/savedURLs (#1251) - [mangadex] update API URLs
- [mangakakalot] fix extraction
- [newgrounds] fix flash file extraction (#1257)
- [sankaku] simplify login process
- [twitter] fix retries after hitting rate limit
1.16.3 - 2021-01-10
Fixes
- fix crash when using a
dictforpath-restrict - [postprocessor:metadata] sanitize custom filenames
1.16.2 - 2021-01-09
Additions
- [derpibooru] add
searchandgalleryextractors (#862) - [foolfuuka] add
boardandsearchextractors (#1044, #1174) - [gfycat] add
datemetadata field (#1138) - [pinterest] add support for getting all boards of a user (#1205)
- [sankaku] add support for book searches (#1204)
- [twitter] fetch media from pinned tweets (#1203)
- [wikiart] add extractor for single paintings (#1233)
- [downloader:http] add MIME type and signature for
.icofiles (#1211) - add
dformat string conversion for timestamp values - add
"ascii"as a specialpath-restrictvalue
Fixes
- [hentainexus] fix extraction (#1234)
- [instagram] categorize single highlight URLs as
highlights(#1222) - [redgifs] fix search results
- [twitter] fix login with username & password
- [twitter] fetch tweets from
homeConversationentries
1.16.1 - 2020-12-27
Additions
- [instagram] add
includeoption (#1180) - [pinterest] implement video support (#1189)
- [sankaku] reimplement login support (#1176, #1182)
- [sankaku] add support for sankaku.app URLs (#1193)
Changes
- [e621] return pool posts in order (#1195)
- [hentaicafe] prefer title of
/hc.fyi/pages (#1106) - [hentaicafe] simplify default filenames
- [sankaku] normalize
created_atmetadata (#1190) - [postprocessor:exec] do not add missing
{}to command (#1185)
Fixes
- [booru] improve error handling
- [instagram] warn about private profiles (#1187)
- [keenspot] improve redirect handling
- [mangadex] respect
chapter-reversesettings (#1194) - [pixiv] output debug message on failed login attempts (#1192)
- increase SQLite connection timeouts (#1173)
Removals
- [mangapanda] remove module
1.16.0 - 2020-12-12
Additions
- [booru] implement generalized extractors for
*booruandmoeboorusites - [postprocessor:metadata] add
eventandfilenameoptions (#315, #866, #984) - [postprocessor:exec] add
eventoption (#992)
Changes
- [flickr] update default directories and improve metadata consistency (#828)
- [sankaku] use API endpoints from
beta.sankakucomplex.com - [downloader:http] improve filename extension handling (#776)
- replace all JPEG filename extensions with
jpgby default
Fixes
- [hentainexus] fix extraction (#1166)
- [instagram] rewrite (#1113, #1122, #1128, #1130, #1149)
- [mangadex] handle external chapters (#1154)
- [nozomi] handle empty
datefields (#1163) - [paheal] create directory for each post (#1147)
- [piczel] update API URLs
- [twitter] update image URL format (#1145)
- [twitter] improve
x-csrf-tokenheader handling (#1170) - [webtoons] update
ageGatecookies
Removals
- [sankaku] remove login support
1.15.4 - 2020-11-27
Fixes
- [2chan] skip external links
- [hentainexus] fix extraction (#1125)
- [mangadex] switch to API v2 (#1129)
- [mangapanda] use http://
- [mangoxo] fix extraction
- [reddit] skip invalid gallery items (#1127)
1.15.3 - 2020-11-13
Additions
- [sankakucomplex] extract videos and embeds (#308)
- [twitter] add support for lists (#1096)
- [postprocessor:metadata] accept string-lists for
content-format(#1080) - implement
modulesandextension-mapoptions
Fixes
- [500px] update query hashes
- [8kun] fix file URLs of older posts (#1101)
- [exhentai] update image URL parsing (#1094)
- [hentaifoundry] update
YII_CSRF_TOKENcookie handling (#1083) - [hentaifoundry] use scheme from input URLs (#1095)
- [mangoxo] fix metadata extraction
- [paheal] fix extraction (#1088)
- collect post processors from
basecategoryentries (#1084)
1.15.2 - 2020-10-24
Additions
- [pinterest] implement login support (#1055)
- [reddit] add
datemetadata field (#1068) - [seiga] add metadata for single image downloads (#1063)
- [twitter] support media from Cards (#937, #1005)
- [weasyl] support api-key authentication (#1057)
- add a
tformat string conversion for trimming whitespace (#1065)
Fixes
- [blogger] handle URLs with specified width/height (#1061)
- [fallenangels] fix extraction of
.5chapters - [gelbooru] rewrite mp4 video URLs (#1048)
- [hitomi] fix image URLs and gallery URL pattern
- [mangadex] unescape more metadata fields (#1066)
- [mangahere] ensure download URLs have a scheme (#1070)
- [mangakakalot] ignore "Go Home" buttons in chapter pages
- [newgrounds] handle embeds without scheme (#1033)
- [newgrounds] provide fallback URLs for video downloads (#1042)
- [xhamster] fix user profile extraction
1.15.1 - 2020-10-11
Additions
- [hentaicafe] add
manga_idmetadata field (#1036) - [hentaifoundry] add support for stories (#734)
- [hentaifoundry] add
includeoption - [newgrounds] extract image embeds (#1033)
- [nijie] add
includeoption (#1018) - [reactor] match URLs without subdomain (#1053)
- [twitter] extend
retweetsoption (#1026) - [weasyl] add extractors (#977)
Fixes
- [500px] update query hashes
- [behance] fix
collectionextraction - [newgrounds] fix video extraction (#1042)
- [twitter] improve twitpic extraction (#1019)
- [weibo] handle posts with more than 9 images (#926)
- [xvideos] fix
titleextraction - fix crash when using
--download-archivewith--no-skip(#1023) - fix issues with
blacklist/whitelistdefaults (#1051, #1056)
Removals
- [kissmanga] remove module
1.15.0 - 2020-09-20
Additions
- [deviantart] support watchers-only/paid deviations (#995)
- [myhentaigallery] add gallery extractor (#1001)
- [twitter] support specifying users by ID (#980)
- [twitter] support
/intent/user?user_id=…URLs (#980) - add
--no-skipcommand-line option (#986) - add
blacklistandwhitelistoptions (#492, #844) - add
filesize-minandfilesize-maxoptions (#780) - add
sleep-extractorandsleep-requestoptions (#788) - write skipped files to archive (#550)
Changes
- [exhentai] update wait time before original image downloads (#978)
- [imgur] use new API endpoints for image/album data
- [tumblr] create directories for each post (#965)
- support format string replacement fields in download archive paths (#985)
- reduce wait time growth rate for HTTP retries from exponential to linear
Fixes
- [500px] update query hash
- [aryion] improve post ID extraction (#981, #982)
- [danbooru] handle posts without
id(#1004) - [furaffinity] update download URL extraction (#988)
- [imgur] fix image/album detection for galleries
- [postprocessor:zip] defer zip file creation (#968)
Removals
- [jaiminisbox] remove extractors
- [worldthree] remove extractors
1.14.5 - 2020-08-30
Additions
- [aryion] add username/password support (#960)
- [exhentai] add ability to specify a custom image limit (#940)
- [furaffinity] add
searchextractor (#915) - [imgur] add
searchandtagextractors (#934)
Fixes
- [500px] fix extraction and update URL patterns (#956)
- [aryion] update folder mime type list (#945)
- [gelbooru] fix extraction without API
- [hentaihand] update to new site layout
- [hitomi] fix redirect processing
- [reddit] handle deleted galleries (#953)
- [reddit] improve gallery extraction (#955)
1.14.4 - 2020-08-15
Additions
- [blogger] add
searchextractor (#925) - [blogger] support searching posts by labels (#925)
- [inkbunny] add
userandpostextractors (#283) - [instagram] support
/reel/URLs - [pinterest] support
pinterest.co.ukURLs (#914) - [reddit] support gallery posts (#920)
- [subscribestar] extract attached media files (#852)
Fixes
- [blogger] improve error messages for missing posts/blogs (#903)
- [exhentai] adjust image limit costs (#940)
- [gfycat] skip malformed gfycat responses (#902)
- [imgur] handle 403 overcapacity responses (#910)
- [instagram] wait before GraphQL requests (#901)
- [mangareader] fix extraction
- [mangoxo] fix login
- [pixnet] detect password-protected albums (#177)
- [simplyhentai] fix
gallery_idextraction - [subscribestar] update
dateparsing - [vsco] handle missing
descriptionfields - [xhamster] fix extraction (#917)
- allow
parent-directoryto work recursively (#905) - skip external OAuth tests (#908)
Removals
- [bobx] remove module
1.14.3 - 2020-07-18
Additions
- [8muses] support
comics.8muses.comURLs - [artstation] add
followingextractor (#888) - [exhentai] add
domainoption (#897) - [gfycat] add
userandsearchextractors - [imgur] support all
/t/...URLs (#880) - [khinsider] add
formatoption (#840) - [mangakakalot] add
mangaandchapterextractors (#876) - [redgifs] support
gifsdeliverynetwork.comURLs (#874) - [subscribestar] add
userandpostextractors (#852) - [twitter] add support for nitter.net URLs (#890)
- add Zsh completion script (#150)
Fixes
- [gfycat] retry 404'ed videos on redgifs.com (#874)
- [newgrounds] fix favorites extraction
- [patreon] yield images and attachments before post files (#871)
- [reddit] fix AttributeError when using
recursion(#879) - [twitter] raise proper exception if a user doesn't exist (#891)
- defer directory creation (#722)
- set pseudo extension for Metadata messages (#865)
- prevent exception on Cloudflare challenges (#868)
1.14.2 - 2020-06-27
Additions
- [artstation] add
datemetadata field (#839) - [mastodon] add
datemetadata field (#839) - [pinterest] add support for board sections (#835)
- [twitter] add extractor for liked tweets (#837)
- [twitter] add option to filter media from quoted tweets (#854)
- [weibo] add
datemetadata field tostatusobjects (#829)
Fixes
- [aryion] fix user gallery extraction (#832)
- [imgur] build directory paths for each file (#842)
- [tumblr] prevent errors when using
reblogs=same-blog(#851) - [twitter] always provide an
authormetadata field (#831, #833) - [twitter] don't download video previews (#833)
- [twitter] improve handling of deleted tweets (#838)
- [twitter] fix search results (#847)
- [twitter] improve handling of quoted tweets (#854)
- fix config lookups when multiple locations are involved (#843)
- improve output of
-K/--list-keywordsfor parent extractors (#825) - call
flush()after writing JSON inDataJob()(#727)
1.14.1 - 2020-06-12
Additions
Changes
- [deviantart] extend
extraoption; also search journals for sta.sh links (#712) - [twitter] rewrite; use new interface (#806, #740)
Fixes
- [kissmanga] work around CAPTCHAs (#818)
- [nhentai] fix extraction (#819)
- [webtoons] generalize comic extraction code (#820)
1.14.0 - 2020-05-31
Additions
- [imagechest] add new extractor for imgchest.com (#750)
- [instagram] add
post_url,tags,location,tagged_usersmetadata (#743) - [redgifs] add image extractor (#724)
- [webtoons] add new extractor for webtoons.com (#761)
- implement
--write-pagesoption (#736) - extend
path-restrictoption (#662) - implement
path-replaceoption (#662, #755) - make
pathandkeywordsavailable in logging messages (#574, #575)
Changes
- [danbooru] change default value of
ugoiratofalse - [downloader:ytdl] change default value of
forward-cookiestofalse - [downloader:ytdl] fix file extensions when merging into
.mkv(#720) - write OAuth tokens to cache (#616)
- use
%APPDATA%\gallery-dlfor config files and cache on Windows - use
util.Formatterfor formatting logging messages - reuse HTTP connections from parent extractors
Fixes
- [deviantart] use private access tokens for Journals (#738)
- [gelbooru] simplify and fix pool extraction
- [imgur] fix extraction of animated images without
mp4entry - [imgur] treat
/t/unmuted/URLs as galleries - [instagram] fix login with username & password (#756, #771, #797, #803)
- [reddit] don't send OAuth headers for file downloads (#729)
- fix/improve Cloudflare bypass code (#728, #757)
- reset filenames on empty file extensions (#733)
1.13.6 - 2020-05-02
Additions
- [patreon] respect filters and sort order in query parameters (#711)
- [speakerdeck] add a new extractor for speakerdeck.com (#726)
- [twitter] add
repliesoption (#705) - [weibo] add
videosoption - [downloader:http] add MIME types for
.psdfiles (#714)
Fixes
- [artstation] improve embed extraction (#720)
- [deviantart] limit API wait times (#721)
- [newgrounds] fix URLs produced by the
followingextractor (#684) - [patreon] improve file hash extraction (#713)
- [vsco] fix user gallery extraction
- fix/improve Cloudflare bypass code (#728)
1.13.5 - 2020-04-27
Additions
- [500px] recognize
web.500px.comURLs - [aryion] support downloading from folders (#694)
- [furaffinity] add extractor for followed users (#515)
- [hitomi] add extractor for tag searches (#697)
- [instagram] add
post_idandnummetadata fields (#698) - [newgrounds] add extractor for followed users (#684)
- [patreon] recognize URLs with creator IDs (#711)
- [twitter] add
replymetadata field (#705) - [xhamster] recognize
xhamster.porncache.netURLs (#700)
Fixes
- [gelbooru] improve post ID extraction in pool listings
- [hitomi] fix extraction of galleries without tags
- [jaiminisbox] update metadata decoding procedure (#702)
- [mastodon] fix pagination (#701)
- [mastodon] improve account searches (#704)
- [patreon] fix hash extraction from download URLs (#693)
- improve parameter extraction when solving Cloudflare challenges
1.13.4 - 2020-04-12
Additions
- [aryion] add
galleryandpostextractors (#390, #673) - [deviantart] detect and handle folders in sta.sh listings (#659)
- [hentainexus] add
circle,event, andtitle_conventionalmetadata fields (#661) - [hiperdex] add
artistextractor (#606) - [mastodon] add access tokens for
mastodon.socialandbaraag.net(#665)
Changes
- [deviantart] retrieve all download URLs through the OAuth API
- automatically read config files in PyInstaller executable directories (#682)
Fixes
- [deviantart] handle "Request blocked" errors (#655)
- [deviantart] improve JPEG quality replacement pattern
- [hiperdex] fix extraction
- [mastodon] handle API rate limits (#665)
- [mastodon] update OAuth credentials for pawoo.net (#665)
- [myportfolio] fix extraction of galleries without title
- [piczel] fix extraction of single images
- [vsco] fix collection extraction
- [weibo] accept status URLs with non-numeric IDs (#664)
1.13.3 - 2020-03-28
Additions
- [instagram] Add support for user's saved medias (#644)
- [nozomi] support multiple images per post (#646)
- [35photo] add
tagextractor
Changes
- [mangadex] transform timestamps from
datefields to datetime objects
Fixes
- [deviantart] handle decode errors for
extended_fetchresults (#655) - [e621] fix bug in API rate limiting and improve pagination (#651)
- [instagram] update pattern for user profile URLs
- [mangapark] fix metadata extraction
- [nozomi] sort search results (#646)
- [piczel] fix extraction
- [twitter] fix typo in
x-twitter-auth-typeheader (#625) - remove trailing dots from Windows directory names (#647)
- fix crash with missing
stdout/stderr/stdinhandles (#653)
1.13.2 - 2020-03-14
Additions
- [furaffinity] extract more metadata
- [instagram] add
post_shortcodemetadata field (#525) - [kabeuchi] add extractor (#561)
- [newgrounds] add extractor for favorited posts (#394)
- [pixiv] implement
avataroption (#595, #623) - [twitter] add extractor for bookmarked Tweets (#625)
Fixes
- [bcy] reduce number of HTTP requests during data extraction
- [e621] update to new interface (#635)
- [exhentai] handle incomplete MIME types (#632)
- [hitomi] improve metadata extraction
- [mangoxo] fix login
- [newgrounds] improve error handling when extracting post data
1.13.1 - 2020-03-01
Additions
- [hentaihand] add extractors (#605)
- [hiperdex] add chapter and manga extractors (#606)
- [oauth] implement option to write DeviantArt refresh-tokens to cache (#616)
- [downloader:http] add more MIME types for
.bmpand.rarfiles (#621, #628) - warn about expired cookies
Fixes
- [bcy] fix partial image URLs (#613)
- [danbooru] fix Ugoira downloads and metadata
- [deviantart] check availability of
/intermediary/URLs (#609) - [hitomi] follow multiple redirects & fix image URLs
- [piczel] improve and update
- [tumblr] replace
-within tag searches (#611) - [vsco] update gallery URL pattern
- fix
--verboseand--quietcommand-line options
1.13.0 - 2020-02-16
Additions
- Support for
furaffinity- https://www.furaffinity.net/ (#284)8kun- https://8kun.top/ (#582)bcy- https://bcy.net/ (#592)
- [blogger] implement video extraction (#587)
- [oauth] add option to specify port number used by local server (#604)
- [pixiv] add
ratingmetadata field (#595) - [pixiv] recognize tags at the end of new bookmark URLs
- [reddit] add
videosoption - [weibo] use youtube-dl to download from m3u8 manifests
- implement
parent-directoryoption (#551) - extend filename formatting capabilities:
Changes
- [reddit] handle reddit-hosted images and videos natively (#551)
- [twitter] change default value for
videostotrue
Fixes
- [cloudflare] unescape challenge URLs
- [deviantart] fix video extraction from
extended_fetchresults - [hitomi] implement workaround for "broken" redirects
- [khinsider] fix and improve metadata extraction
- [patreon] filter duplicate files per post (#590)
- [piczel] fix extraction
- [pixiv] fix user IDs for bookmarks API calls (#596)
- [sexcom] fix image URLs
- [twitter] force old login page layout (#584, #598)
- [vsco] skip "invalid" entities
- improve functions to load/save cookies.txt files (#586)
Removals
- [yaplog] remove module
1.12.3 - 2020-01-19
Additions
- [hentaifoundry] extract more metadata (#565)
- [twitter] add option to extract TwitPic embeds (#579)
- implement a post-processor module to compare file versions (#530)
Fixes
- [hitomi] update image URL generation
- [mangadex] revert domain to
mangadex.org - [pinterest] improve detection of invalid pin.it links
- [pixiv] update URL patterns for user profiles and bookmarks (#568)
- [twitter] Fix stop before real end (#573)
- remove temp files before downloading from fallback URLs
Removals
- [erolord] remove extractor
1.12.2 - 2020-01-05
Additions
- [deviantart] match new search/popular URLs (#538)
- [deviantart] match
/favourites/allURLs (#555) - [deviantart] add extractor for followed users (#515)
- [pixiv] support listing followed users (#515)
- [imagefap] handle beta.imagefap.com URLs (#552)
- [postprocessor:metadata] add
directoryoption (#520)
Fixes
- [artstation] fix search result pagination (#537)
- [directlink] send Referer headers (#536)
- [exhentai] restrict default directory name length (#545)
- [mangadex] change domain to mangadex.cc (#559)
- [mangahere] send
isAdultcookies (#556) - [newgrounds] fix tags metadata extraction
- [pixiv] retry after rate limit errors (#535)
- [twitter] handle quoted tweets (#526)
- [twitter] handle API rate limits (#526)
- [twitter] fix URLs forwarded to youtube-dl (#540)
- prevent infinite recursion when spawning new extractors (#489)
- improve output of
--list-keywordsfor "parent" extractors (#548) - provide fallback for SQLite versions with missing
WITHOUT ROWIDsupport (#553)
1.12.1 - 2019-12-22
Additions
- [4chan] add extractor for entire boards (#510)
- [realbooru] add extractors for pools, posts, and tag searches (#514)
- [instagram] implement a
videosoption (#521) - [vsco] implement a
videosoption - [postprocessor:metadata] implement a
bypostoption for downloading the metadata of an entire post (#511)
Changes
- [reddit] change the default value for
commentsto0 - [vsco] improve image resolutions
- make filesystem-related errors during file downloads non-fatal (#512)
Fixes
- [foolslide] add fallback for chapter data extraction
- [instagram] ignore errors during post-page extraction
- [patreon] avoid errors when fetching user info (#508)
- [patreon] improve URL pattern for single posts
- [reddit] fix errors with
t1submissions - [vsco] fix user profile extraction … again
- [weibo] handle unavailable/deleted statuses
- [downloader:http] improve rate limit handling
- retain trailing zeroes in Cloudflare challenge answers
1.12.0 - 2019-12-08
Additions
- [flickr] support 3k, 4k, 5k, and 6k photo sizes (#472)
- [imgur] add extractor for subreddit links (#500)
- [newgrounds] add extractors for
audiolistings and generalmediafiles (#394) - [newgrounds] implement login support (#394)
- [postprocessor:metadata] implement a
extension-formatoption (#477) --exec-after
Changes
- [deviantart] ensure consistent username capitalization (#455)
- [directlink] split
{path}into{path}/{filename}.{extension} - [twitter] update metadata fields with user/author information
- [postprocessor:metadata] filter private entries & rename
formattocontent-format - Enable
cookies-updateby default
Fixes
- [2chan] fix metadata extraction
- [behance] get images from 'media_collection' modules
- [bobx] fix image downloads by randomly generating session cookies (#482)
- [deviantart] revert to getting download URLs from OAuth API calls (#488)
- [deviantart] fix URL generation from '/extended_fetch' results (#505)
- [flickr] adjust OAuth redirect URI (#503)
- [hentaifox] fix extraction
- [imagefap] adapt to new image URL format
- [imgbb] fix error in galleries without user info (#471)
- [instagram] prevent errors with missing 'video_url' fields (#479)
- [nijie] fix
dateparsing - [pixiv] match new search URLs (#507)
- [plurk] fix comment pagination
- [sexcom] send specific Referer headers when downloading videos
- [twitter] fix infinite loops (#499)
- [vsco] fix user profile and collection extraction (#480)
- Fix Cloudflare DDoS protection bypass
Removals
--abort-on-skip
1.11.1 - 2019-11-09
Fixes
- Fix inclusion of bash completion and man pages in source distributions
1.11.0 - 2019-11-08
Additions
- Support for
blogger- https://www.blogger.com/ (#364)nozomi- https://nozomi.la/ (#388)issuu- https://issuu.com/ (#413)naver- https://blog.naver.com/ (#447)
- Extractor for
twittersearch results (#448) - Extractor for
deviantartuser profiles with configurable targets (#377, #419) --ugoira-conv-lossless(#432)cookies-updateoption to allow updating cookies.txt files (#445)- Optional
cloudflareandvideoinstallation targets (#460) - Allow executing commands with the
execpost-processor after all files are downloaded (#413, #421)
Changes
- Rewrite
imgurusing its public API (#446) - Rewrite
luscioususing GraphQL queries (#457) - Adjust default
nijiefilenames to matchpixiv - Change enumeration index for gallery extractors from
pagetonum - Return non-zero exit status when errors occurred
- Forward proxy settings to youtube-dl downloader
- Install bash completion script into
share/bash-completion/completions
Fixes
- Adapt to new
instagrampage layout when logged in (#391) - Support protected
twittervideos (#452) - Extend
hitomiURL pattern and fix gallery extraction - Restore OAuth2 authentication error messages
- Miscellaneous fixes for
patreon(#444),deviantart(#455),sexcom(#464),imgur(#467),simplyhentai
1.10.6 - 2019-10-11
Additions
--execcommand-line option to specify a command to run after each file download (#421)
Changes
- Include titles in
gfycatdefault filenames (#434)
Fixes
- Fetch working download URLs for
deviantart(#436) - Various fixes and improvements for
yaplogblogs (#443) - Fix image URL generation for
hitomigalleries - Miscellaneous fixes for
behanceandxvideos
1.10.5 - 2019-09-28
Additions
instagram.highlightsoption to include highlighted stories when downloading user profiles (#329)- Support for
/user/URLs onreddit(#350) - Support for
imguruser profiles and favorites (#420) - Additional metadata fields on
nijie(#423)
Fixes
- Improve handling of private
deviantartartworks (#414) and 429 status codes (#424) - Prevent fatal errors when trying to open download-archive files (#417)
- Detect and ignore unavailable videos on
weibo(#427) - Update the
scopeof newredditrefresh-tokens (#428) - Fix inconsistencies with the
reddit.commentsoption (#429) - Extend URL patterns for
hentaicafemanga andpixivartworks - Improve detection of unavailable albums on
lusciousandimgbb - Miscellaneous fixes for
tsumino
1.10.4 - 2019-09-08
Additions
- Support for
lineblog- https://www.lineblog.me/ (#404)fuskator- https://fuskator.com/ (#407)
ugoiraoption fordanbooruto download pre-rendered ugoira animations (#406)
Fixes
- Download the correct files from
twitterreplies (#403) - Prevent crash when trying to use unavailable downloader modules (#405)
- Fix
pixivauthentication (#411) - Improve
exhentaiimage limit checks - Miscellaneous fixes for
hentaicafe,simplyhentai,tumblr
1.10.3 - 2019-08-30
Additions
- Provide
filenamemetadata for alldeviantartfiles (#392, #400) - Implement a
ytdl.outtmploption to let youtube-dl handle filenames by itself (#395) - Support
seigamobile URLs (#401)
Fixes
- Extract more than the first 32 posts from
piczelgalleries (#396) - Fix filenames of archives created with
--zip(#397) - Skip unavailable images and videos on
flickr(#398) - Fix filesystem paths on Windows with Python 3.6 and lower (#402)
1.10.2 - 2019-08-23
Additions
- Support for
instagramstories and IGTV (#371, #373) - Support for individual
imgbbimages (#363) deviantart.qualityoption to set the JPEG compression quality for newer images (#369)enumerateoption forextractor.skip(#306)adjust-extensionsoption to control filename extension adjustmentspath-removeoption to remove control characters etc. from filesystem paths
Changes
- Rename
restrict-filenamestopath-restrict - Adjust
pixivmetadata and default filename format (#366)- Set
filenameto"{category}_{user[id]}_{id}{suffix}.{extension}"to restore the old default
- Set
- Improve and optimize directory and filename generation
Fixes
- Allow the
classifypost-processor to handle files with unknown filename extension (#138) - Fix rate limit handling for OAuth APIs (#368)
- Fix artwork and scraps extraction on
deviantart(#376, #392) - Distinguish between
imguralbum and gallery URLs (#380) - Prevent crash when using
--ugoira-conv(#382) - Handle multi-image posts on
patreon(#383) - Miscellaneous fixes for
*reactor,simplyhentai
1.10.1 - 2019-08-02
Fixes
- Use the correct domain for exhentai.org input URLs
1.10.0 - 2019-08-01
Warning
- Prior to version 1.10.0 all cache files were created world readable (mode
644) leading to possible sensitive information disclosure on multi-user systems - It is recommended to restrict access permissions of already existing files
(
/tmp/.gallery-dl.cache) withchmod 600 - Windows users should not be affected
Additions
- Support for
vsco- https://vsco.co/ (#331)imgbb- https://imgbb.com/ (#361)adultempire- https://www.adultempire.com/ (#340)
restrict-filenamesoption to create Windows-compatible filenames on any platform (#348)forward-cookiesoption to control cookie forwarding to youtube-dl (#352)
Changes
- The default cache file location on non-Windows systems is now
$XDG_CACHE_HOME/gallery-dl/cache.sqlite3or~/.cache/gallery-dl/cache.sqlite3
- New cache files are created with mode
600 exhentaiextractors will always usee-hentai.orgas domain
Fixes
- Better handling of
exhentaiimage limits and errors (#356, #360) - Try to prevent ZIP file corruption (#355)
- Miscellaneous fixes for
behance,ngomik
1.9.0 - 2019-07-19
Additions
- Support for
erolord- http://erolord.com/ (#326)
- Add login support for
instagram(#195) - Add
--no-downloadandextractor.*.downloaddisable file downloads (#220) - Add
-A/--abortto specify the number of consecutive download skips before aborting - Interpret
-1as infinite retries (#300) - Implement custom log message formats per log-level (#304)
- Implement an
mtimepost-processor that sets file modification times according to metadata fields (#332) - Implement a
twitter.contentoption to enable tweet text extraction (#333, #338) - Enable
date-min/-max/-formatoptions fortumblr(#337)
Changes
- Set file modification times according to their
Last-Modifiedheader when downloading (#236, #277)- Use
--no-mtimeordownloader.*.mtimeto disable this behavior
- Use
- Duplicate download URLs are no longer silently ignored (controllable with
extractor.*.image-unique) - Deprecate
--abort-on-skip
Fixes
- Retry downloads on OpenSSL exceptions (#324)
- Ignore unavailable pins on
sexcominstead of raising an exception (#325) - Use Firefox's SSL/TLS ciphers to prevent Cloudflare CAPTCHAs (#342)
- Improve folder name matching on
deviantart(#343) - Forward cookies to
youtube-dlto allow downloading private videos - Miscellaneous fixes for
35photo,500px,newgrounds,simplyhentai
1.8.7 - 2019-06-28
Additions
- Support for
vanillarock- https://vanilla-rock.com/ (#254)nsfwalbum- https://nsfwalbum.com/ (#287)
artistandtagsmetadata forhentaicafe(#238)descriptionmetadata forinstagram(#310)- Format string option to replace a substring with another -
R<old>/<new>/(#318)
Changes
- Delete empty archives created by the
zippost-processor (#316)
Fixes
- Handle
hitomiGame CG galleries correctly (#321) - Miscellaneous fixes for
deviantart,hitomi,pururin,kissmanga,keenspot,mangoxo,imagefap
1.8.6 - 2019-06-14
Additions
- Support for
slickpic- https://www.slickpic.com/ (#249)xhamster- https://xhamster.com/ (#281)pornhub- https://www.pornhub.com/ (#282)8muses- https://www.8muses.com/ (#305)
extraoption fordeviantartto download Sta.sh content linked in description texts (#302)
Changes
- Detect
directlinkURLs with upper case filename extensions (#296)
Fixes
- Improved error handling for
tumblrAPI calls (#297) - Fixed extraction of
livedoorblogs (#301) - Fixed extraction of special
deviantartSta.sh items (#307) - Fixed pagination for specific
keenspotcomics
1.8.5 - 2019-06-01
Additions
- Support for
keenspot- http://keenspot.com/ (#223)sankakucomplex- https://www.sankakucomplex.com (#258)
foldersoption fordeviantartto add a list of containing folders to each file (#276)captchaoption forkissmangaandreadcomiconlineto control CAPTCHA handling (#279)filenamemetadata for files downloaded with youtube-dl (#291)
Changes
- Adjust
wallhavenextractors to new page layout:- use API and add
api-keyoption - removed traditional login support
- use API and add
- Provide original filenames for
patreondownloads (#268) - Use e-hentai.org or exhentai.org depending on input URL (#278)
Fixes
- Fix pagination over
sankakupopular listings (#265) - Fix folder and collection extraction on
deviantart(#271) - Detect "AreYouHuman" redirects on
readcomiconline(#279) - Miscellaneous fixes for
hentainexus,livedoor,ngomik
1.8.4 - 2019-05-17
Additions
- Support for
patreon- https://www.patreon.com/ (#226)hentainexus- https://hentainexus.com/ (#256)
datemetadata fields forpixiv(#248),instagram(#250),exhentai, andnewgrounds
Changes
- Improved
flickrmetadata and video extraction (#246)
Fixes
- Download original GIF animations from
deviantart(#242) - Ignore missing
edge_media_to_commentfields oninstagram(#250) - Fix serialization of
datetimeobjects for--write-metadata(#251, #252) - Allow multiple post-processor command-line options at once (#253)
- Prevent crash on
boorusites when no tags are available (#259) - Fix extraction on
instagramafterrhx_gisfield removal (#266) - Avoid Cloudflare CAPTCHAs for Python interpreters built against OpenSSL < 1.1.1
- Miscellaneous fixes for
luscious
1.8.3 - 2019-05-04
Additions
- Support for
plurk- https://www.plurk.com/ (#212)sexcom- https://www.sex.com/ (#147)
--clear-cachedatemetadata fields fordeviantart,twitter, andtumblr(#224, #232)
Changes
- Standalone executables are now built using PyInstaller:
- uses the latest CPython interpreter (Python 3.7.3)
- available on several platforms (Windows, Linux, macOS)
- includes the
certifiCA bundle,youtube-dl, andpyOpenSSLon Windows
Fixes
- Patch
urllib3's default list of SSL/TLS ciphers to prevent Cloudflare CAPTCHAs (#227) (Windows users need to installpyOpenSSLfor this to take effect) - Provide fallback URLs for
twitterimages (#237) - Send
Refererheaders when downloading fromhitomi(#239) - Updated login procedure on
mangoxo
1.8.2 - 2019-04-12
Additions
- Support for
pixnet- https://www.pixnet.net/ (#177)wikiart- https://www.wikiart.org/ (#179)mangoxo- https://www.mangoxo.com/ (#184)yaplog- https://yaplog.jp/ (#190)livedoor- http://blog.livedoor.jp/ (#190)
- Login support for
mangoxo(#184) andtwitter(#214)
Changes
- Increased required
Requestsversion to 2.11.0
Fixes
- Improved image quality on
reactorsites (#210) - Support
imagebamgalleries with more than 100 images (#219) - Updated Cloudflare bypass code
1.8.1 - 2019-03-29
Additions
- Support for:
35photo- https://35photo.pro/ (#162)500px- https://500px.com/ (#185)
instagramextractor for hashtags (#202)- Option to get more metadata on
deviantart(#189) - Man pages and bash completion (#150)
- Snap improvements (#197, #199, #207)
Changes
- Better FFmpeg arguments for
--ugoira-conv - Adjusted metadata for
lusciousalbums
Fixes
- Proper handling of
instagrammulti-image posts (#178, #201) - Fixed
tumblravatar URLs when not using OAuth1.0 (#193) - Miscellaneous fixes for
exhentai,komikcast
1.8.0 - 2019-03-15
Additions
- Support for:
weibo- https://www.weibo.com/pururin- https://pururin.io/ (#174)fashionnova- https://www.fashionnova.com/ (#175)shopifysites in general (#175)
- Snap packaging (#169, #170, #187, #188)
- Automatic Cloudflare DDoS protection bypass
- Extractor and Job information for logging format strings
dynastyscansimage and search extractors (#163)deviantartscraps extractor (#168)artstationextractor for artwork listings (#172)smugmugvideo support and improved image format selection (#183)
Changes
- More metadata for
nhentaigalleries - Combined
myportfolioextractors into one - Renamed
namemetadata field tofilenameand removed the originalfilenamefield - Simplified and improved internal data structures
- Optimized creation of child extractors
Fixes
- Filter empty
tumblrURLs (#165) - Filter ads and improve connection speed on
hentaifoundry - Show proper error messages if
lusciousgalleries are unavailable - Miscellaneous fixes for
mangahere,ngomik,simplyhentai,imgspice
Removals
seaotterscans
1.7.0 - 2019-02-05
- Added support for:
photobucket- http://photobucket.com/ (#117)hentaifox- https://hentaifox.com/ (#160)tsumino- https://www.tsumino.com/ (#161)
- Added the ability to dynamically generate extractors based on a user's config file for
- Added an extractor for
behancecollections (#157) - Added login support for
luscious(#159) andtsumino(#161) - Added an option to stop downloading if the
exhentaiimage limit is exceeded (#141) - Fixed extraction issues for
behanceandmangapark
1.6.3 - 2019-01-18
- Added
metadatapost-processor to write image metadata to an external file (#135) - Added option to reverse chapter order of manga extractors (#149)
- Added authentication support for
danbooru(#151) - Added tag metadata for
exhentaiandhbrowsegalleries - Improved
*reactorextractors (#148) - Fixed extraction issues for
nhentai(#156),pinterest,mangapark
1.6.2 - 2019-01-01
- Added support for:
instagram- https://www.instagram.com/ (#134)
- Added support for multiple items on sta.sh pages (#113)
- Added option to download
tumblravatars (#137) - Changed defaults for visited post types and inline media on
tumblr - Improved inline extraction of
tumblrposts (#133, #137) - Improved error handling and retry behavior of all API calls
- Improved handling of missing fields in format strings (#136)
- Fixed hash extraction for unusual
tumblrURLs (#129) - Fixed image subdomains for
hitomigalleries (#142) - Fixed and improved miscellaneous issues for
kissmanga(#20),luscious,mangapark,readcomiconline
1.6.1 - 2018-11-28
- Added support for:
joyreactor- http://joyreactor.cc/ (#114)pornreactor- http://pornreactor.cc/ (#114)newgrounds- https://www.newgrounds.com/ (#119)
- Added extractor for search results on
luscious(#127) - Fixed filenames of ZIP archives (#126)
- Fixed extraction issues for
gfycat,hentaifoundry(#125),mangafox
1.6.0 - 2018-11-17
- Added support for:
wallhaven- https://alpha.wallhaven.cc/yuki- https://yuki.la/
- Added youtube-dl integration and video downloads for
twitter(#99),behance,artstation - Added per-extractor options for network connections (
retries,timeout,verify) - Added a
--no-check-certificatecommand-line option - Added ability to specify the number of skipped downloads before aborting/exiting (#115)
- Added extractors for scraps, favorites, popular and recent images on
hentaifoundry(#110) - Improved login procedure for
pixivto avoid unwanted emails on each new login - Improved album metadata and error handling for
flickr(#109) - Updated default User-Agent string to Firefox 62 (#122)
- Fixed
twitterAPI response handling when logged in (#123) - Fixed issue when converting Ugoira using H.264
- Fixed miscellaneous issues for
2chan,deviantart,fallenangels,flickr,imagefap,pinterest,turboimagehost,warosu,yuki(#112)
1.5.3 - 2018-09-14
- Added support for:
hentaicafe- https://hentai.cafe/ (#101)bobx- http://www.bobx.com/dark/
- Added black-/whitelist options for post-processor modules
- Added support for
tumblrinline videos (#102) - Fixed extraction of
smugmugalbums without owner (#100) - Fixed issues when using default config values with
redditextractors (#104) - Fixed pagination for user favorites on
sankaku(#106) - Fixed a crash when processing
deviantartjournals (#108)
1.5.2 - 2018-08-31
- Added support for
twittertimelines (#96) - Added option to suppress FFmpeg output during ugoira conversions
- Improved filename formatter performance
- Improved inline image quality on
tumblr(#98) - Fixed image URLs for newly released
mangadexchapters - Fixed a smaller issue with
deviantartjournals - Replaced
subapicswithngomik
1.5.1 - 2018-08-17
- Added support for:
piczel- https://piczel.tv/
- Added support for related pins on
pinterest - Fixed accessing "offensive" galleries on
exhentai(#97) - Fixed extraction issues for
mangadex,komikcastandbehance - Removed original-image functionality from
tumblr, since "raw" images are no longer accessible
1.5.0 - 2018-08-03
- Added support for:
behance- https://www.behance.net/myportfolio- https://www.myportfolio.com/ (#95)
- Added custom format string options to handle long strings (#92, #94)
- Slicing:
"{field[10:40]}" - Replacement:
"{field:L40/too long/}"
- Slicing:
- Improved frame rate handling for ugoira conversions
- Improved private access token usage on
deviantart - Fixed metadata extraction for some images on
nijie - Fixed chapter extraction on
mangahere - Removed
whatisthisimnotgoodwithcomputers - Removed support for Python 3.3
1.4.2 - 2018-07-06
- Added image-pool extractors for
safebooruandrule34 - Added option for extended tag information on
boorusites (#92) - Added support for DeviantArt's new URL format
- Added support for
mangaparkmirrors - Changed
imagefapextractors to use HTTPS - Fixed crash when skipping downloads for files without known extension
1.4.1 - 2018-06-22
- Added an
ugoirapost-processor to convertpixivanimations to WebM - Added
--zipand--ugoira-convcommand-line options - Changed how ugoira frame information is handled
- instead of being written to a separate file, it is now made available as metadata field of the ZIP archive
- Fixed manga and chapter titles for
mangadex - Fixed file deletion by post-processors
1.4.0 - 2018-06-08
- Added support for:
simplyhentai- https://www.simply-hentai.com/ (#89)
- Added extractors for
pixivsearch results and followed usersdeviantartsearch results and popular listings
- Added post-processors to perform actions on downloaded files
- Added options to configure logging behavior
- Added OAuth support for
smugmug - Changed
pixivextractors to use the AppAPI- this breaks
favoritearchive IDs and changes some metadata fields
- this breaks
- Changed the default filename format for
tumblrand renamedoffsettonum - Fixed a possible UnicodeDecodeError during installation (#86)
- Fixed extraction of
mangadexmanga with more than 100 chapters (#84) - Fixed miscellaneous issues for
imgur,reddit,komikcast,mangafoxandimagebam
1.3.5 - 2018-05-04
- Added support for:
smugmug- https://www.smugmug.com/
- Added title information for
mangadexchapters - Improved the
pinterestAPI implementation (#83) - Improved error handling for
deviantartandtumblr - Removed
gomangaandpuremashiro
1.3.4 - 2018-04-20
- Added support for custom OAuth2 credentials for
pinterest - Improved rate limit handling for
tumblrextractors - Improved
hentaifoundryextractors - Improved
imgurURL patterns - Fixed miscellaneous extraction issues for
lusciousandkomikcast - Removed
loveisoverandspectrumnexus
1.3.3 - 2018-04-06
- Added extractors for
nhentaisearch resultsexhentaisearch results and favoritesnijiedoujins and favorites
- Improved metadata extraction for
exhentaiandnijie - Improved
tumblrextractors by avoiding unnecessary API calls - Fixed Cloudflare DDoS protection bypass
- Fixed errors when trying to print unencodable characters
1.3.2 - 2018-03-23
- Added extractors for
artstationalbums, challenges and search results - Improved URL and metadata extraction for
hitomiandnhentai - Fixed page transitions for
danbooruAPI results (#82)
1.3.1 - 2018-03-16
- Added support for:
mangadex- https://mangadex.org/artstation- https://www.artstation.com/
- Added Cloudflare DDoS protection bypass to
komikcastextractors - Changed archive ID formats for
deviantartfolders and collections - Improved error handling for
deviantartAPI calls - Removed
imgchiliand various smaller image hosts
1.3.0 - 2018-03-02
- Added
--proxyto explicitly specify a proxy server (#76) - Added options to customize archive ID formats and undefined replacement fields
- Changed various archive ID formats to improve their behavior for favorites / bookmarks / etc.
- Affected modules are
deviantart,flickr,tumblr,pixivand all …boorus
- Affected modules are
- Improved
sankakuandidolcomplexsupport by- respecting
pageandnextURL parameters (#79) - bypassing the page-limit for unauthenticated users
- respecting
- Improved
directlinkmetadata by properly unquoting it - Fixed
pixivugoira extraction (#78) - Fixed miscellaneous extraction issues for
mangastreamandtumblr - Removed
yeet,chronos,coreimg,hosturimage,imageontime,img4ever,imgmaid,imgupload
1.2.0 - 2018-02-16
- Added support for:
paheal- https://rule34.paheal.net/ (#69)komikcast- https://komikcast.com/ (#70)subapics- http://subapics.com/ (#70)
- Added
--download-archiveto record downloaded files in an archive file - Added
--write-logto write logging output to a file - Added a filetype check on download completion to fix incorrectly assigned filename extensions (#63)
- Added the
tumblr:...pseudo URI scheme to support custom domains for Tumblr blogs (#71) - Added fallback URLs for
tumblrimages (#64) - Added support for
reddit-hosted images (#68) - Improved the input file format by allowing comments and per-URL options
- Fixed OAuth 1.0 signature generation for Python 3.3 and 3.4 (#75)
- Fixed smaller issues for
luscious,hentai2read,hentaihereandimgur - Removed the
batotomodule
1.1.2 - 2018-01-12
- Added support for:
puremashiro- http://reader.puremashiro.moe/ (#66)idolcomplex- https://idol.sankakucomplex.com/
- Added an option to filter reblogs on
tumblr(#61) - Added OAuth user authentication for
tumblr(#65) - Added support for
slidesharemobile URLs (#67) - Improved pagination for various …booru sites to work around page limits
- Fixed chapter information parsing for certain manga on
kissmanga(#58) andbatoto(#60)
1.1.1 - 2017-12-22
- Added support for:
slideshare- https://www.slideshare.net/ (#54)
- Added pool- and post-extractors for
sankaku - Added OAuth user authentication for
deviantart - Updated
lusciousto supportmembers.luscious.netURLs (#55) - Updated
mangahereto use their new domain name (mangahere.cc) and support mobile URLs - Updated
gelbooruto not be restricted to the first 20,000 images (#56) - Fixed extraction issues for
nhentaiandkhinsider
1.1.0 - 2017-12-08
- Added the
-r/--limit-ratecommand-line option to set a maximum download rate - Added the
--sleepcommand-line option to specify the number of seconds to sleep before each download - Updated
gelbooruto no longer use their now disabled API - Fixed SWF extraction for
sankaku(#52) - Fixed extraction issues for
hentai2readandkhinsider - Removed the deprecated
--imagesand--chaptersoptions - Removed the
mangazukimodule
1.0.2 - 2017-11-24
- Added an option to set a custom user-agent string
- Improved retry behavior for failed HTTP requests
- Improved
seigaby providing better metadata and getting more than the latest 200 images - Improved
tumblrby adding support for all post types, scanning for inline images and following external links (#48) - Fixed extraction issues for
hbrowse,khinsiderandsenmanga
1.0.1 - 2017-11-10
- Added support for:
xvideos- https://www.xvideos.com/ (#45)
- Fixed exception handling during file downloads which could lead to a premature exit
- Fixed an issue with
tumblrwhere not all images would be downloaded when using tags (#48) - Fixed extraction issues for
imgbox(#47),mangastream(#49) andmangahere
1.0.0 - 2017-10-27
- Added support for:
warosu- https://warosu.org/b4k- https://arch.b4k.co/
- Added support for
pixivranking lists - Added support for
boorupopular lists (danbooru,e621,konachan,yandere,3dbooru) - Added the
--cookiescommand-line andcookiesconfig option to load additional cookies - Added the
--filterand--chapter-filtercommand-line options to select individual images or manga-chapters by their metadata using simple Python expressions (#43) - Added the
verifyconfig option to control certificate verification during file downloads - Added config options to overwrite internally used API credentials (API Tokens & IDs)
- Added
-Kas a shortcut for--list-keywords - Changed the
--imagesand--chapterscommand-line options to--rangeand--chapter-range - Changed keyword names for various modules to make them accessible by
--filter. In general minus signs have been replaced with underscores (e.g.gallery-id->gallery_id). - Changed default filename formats for manga extractors to optionally use volume and title information
- Improved the downloader modules to use
.partfiles and support resuming incomplete downloads (#29) - Improved
deviantartby distinguishing between users and groups (#26), always using HTTPS, and always downloading full-sized original images - Improved
sankakuby adding authentication support and fixing various other issues (#44) - Improved URL pattern for direct image links (#30)
- Fixed an issue with
lusciousnot getting original image URLs (#33) - Fixed various smaller issues for
batoto,hentai2read(#38),jaiminisbox,khinsider,kissmanga(#28, #46),mangahere,pawoo,twitter - Removed
kisscomicandyonkouprodmodules
0.9.1 - 2017-07-24
- Added support for:
2chan- https://www.2chan.net/4plebs- https://archive.4plebs.org/archivedmoe- https://archived.moe/archiveofsins- https://archiveofsins.com/desuarchive- https://desuarchive.org/fireden- https://boards.fireden.net/loveisover- https://archive.loveisover.me/nyafuu- https://archive.nyafuu.org/rbt- https://rbt.asia/thebarchive- https://thebarchive.com/mangazuki- https://mangazuki.co/
- Improved
redditto allow submission filtering by ID and human-readable dates - Improved
deviantartto support group galleries and gallery folders (#26) - Changed
deviantartto use better default path formats - Fixed extraction of larger
imguralbums - Fixed some smaller issues for
pixiv,batotoandfallenangels
0.9.0 - 2017-06-28
- Added support for:
reddit- https://www.reddit.com/ (#15)flickr- https://www.flickr.com/ (#16)gfycat- https://gfycat.com/
- Added support for direct image links
- Added user authentication via OAuth for
redditandflickr - Added support for user authentication data from
.netrcfiles (#22) - Added a simple progress indicator for multiple URLs (#19)
- Added the
--write-unsupportedcommand-line option to write unsupported URLs to a file - Added documentation for all available config options (configuration.rst)
- Improved
pixivto support tags for user downloads (#17) - Improved
pixivto support shortened and http://pixiv.me/... URLs (#23) - Improved
imgurto properly handle.gifvimages and provide better metadata - Fixed an issue with
kissmangawhere metadata parsing for some series failed (#20) - Fixed an issue with getting filename extensions from
Content-Typeresponse headers
0.8.4 - 2017-05-21
- Added the
--abort-on-skipoption to stop extraction if a download would be skipped - Improved the output format of the
--list-keywordsoption - Updated
deviantartto support all media types and journals - Updated
fallenangelsto support their Vietnamese version - Fixed an issue with multiple tags on ...booru sites
- Removed the
yomangamodule
0.8.3 - 2017-05-01
- Added support for https://pawoo.net/
- Added manga extractors for all FoOlSlide-based modules
- Added the
-q/--quietand-v/--verboseoptions to control output verbosity - Added the
-j/--dump-jsonoption to dump extractor results in JSON format - Added the
--ignore-configoption - Updated the
exhentaiextractor to fall back to using the e-hentai version if no username is given - Updated
deviantartto support sta.sh URLs - Fixed an issue with
kissmangawhich prevented image URLs from being decrypted properly (again) - Fixed an issue with
pixhostwhere for an image inside an album it would always download the first image of that album (#13) - Removed the
mangashareandreadcomicsmodules
0.8.2 - 2017-04-10
- Fixed an issue in
kissmangawhich prevented image URLs from being decrypted properly
0.8.1 - 2017-04-09
- Added new extractors:
kireicake- https://reader.kireicake.com/seaotterscans- https://reader.seaotterscans.com/
- Added a favourites extractor for
deviantart - Re-enabled the
kissmangamodule - Updated
nijieto support multi-page image listings - Updated
mangastreamto support readms.net URLs - Updated
exhentaito support e-hentai.org URLs - Updated
fallenangelsto support their new domain and site layout
0.8.0 - 2017-03-28
- Added logging support
- Added the
-R/--retriesoption to specify how often a download should be retried before giving up - Added the
--http-timeoutoption to set a timeout for HTTP connections - Improved error handling/tolerance during HTTP file downloads (#10)
- Improved option parsing and the help message from
-h/--help - Changed the way configuration values are used by prioritizing top-level values
- This allows for cmdline options like
-u/--usernameto overwrite values set in configuration files
- This allows for cmdline options like
- Fixed an issue with
imagefap.comwhere incorrectly reported gallery sizes would cause the extractor to fail (#9) - Fixed an issue with
seiga.nicovideo.jpwhere invalid characters in an API response caused the XML parser to fail - Fixed an issue with
seiga.nicovideo.jpwhere the filename extension for the first image would be used for all others - Removed support for old configuration paths on Windows
- Removed several modules:
mangamint: site is downwhentai: now requires account with VIP status for original imageskissmanga: encrypted image URLs (will be re-added later)
0.7.0 - 2017-03-06
- Added
--imagesand--chaptersoptions- Specifies which images (or chapters) to download through a comma-separated list of indices or index-ranges
- Example:
--images -2,4,6-8,10-will select images with index 1, 2, 4, 6, 7, 8 and 10 up to the last one
- Changed the
-g/--get-urlsoption- The amount of how often the -g option is given now determines up until which level URLs are resolved.
- See
3bca866185
- Changed several option keys:
directory_fmt->directoryfilename_fmt->filenamedownload-original->original
- Improved FoOlSlide-based extractors
- Fixed URL extraction for hentai2read
- Fixed an issue with deviantart, where the API access token wouldn't get refreshed
0.6.4 - 2017-02-13
- Added new extractors:
- fallenangels (famatg.com)
- Fixed url- and data-extraction for:
- nhentai
- mangamint
- imagetwist
- Disabled InsecureConnectionWarning when no certificates are available
0.6.3 - 2017-01-25
- Added new extractors:
- gomanga
- yomanga
- mangafox
- Fixed deviantart extractor failing - switched to using their API
- Fixed an issue with SQLite on Python 3.6
- Automated test builds via Travis CI
- Standalone executables for Windows
0.6.2 - 2017-01-05
- Added new extractors:
- kisscomic
- readcomics
- yonkouprod
- jaiminisbox
- Added manga extractor to batoto-module
- Added user extractor to seiga-module
- Added
-i/--input-fileargument to allow local files and stdin as input (like wget) - Added basic support for
file://URLs- this allows for the recursive extractor to be applied to local files:
$ gallery-dl r:file://[path to file]
- Added a utility extractor to run unit test URLs
- Updated luscious to deal with API changes
- Fixed twitter to provide the original image URL
- Minor fixes to hentaifoundry
- Removed imgclick extractor
0.6.1 - 2016-11-30
- Added new extractors:
- whentai
- readcomiconline
- sensescans, worldthree
- imgmaid, imagevenue, img4ever, imgspot, imgtrial, pixhost
- Added base class for extractors of FoOlSlide-based sites
- Changed default paths for configuration files on Windows
- old paths are still supported, but that will change in future versions
- Fixed aborting downloads if a single one failed (#5)
- Fixed cloudflare-bypass cache containing outdated cookies
- Fixed image URLs for hitomi and 8chan
- Updated deviantart to always provide the highest quality image
- Updated README.rst
- Removed doujinmode extractor
0.6.0 - 2016-10-08
- Added new extractors:
- hentaihere
- dokireader
- rapidimg, picmaniac
- Added support to find filename extensions by Content-Type response header
- Fixed filename/path issues on Windows (#4):
- Enable path names with more than 260 characters
- Remove trailing spaces in path segments
- Updated Job class to automatically set category/subcategory keywords
0.5.2 - 2016-09-23
- Added new extractors:
- rule34
- dynastyscans
- imagebam, coreimg, imgcandy, imgtrex
- Added login capabilities for batoto
- Added
--versioncmdline argument to print the current program version and exit - Added
--list-extractorscmdline argument to print names of all extractor classes together with descriptions and example URLs - Added proper error messages if an image/user does not exist
- Added unittests for every extractor
0.5.1 - 2016-08-22
- Added new extractors:
- luscious
- doujinmode
- hentaibox
- seiga
- imagefap
- Changed error output to use stderr instead of stdout
- Fixed broken pipes causing an exception-dump by catching BrokenPipeErrors
0.5.0 - 2016-07-25
0.4.1 - 2015-12-03
- New modules (imagetwist, turboimagehost)
- Manga-extractors: Download entire manga and not just single chapters
- Generic extractor (provisional)
- Better and configurable console output
- Windows support