github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
daac2c6e04be7352fee7b9368b119b5de2649d6f
gallery-dl/gallery_dl/extractor
History
CasualYouTuber31 daac2c6e04 [tiktok] add support (#3061 #4177 #5646 #6878 #6708) 
* Add TikTok photo support

#3061
#4177

* Address linting errors

* Fix more test failures

* Forgot to update category names in tests

* Looking into re issue

* Follow default yt-dlp output template

* Fix format string error on 3.5

* Support downloading videos and audio

Respond to comments
Improve archiving and file naming

* Forgot to update supportedsites.md

* Support user profiles

* Fix indentation

* Prevent matching with more than one TikTok extractor

* Fix TikTok regex

* Support TikTok profile avatars

* Fix supportedsites.md

* TikTok: Ignore no formats error

In my limited experience, this doesn't mean that gallery-dl can't download the photo post (but this could mean that you can't download the audio)

* Fix error reporting message

* TikTok: Support more URL formats

vt.tiktok.com
www.tiktok.com/t/

* TikTok: Only download avatar when extracting user profile

* TikTok: Document profile avatar limitation

* TikTok: Add support for www.tiktokv.com/share links

* Address Share -> Sharepost issue

* TikTok: Export post's creation date in JSON (ISO 8601)

* [tiktok] update

* [tiktok] update 'vmpost' handling

just perform a HEAD request and handle its response

* [tiktok] build URLs from post IDs

instead of reusing unchanged input URLs

* [tiktok] combine 'post' and 'sharepost' extractors

* [tiktok] update default filenames

put 'id' and 'num' first to ensure better file order

* [tiktok] improve ytdl usage

- speed up extraction by passing '"extract_flat": True'
- pass more user options and cookies
- pre-define 'TikTokUser' extractor usage

* [tiktok] Add _COOKIES entry to AUTH_MAP

* [tiktok] Always download user avatars

* [tiktok] Add more documentation to supportedsites.md

* [tiktok] Address review comments

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2025-02-25 20:10:48 +01:00
..
__init__.py
[tiktok] add support (#3061 #4177 #5646 #6878 #6708)
2025-02-25 20:10:48 +01:00
2ch.py
[2ch] update
2024-01-15 04:09:05 +01:00
2chan.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
2chen.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
3dbooru.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
4archive.py
[4archive] fix 'TypeError' when processing 'date' metadata
2025-01-26 12:55:01 +01:00
4chan.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
4chanarchives.py
[4chanarchives] disable Referer headers by default (#4686)
2023-10-20 17:39:50 +02:00
8chan.py
[8chan] don't make network requests in '_init()' (#6387)
2024-10-27 19:36:49 +01:00
8muses.py
[8muses] skip albums without valid 'permalink' (#6717)
2024-12-24 13:49:19 +01:00
35photo.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
500px.py
send Referer headers by default
2023-09-19 00:02:04 +02:00
adultempire.py
[adultempire] bypass age confirmation check
2025-01-26 11:24:48 +01:00
agnph.py
[agnph] implement 'tags' option (#5284)
2024-07-30 14:16:19 +02:00
ao3.py
[ao3] add 'subscriptions' extractor (#6247)
2024-09-29 13:01:51 +02:00
architizer.py
[architizer] fix extraction
2025-01-26 17:28:06 +01:00
artstation.py
[artstation] restore 'browser' and 'tls12' defaults
2025-01-28 11:36:06 +01:00
aryion.py
[aryion] update favorite extractor
2024-07-21 12:34:06 +02:00
batoto.py
[batoto] use 'chapter_id' in default archive IDs (#6835)
2025-01-15 14:52:18 +01:00
bbc.py
[common] add '_extract_jsonld' method (#5272)
2025-01-12 11:07:48 +01:00
behance.py
[dl:ytdl] implement explicit HLS/DASH handling
2024-10-16 15:16:21 +02:00
bilibili.py
add bilibili User Articles FavList support
2025-02-23 18:18:45 +01:00
blogger.py
[blogger] use default API key when 'api-key' is empty
2024-11-20 16:02:16 +01:00
bluesky.py
[bluesky] default to /posts if reposts/quoted is enabled (#6583)
2024-12-13 22:24:37 +01:00
booru.py
[booru] allow multiple 'url' keys (#5859)
2024-07-17 20:49:52 +02:00
boosty.py
[boosty] added new direct message extractor
2025-02-23 18:14:59 +01:00
bunkr.py
[bunkr] implement fast '--range' support (#6985)
2025-02-14 18:21:32 +01:00
catbox.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
chevereto.py
[chevereto] remove 'deltaporno.com'
2024-10-10 13:07:08 +02:00
cien.py
[common] add '_extract_jsonld' method (#5272)
2025-01-12 11:07:48 +01:00
civitai.py
[civitai] add 'user-videos' extractor (#6644)
2024-12-26 10:18:54 +01:00
comicvine.py
fix bug in test_init and update example URLs
2023-09-14 13:27:03 +02:00
common.py
[executables] fix SSLError when using HTTPAdapter (#6393)
2025-01-31 20:36:41 +01:00
cyberdrop.py
[cyberdrop] add extractor for media URLs (#2496)
2024-12-08 20:57:12 +01:00
danbooru.py
[danbooru][e621] use lower 'threshold' value by default (#6886)
2025-01-27 20:41:24 +01:00
desktopography.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
deviantart.py
[deviantart] prevent crash when accessing premium_folder_data (#6873)
2025-01-24 19:08:32 +01:00
directlink.py
[directlink] use domain as 'subcategory' (#6703)
2024-12-22 17:19:56 +01:00
discord.py
[discord] add support (#6836)
2025-02-18 18:45:39 +01:00
dynastyscans.py
[dynastyscans] extract chapter 'tags'
2024-07-29 16:50:47 +05:30
e621.py
[e621] support e621.cc and e621.anthro.fr frontend URLs (#6809)
2025-01-15 14:35:37 +01:00
erome.py
[erome] extract 'date' metadata (#5796)
2024-06-27 21:07:23 +02:00
everia.py
[everia] update
2024-11-03 17:51:04 +01:00
exhentai.py
[exhentai] implement 'tags' option (#2117)
2024-11-15 21:47:13 +01:00
facebook.py
[facebook] evaluate 'directory' per photo (#6874)
2025-01-24 21:37:20 +01:00
fanbox.py
[fanbox] add 'comments' option, extend 'metadata' option (#6287)
2024-10-06 22:31:41 +02:00
fantia.py
[fantia] simplify 'tags' to a list of strings (#4752)
2023-11-02 15:25:01 +01:00
fapachi.py
[fapachi] fix extraction (#6881)
2025-01-28 11:49:55 +01:00
fapello.py
fapello.py Fullsize image
2024-03-21 18:08:18 +01:00
flickr.py
[flickr] fix video download URLs (#6464)
2024-11-13 20:56:37 +01:00
foolfuuka.py
[b4k] keep support for previous 'arch.b4k.co' domain
2025-02-09 11:11:38 +01:00
foolslide.py
[foolslide] remove 'powermanga.org'
2023-11-26 23:19:05 +01:00
furaffinity.py
[furaffinity] fix 'favorite' pagination (#6151)
2024-09-06 08:52:05 +02:00
furry34.py
[furry34] add support (#1078 #7018)
2025-02-19 16:35:48 +01:00
fuskator.py
[fuskator] make metadata extraction non-fatal (#5039)
2024-01-08 19:18:12 +01:00
gelbooru_v01.py
fix bug in test_init and update example URLs
2023-09-14 13:27:03 +02:00
gelbooru_v02.py
[realbooru] fix extraction (#6543)
2024-12-07 17:39:25 +01:00
gelbooru.py
[gelbooru] improve pagination logic for meta tags (#5478)
2024-04-15 23:14:48 +02:00
generic.py
[generic] fix config lookups by subcategory
2025-02-22 10:08:59 +01:00
gofile.py
[gofile] fix website token extraction (#6596)
2024-12-03 12:04:05 +01:00
hatenablog.py
[hatenablog] implement additional suggestion
2024-01-13 10:43:25 +11:00
hentai2read.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
hentaicosplays.py
[hentaicosplays] update domains (#6578)
2024-12-03 13:56:32 +01:00
hentaifoundry.py
[hentaifoundry] add 'tag' extractor (#6465)
2024-11-13 20:56:37 +01:00
hentaifox.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
hentaihand.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
hentaihere.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
hentainexus.py
[hentainexus] fix error for spread pages (#5827)
2024-07-05 21:36:29 +02:00
hiperdex.py
[hiperdex] update domain to 'hiperdex.com'
2025-01-26 19:26:03 +01:00
hitomi.py
[hitomi] provide 'search_tags' metadata for search/tag results
2025-01-02 17:49:30 +01:00
hotleak.py
[hotleak] fix AttributeError (#5950)
2024-08-07 08:26:23 +02:00
idolcomplex.py
[idolcomplex] fix metadata extraction
2024-03-22 01:43:05 +01:00
imagebam.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
imagechest.py
[imagechest] simplify
2024-11-16 09:17:13 +01:00
imagefap.py
[common] add '_extract_jsonld' method (#5272)
2025-01-12 11:07:48 +01:00
imagehosts.py
[turboimagehost] add support for galleries
2025-01-19 17:28:45 +01:00
imgbb.py
replace '24*3600' with '86400'
2023-12-18 23:57:22 +01:00
imgbox.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
imgth.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
imgur.py
[imgur] replace AuthorizationError exception with logging message
2025-02-15 15:36:39 +01:00
imhentai.py
[imhentai] improve pagination duplicate filtering
2025-02-20 20:53:35 +01:00
inkbunny.py
[inkbunny] fix re-login loop (#6618)
2024-12-07 17:39:25 +01:00
instagram.py
[instagram] fix 'pinned' values for '/reels' results (#6719)
2024-12-25 19:42:50 +01:00
issuu.py
[issuu] unescape HTML entities
2025-02-02 18:33:18 +01:00
itaku.py
[itaku] support gallery section URLs (#6951)
2025-02-13 14:29:36 +01:00
itchio.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
jschan.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
kabeuchi.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
keenspot.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
kemonoparty.py
[kemonoparty] Support /posts endpoint and Creator Tag Calls
2025-01-21 19:20:22 +01:00
khinsider.py
[khinsider] add 'covers' option (#6844)
2025-01-18 15:57:56 +01:00
koharu.py
[koharu] implement 'tags' option
2024-11-15 23:49:58 +01:00
komikcast.py
[komikcast] update domain to 'komikcast.la'
2025-01-26 16:54:14 +01:00
lensdump.py
[lensdump] fix extraction (#6313)
2024-10-24 13:59:39 +02:00
lexica.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
lightroom.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
livedoor.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
lofter.py
Add error messages in lofter extractor
2025-01-21 21:38:20 +09:00
lolisafe.py
[saint] fix metadata names/IDs of '/d/' URLs
2025-01-27 11:04:41 +01:00
luscious.py
[luscious] fix IndexError for files without thumbnail (#5122)
2024-01-28 01:43:29 +01:00
lynxchan.py
[lynxchan] update 'bbw-chan' domain (#4970)
2023-12-25 15:29:05 +01:00
mangadex.py
[common] allow overriding more default 'User-Agent' headers (#6496)
2024-11-26 21:50:28 +01:00
mangafox.py
[mangafox] fix chapter extraction
2025-01-26 21:11:36 +01:00
mangahere.py
[mangahere] fix chapter extraction
2025-01-26 21:16:24 +01:00
mangakakalot.py
use 'v[0] == "c"' instead of 'v.startswith("c")'
2024-10-15 08:24:06 +02:00
manganelo.py
[manganelo] fix extraction & recognize '.to' TLDs (#5005)
2024-01-01 22:05:21 +01:00
mangapark.py
[common] add '_extract_nextdata' method
2025-01-12 11:48:36 +01:00
mangaread.py
[mangaread] fix manga metadata extraction
2025-01-26 21:28:24 +01:00
mangasee.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
mangoxo.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
mastodon.py
allow passing cookies to OAuth extractors
2024-11-09 18:06:27 +01:00
message.py
remove 'Message.Metadata' (#866)
2021-01-31 02:12:37 +01:00
misskey.py
[misskey] support 'misskey.design' (#4713)
2023-10-25 17:47:03 +02:00
moebooru.py
[moebooru] add 'metadata' option (#4646)
2023-10-12 21:34:25 +02:00
motherless.py
[motherless] add 'media' and 'gallery' extractors
2024-11-22 21:06:32 +01:00
myhentaigallery.py
[myhentaigallery] fix 'tags' extraction
2024-10-10 17:06:17 +02:00
myportfolio.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
naver.py
[naver] unescape post 'title' and 'description'
2024-03-06 00:46:19 +01:00
naverwebtoon.py
[naverwebtoon] support '/webtoon/' paths for all comics (#5123)
2024-02-04 21:38:46 +01:00
nekohouse.py
[nekohouse] fix pagination (#6871)
2025-01-24 10:18:52 +01:00
newgrounds.py
[newgrounds] provide 'comment_html' metadata (#7038)
2025-02-22 10:08:58 +01:00
nhentai.py
[nhentai] select random file servers for download URLs (#6620)
2024-12-07 17:39:25 +01:00
nijie.py
[nijie] fix image URLs for single image posts (#5842)
2024-07-10 15:06:25 +02:00
nitter.py
[nitter] remove instances
2024-06-02 18:15:53 +02:00
noop.py
[noop] add 'noop' extractor
2024-10-28 19:45:24 +01:00
nozomi.py
use 'True if COND else False' for bool conversions
2024-10-14 20:57:47 +02:00
nsfwalbum.py
[nsfwalbum] detect '/error.jpg' images (#4598)
2023-10-02 15:38:22 +02:00
oauth.py
[oauth] prevent empty browser names
2024-10-01 08:22:50 +02:00
paheal.py
[paheal] implement fast '--range' support (#5905)
2024-07-30 12:55:57 +02:00
patreon.py
[patreon] extract 'campaign' metadata (#6989)
2025-02-13 14:13:43 +01:00
pexels.py
[pexels] add support (#2286, #4214, #6769)
2025-01-12 16:50:12 +01:00
philomena.py
[philomena] download 'full' URLs (#6922)
2025-02-02 18:23:46 +01:00
photovogue.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
picarto.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
piczel.py
[piczel] fix extraction (#6735)
2024-12-27 15:08:08 +01:00
pillowfort.py
[pillowfort] provide 'count' metadata field (#6478)
2024-11-15 08:27:52 +01:00
pinterest.py
[pinterest] update API headers (#6513)
2024-11-22 08:41:10 +01:00
pixeldrain.py
[pixeldrain] integrate into 'album' extractor
2024-05-25 23:42:23 +02:00
pixiv.py
[pixiv] prevent exceptions during 'comments' extraction (#6965)
2025-02-10 09:56:09 +01:00
pixnet.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
plurk.py
[plurk] fix 'user' data extraction and make it non-fatal (#6742)
2025-01-06 20:27:37 +01:00
poipiku.py
[poipiku] always query 'ShowAppendFileF' when post has warning (#6736)
2024-12-27 20:32:50 +01:00
poringa.py
[poringa] improvements and fixes
2023-12-30 20:37:09 +01:00
pornhub.py
[pornhub] fix KeyError when album images are missing (#6299)
2024-10-09 17:18:06 +02:00
pornpics.py
[pornpics] fix pagination results from HTML page
2025-01-27 16:12:19 +01:00
postmill.py
use 'v[0] == "c"' instead of 'v.startswith("c")'
2024-10-15 08:24:06 +02:00
reactor.py
replace static 'sleep-request' defaults with dynamic ones
2023-12-18 22:06:26 +01:00
readcomiconline.py
[readcomiconline] fix chapter extractor (#6070, #6335)
2024-12-03 10:54:58 +01:00
realbooru.py
[realbooru] fix extraction (#6543)
2024-12-07 17:39:25 +01:00
recursive.py
[recursive] add 'https://' to URLs if not present
2024-12-10 17:16:52 +01:00
reddit.py
[reddit] restrict subreddit search results (#7025)
2025-02-19 20:05:48 +01:00
redgifs.py
Update redgifs.py to include 0-9 regex
2024-07-28 02:34:17 -04:00
rule34us.py
[rule34us] make 'file_url' extraction more robust (#6289)
2024-10-10 09:11:38 +02:00
rule34vault.py
[rule34vault] update
2024-11-03 09:59:25 +01:00
rule34xyz.py
[rule34xyz] fix 'date' and 'tags' extraction
2025-01-27 10:48:11 +01:00
saint.py
[saint] fix metadata names/IDs of '/d/' URLs
2025-01-27 11:04:41 +01:00
sankaku.py
[sankaku] support alphanumeric book/pool IDs (#6757)
2025-01-02 15:49:07 +01:00
sankakucomplex.py
[sankakucomplex] update domain to 'news.sankakucomplex.com'
2024-07-26 20:39:55 +02:00
scrolller.py
[scrolller] ignore posts without 'mediaSources' (#5051)
2024-11-26 21:50:28 +01:00
seiga.py
[seiga] re-implement login with username & password
2024-05-23 18:52:53 +02:00
senmanga.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
sexcom.py
[sexcom] add 'likes' extractor (#6149)
2024-09-06 07:44:06 +02:00
shimmie2.py
[shimmie2] remove 'tentaclerape.net'
2025-01-26 17:02:07 +01:00
shopify.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
simplyhentai.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
skeb.py
[skeb] prevent 429 errors and need for 'request_key' cookie
2024-09-26 15:15:37 +02:00
slickpic.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
slideshare.py
[common] add '_extract_nextdata' method
2025-01-12 11:48:36 +01:00
smugmug.py
allow passing cookies to OAuth extractors
2024-11-09 18:06:27 +01:00
soundgasm.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
speakerdeck.py
[speakerdeck] inherit from GalleryExtractor
2024-06-15 21:56:51 +02:00
steamgriddb.py
[steamgriddb] disable 'adjust-extensions' for fake-png files (#5274)
2024-11-20 19:19:23 +01:00
subscribestar.py
[subscribestar] fix 'post' extractor (#6582)
2025-02-22 10:08:59 +01:00
szurubooru.py
[szurubooru] remove 'booru.foalcon.com'
2025-01-26 16:42:49 +01:00
tapas.py
[tapas] fix TypeError for locked episodes (#6700)
2024-12-21 15:17:51 +01:00
tcbscans.py
[tcbscans] support other domains
2024-06-21 21:29:11 +05:30
telegraph.py
use 'v[0] == "c"' instead of 'v.startswith("c")'
2024-10-15 08:24:06 +02:00
tiktok.py
[tiktok] add support (#3061 #4177 #5646 #6878 #6708)
2025-02-25 20:10:48 +01:00
tmohentai.py
[tmohentai] 'categories' -> 'genres'
2023-11-21 22:11:43 +01:00
toyhouse.py
[toyhouse] fix metadata extraction
2025-01-27 11:33:17 +01:00
tsumino.py
use 'v[0] == "c"' instead of 'v.startswith("c")'
2024-10-15 08:24:06 +02:00
tumblr.py
allow passing cookies to OAuth extractors
2024-11-09 18:06:27 +01:00
tumblrgallery.py
[tumblrgallery] fix file downloads (#6391)
2024-10-29 08:38:59 +01:00
twibooru.py
[twibooru] match URLs with 'www' subdomain (#6903)
2025-01-30 19:20:02 +01:00
twitter.py
[twitter] revert generated CSRF token length to 32 characters (#6895)
2025-01-30 19:16:10 +01:00
unsplash.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
uploadir.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
urlgalleries.py
[urlgalleries] support new URL format
2025-01-26 17:20:28 +01:00
urlshortener.py
add 'config_instance' method
2024-01-18 03:20:36 +01:00
vanillarock.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
vichan.py
[vichan] remove 'wikieat.club'
2024-06-02 18:16:24 +02:00
vipergirls.py
[vipergirls] improve 'thread' URL pattern
2024-06-29 17:38:57 +02:00
vk.py
[vk] document 'offset', add '--range' support
2024-10-17 21:20:21 +02:00
vsco.py
[vsco] improve 'm3u8' handling
2025-02-12 20:44:43 +01:00
wallhaven.py
[wallhaven] extract 'search[tags]' and 'search[tag_id]' metadata
2025-01-06 17:18:04 +01:00
wallpapercave.py
[wallpapercave] support 'album' listings (#5925)
2024-08-02 16:13:09 +02:00
warosu.py
[warosu] fix extraction
2024-07-26 21:09:07 +02:00
weasyl.py
[weasyl:favorite] update
2024-09-25 20:02:01 +02:00
webmshare.py
fix bug in test_init and update example URLs
2023-09-14 13:27:03 +02:00
webtoons.py
[webtoons] fix 'username' and 'author_name' extraction
2025-01-27 12:05:40 +01:00
weebcentral.py
[weebcentral] fix extracting wrong number of chapter pages (#6966)
2025-02-10 16:00:13 +01:00
weibo.py
[weibo] add 'movies' option (#6988)
2025-02-13 18:01:20 +01:00
wikiart.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
wikifeet.py
remove test results in extractor modules
2023-09-11 16:30:55 +02:00
wikimedia.py
[wikimedia] fix non-English Fandom/wiki.gg articles (#6370)
2024-10-24 13:00:02 +02:00
xfolio.py
[xfolio] add initial support (#5514, #6351, #6837)
2025-01-18 15:57:56 +01:00
xhamster.py
[xhamster] fix 'gallery' extractor (#6818)
2025-01-13 18:58:08 +01:00
xvideos.py
[xvideos] fix galleries with more than 500 images (#5244)
2024-02-26 15:36:41 +01:00
yiffverse.py
[yiffverse] add support (#6611)
2024-12-11 10:57:21 +01:00
ytdl.py
[ytdl] simplify '_process_entries()'
2024-09-03 21:17:31 +02:00
zerochan.py
[zerochan] fix parsing regular JSON
2024-12-14 20:41:08 +01:00
zzup.py
[zzup] support 'up.zzup.com' galleries (#6181)
2024-09-15 11:01:27 +02:00