* [girlsreleased] add support
* Lint
* Change "galleries" to "sets"
As it is on the site
* Add pagination
Add tests
* Fix tests
* Remove leftover print()
* Don't remove first set
* Yield pages
* Add filename metadata
* [girlsreleased] Refactor
* Return models as array
* Add filename numbering
Add date metadata
* Add URL metadata
* Spawn set extractor the right way
* Adjust model/site regex
* update
- restructure some code
- remove constructors
- use f-strings
* expand tests
* update docs/supportedsites
category changes:
- kemonoparty -> kemono
- coomerparty -> coomer
- koharu -> schalenetwork
also wanted to rename '2chan' -> 'sturdychan',
but the site's main page is still titled '2chen'
* Add TikTok photo support
#3061#4177
* Address linting errors
* Fix more test failures
* Forgot to update category names in tests
* Looking into re issue
* Follow default yt-dlp output template
* Fix format string error on 3.5
* Support downloading videos and audio
Respond to comments
Improve archiving and file naming
* Forgot to update supportedsites.md
* Support user profiles
* Fix indentation
* Prevent matching with more than one TikTok extractor
* Fix TikTok regex
* Support TikTok profile avatars
* Fix supportedsites.md
* TikTok: Ignore no formats error
In my limited experience, this doesn't mean that gallery-dl can't download the photo post (but this could mean that you can't download the audio)
* Fix error reporting message
* TikTok: Support more URL formats
vt.tiktok.com
www.tiktok.com/t/
* TikTok: Only download avatar when extracting user profile
* TikTok: Document profile avatar limitation
* TikTok: Add support for www.tiktokv.com/share links
* Address Share -> Sharepost issue
* TikTok: Export post's creation date in JSON (ISO 8601)
* [tiktok] update
* [tiktok] update 'vmpost' handling
just perform a HEAD request and handle its response
* [tiktok] build URLs from post IDs
instead of reusing unchanged input URLs
* [tiktok] combine 'post' and 'sharepost' extractors
* [tiktok] update default filenames
put 'id' and 'num' first to ensure better file order
* [tiktok] improve ytdl usage
- speed up extraction by passing '"extract_flat": True'
- pass more user options and cookies
- pre-define 'TikTokUser' extractor usage
* [tiktok] Add _COOKIES entry to AUTH_MAP
* [tiktok] Always download user avatars
* [tiktok] Add more documentation to supportedsites.md
* [tiktok] Address review comments
---------
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
* first commit
* add --
* skip video embeds
* fix typo
* removed ambiguity
* add category support
* code tweaks
* more reliable embed extraction
* handle 403 errors (testing done)
* added "parent_id" keyword
* added "parent", "parent_type" keywords
the extractor should be now ready to merge!
* removed unnecessary dict unpacking
* added empty text messages extraction
* added "channel_topic"
* even more metadata extraction
can now extract all embeds images & text, as well as server banners. also code is much better.
* added user avatar and banner
* better pagination
* fix regression
* minor tweaks
* Made requested changes
* [facebook] add initial support
* renamed extractors & subcategories
* better stability, modularity & naming
* added single photo extractor, warnings & retries
* more metadata + extract author followups
* renamed "album" mentions to "set" for consistency
* cookies are now only used when necessary
also added author followups for singular images
* removed f-strings
* added way to continue extraction from where it left off
also fixed some bugs
* fixed bug wrong subcategory
* added individual video extraction
* extract audio + added ytdl option
* updated setextract regex
* added option to disable start warning
the extractor should be ready :)
* fixed description metadata bug
* removed cookie "safeguard" + fixed for private profiles
I have removed the cookie "safeguard" (not using cookies until they are necessary) as I've come to the conclusion that it does more harm than good. There is no way to detect whether the extractor has skipped private images, that could have been possibly extracted otherwise. Also, doing this provides little to no advantages.
* fixed a few bugs regarding profile parsing
* a few bugfixes
Fixed some metadata attributes from not decoding correctly from non-latin languages, or not showing at all.
Also improved few patterns.
* retrigger checks
* Final cleanups
-Added tests
-Fixed video extractor giving incorrect URLs
-Removed start warning
-Listed supported site correctly
* fixed regex
* trigger checks
* fixed livestream playback extraction + bugfixes
I've chosen to remove the "reactions", "comments" and "views" attributes as I've felt that they require additional maintenance even though nobody would ever actually use them to order files.
I've also removed the "title" and "caption" video attributes for their inconsistency across different videos.
Feel free to share your thoughts.
* fixed regex
* fixed filename fallback
* fixed retrying when a photo url is not found
* fixed end line
* post url fix + better naming
* fix posts
* fixed tests
* added profile.php url
* made most of the requested changes
* flake
* archive: false
* removed unnecessary url extract
* [facebook] update
- more 'Sec-Fetch-…' headers
- simplify 'text.nameext_from_url()' calls
- replace 'sorted(…)[-1]' with 'max(…)'
- fix '_interval_429' usage
- use replacement fields in logging messages
* [facebook] update URL patterns
get rid of '.*' and '.*?'
* added few remaining tests
---------
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
- Unescape title and URL
- Add tags and categories metadata
Lookup tag id with API instead of downloading tag page
- Add category extractor
- Add tests
- Rename EveriaExtractor to EveriaPostExtractor
- Fix EveriaPostExtractor example
- Lookup tags/categories by post id
- Add date extractor
- Remove leftover pages parameter
- Add error handling for invalid dates.
- Add filename numbering
Parse date
- Rename extract() to images()
- Remove html import
- Fix search/date URLs with page number
- Fix tag/category search
- Fix post extractor
- Fix tag, category extractors
- Fix search extractor
- Only load first page once
- Fix date extractor
- Fix tests
- Clean up search extractor
- Added playlists support for rule34vault
- Added support for posts in rule34vault
- Fixed supported sites with script
- Fixed posts pattern in rule34vault
- Added tests for rule34vault
- Clean
- Fixed lint warnings
