* [iwara] Add initial support
* [iwara] Add search support
* [iwara] Code cleanup
* [iwara] Small fixes and additions
* [iwara] Add tag support
* [iwara] Add mime-type to metadata
* [iwara] Refactor patterns/matching using urllib
* [iwara] Add unit tests
* [iwara] Update docs
* [iwara] Fix linting on older Python versions
* [iwara] update 'IwaraAPI' interface class
- define endpoints inside methods
- implement and use _call() and _pagination()
- cache auth tokens
* [iwara] split and rename 'profile' extractor
TODO:
- update test results
- simplify code
* [iwara] simplify '_user_params()' usage
* [iwara] update 'video' extractor
and move user data extraction into 'yield_video'
* [iwara] update 'image' extractor
and move user info extraction into 'yield_image()'
* [iwara] update 'playlist' extractor
* [iwara] update 'search' extractor
* [iwara] update 'tag' extractor
* [iwara] simplify 'yield_image' usage
perform API calls to get full 'files' list inside the function
* [iwara] add video "image" test
* [iwara] provide 'date' metadata
* [iwara] simplify 'source()'
remove urllib.parse usage
* [iwara] small optimizations
* get("key", {}) -> get("key") or {}
* split("…", 1) -> partition("…")
* use f-strings for all patterns
* [iwara] add missing 'keyarg=1' to profile() memcache decorator
* [tests/iwara] update results
* [iwara] extract more 'user' metadata
* [iwara] update default format strings
include 'date' in filenames to order them chronologically
* [iwara] restructure image/video handling
- use less generators
- make processing individual media items non-fatal
* [iwara] fix login and token handling
* [iwara] add 'favorite' extractor
* [iwara] add 'following' and 'followers' extractors
---------
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
category changes:
- kemonoparty -> kemono
- coomerparty -> coomer
- koharu -> schalenetwork
also wanted to rename '2chan' -> 'sturdychan',
but the site's main page is still titled '2chen'
* Add TikTok photo support
#3061#4177
* Address linting errors
* Fix more test failures
* Forgot to update category names in tests
* Looking into re issue
* Follow default yt-dlp output template
* Fix format string error on 3.5
* Support downloading videos and audio
Respond to comments
Improve archiving and file naming
* Forgot to update supportedsites.md
* Support user profiles
* Fix indentation
* Prevent matching with more than one TikTok extractor
* Fix TikTok regex
* Support TikTok profile avatars
* Fix supportedsites.md
* TikTok: Ignore no formats error
In my limited experience, this doesn't mean that gallery-dl can't download the photo post (but this could mean that you can't download the audio)
* Fix error reporting message
* TikTok: Support more URL formats
vt.tiktok.com
www.tiktok.com/t/
* TikTok: Only download avatar when extracting user profile
* TikTok: Document profile avatar limitation
* TikTok: Add support for www.tiktokv.com/share links
* Address Share -> Sharepost issue
* TikTok: Export post's creation date in JSON (ISO 8601)
* [tiktok] update
* [tiktok] update 'vmpost' handling
just perform a HEAD request and handle its response
* [tiktok] build URLs from post IDs
instead of reusing unchanged input URLs
* [tiktok] combine 'post' and 'sharepost' extractors
* [tiktok] update default filenames
put 'id' and 'num' first to ensure better file order
* [tiktok] improve ytdl usage
- speed up extraction by passing '"extract_flat": True'
- pass more user options and cookies
- pre-define 'TikTokUser' extractor usage
* [tiktok] Add _COOKIES entry to AUTH_MAP
* [tiktok] Always download user avatars
* [tiktok] Add more documentation to supportedsites.md
* [tiktok] Address review comments
---------
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
* first commit
* add --
* skip video embeds
* fix typo
* removed ambiguity
* add category support
* code tweaks
* more reliable embed extraction
* handle 403 errors (testing done)
* added "parent_id" keyword
* added "parent", "parent_type" keywords
the extractor should be now ready to merge!
* removed unnecessary dict unpacking
* added empty text messages extraction
* added "channel_topic"
* even more metadata extraction
can now extract all embeds images & text, as well as server banners. also code is much better.
* added user avatar and banner
* better pagination
* fix regression
* minor tweaks
* Made requested changes
* [facebook] add initial support
* renamed extractors & subcategories
* better stability, modularity & naming
* added single photo extractor, warnings & retries
* more metadata + extract author followups
* renamed "album" mentions to "set" for consistency
* cookies are now only used when necessary
also added author followups for singular images
* removed f-strings
* added way to continue extraction from where it left off
also fixed some bugs
* fixed bug wrong subcategory
* added individual video extraction
* extract audio + added ytdl option
* updated setextract regex
* added option to disable start warning
the extractor should be ready :)
* fixed description metadata bug
* removed cookie "safeguard" + fixed for private profiles
I have removed the cookie "safeguard" (not using cookies until they are necessary) as I've come to the conclusion that it does more harm than good. There is no way to detect whether the extractor has skipped private images, that could have been possibly extracted otherwise. Also, doing this provides little to no advantages.
* fixed a few bugs regarding profile parsing
* a few bugfixes
Fixed some metadata attributes from not decoding correctly from non-latin languages, or not showing at all.
Also improved few patterns.
* retrigger checks
* Final cleanups
-Added tests
-Fixed video extractor giving incorrect URLs
-Removed start warning
-Listed supported site correctly
* fixed regex
* trigger checks
* fixed livestream playback extraction + bugfixes
I've chosen to remove the "reactions", "comments" and "views" attributes as I've felt that they require additional maintenance even though nobody would ever actually use them to order files.
I've also removed the "title" and "caption" video attributes for their inconsistency across different videos.
Feel free to share your thoughts.
* fixed regex
* fixed filename fallback
* fixed retrying when a photo url is not found
* fixed end line
* post url fix + better naming
* fix posts
* fixed tests
* added profile.php url
* made most of the requested changes
* flake
* archive: false
* removed unnecessary url extract
* [facebook] update
- more 'Sec-Fetch-…' headers
- simplify 'text.nameext_from_url()' calls
- replace 'sorted(…)[-1]' with 'max(…)'
- fix '_interval_429' usage
- use replacement fields in logging messages
* [facebook] update URL patterns
get rid of '.*' and '.*?'
* added few remaining tests
---------
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
- use self.groups[…] to access matched values
- extract more metadata (count, width, height, size)
- remove type hint
- add tests
- update docs/supportedsites
