github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,698 Commits 1 Branch 0 Tags
e00c717b156fb3717bb5d6302fbb8ffaaa56c39d
Commit Graph

567 Commits

Author SHA1 Message Date
brerk
e00c717b15 [turbo] update 'saint' extractors (#8893 #8896) 
* Implements turbo.py & remove from domain pattern from saints.py
* Remove leftover commented pattern from saints.py
* Make turbo.py comply with flake8
* Add album support
* Improved metadata extracion for albums and single files & created turbo.py tests using saints.py test
* Align turbo.py extractor with flake8 rules
* Fix #class name on turbo.py tests
* Fix #category test
* Fix #category test x2
* Fix #category tests
* Fix #category tests
* Fix TurboMediaExtractor self.groups unpacking

* update basic module formatting
* replace 'saint' with 'turbo' in modules list
* remove saint extractors and tests
* update & simplify 'media' extractor
* update & simplify 'album' extractor
* update tests
* update supportedsites
* update 'category-map' & 'config-map'

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2026-01-19 22:20:13 +01:00
Mike Fährmann
cc5bfa6eb0 [xenforo] support 'celebforum.to' (#8902) 2026-01-19 16:04:33 +01:00
Mike Fährmann
235fb25f6b [scripts/release] add more checks before committing release 
- check if CHANGELOG was updated
- sync with origin/master
 2026-01-17 20:40:48 +01:00
Stephon Parker
43387c535d [thefap] add support (#8821 #8822) 
* adding site support for thefap.com
* fixing typo in url tld
* improve & simplify 'model' extractor
* update 'post' extractor
* update docs/supportedsites
* add tests

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2026-01-14 21:11:56 +01:00
Mike Fährmann
200ad21cbd [scripts/generate_result] fix 'small()' for empty objects 2026-01-11 22:55:19 +01:00
Mike Fährmann
e9346aa698 [scripts/generate_result] reorder '#range' when using '#pattern' 2026-01-11 22:27:52 +01:00
Mike Fährmann
921de81931 [scripts/generate_result] use child extractor 'pattern' 2026-01-11 22:27:07 +01:00
Mike Fährmann
d7c1c30c62 [booth] add 'category' extractor (#8867) 2026-01-11 22:17:01 +01:00
Mike Fährmann
76930ad44c [saint] support 'turbovid.cr' URLs (#8851) 2026-01-10 18:08:13 +01:00
Mike Fährmann
bb7d540c7a [xenforo] add 'media-category' extractor (#8785) 2026-01-10 18:08:05 +01:00
Mike Fährmann
60104f20af [xenforo] implement 'media' support (#8785) 
add 'media-user' and 'media-item' extractors
TODO: 'media-category' extractor (?)
 2026-01-09 21:21:19 +01:00
Mike Fährmann
51c155df94 [yourlesbians] add 'album' extractor (#8713) 2025-12-30 20:33:50 +01:00
CasualYouTuber31
a6c845bdc8 [tiktok] remove yt-dlp dependency & add support for more post types (#8715) 
#7246 #8035 #8466 #8730

* [tiktok] support extracting videos directly without yt-dlp
* [tiktok] support extracting users directly without yt-dlp
* [tiktok] fixing logic, tests, linting errors
* [tiktok] implement tiktok-range support for non-yt-dlp user extractor
* [tiktok] Skip range filter if no ranges are given
* [tiktok] Remove debug code
* [tiktok] only check for faulty device IDs during the first couple of passes
    I think the original yt-dlp solution assumes that if a device ID works once, it will always work.
    Plus, my approach would cause needless retries in certain cases if hasMorePrevious does end up being wrong like the original algorithm accounts for. So let's copy the original algorithm here, too.
* [tiktok] support stories
* [tiktok] you can now extract audio without extracting photos
* [tiktok] add TiktokFollowingExtractor
* [tiktok] update supportedsites to include stories
* [tiktok] Keep tiktok-range option for no content user account test
    It acts as a nice guard against that account suddenly having lots of posts to extract
* [tiktok] TiktokUserExtractor and TiktokFollowingExtractor rewrite
* [tiktok] Fix avatar naming convention to match that of posts
* [tiktok] remove type hints for compatibility with older Python versions
* [tiktok] Improve performance of TiktokFollowingExtractor
    This was largely achieved using the story/batch/item_list endpoint
* [tiktok] Forgot to run flake8
* [tiktok] remove old constant
* [tiktok] Support order-posts config item
* [tiktok] flake8
* [tiktok] Older Python versions don't support match
* [tiktok] always ask for posts in chronological order when in "desc" mode
    We should aim to avoid having pinned posts returned before non-pinned ones
* [tiktok] Add liked posts extraction
* [tiktok] Add reposts extraction
* [tiktok] Add saved posts extraction

* cleanup imports
* remove '# MARK:' comments
* remove & simplify 'except' statements
    KeyboardInterrupt & SystemExit inherit from BaseException (not Exception)
    and therefore don't need special handling
* split 'user' extractor
* move PATTERNs into their respective functions
* use dict comprehensions
* add only-matching test URLs for split user extractors
* update config docs
    rename 'tiktok-user-extractor' to 'ytdl'
* document '"popular"' 'order-posts' value
* inline and remove 'util.chunk()'
 2025-12-30 17:17:57 +01:00
Mike Fährmann
c2917ad5c5 [ahottie] add aupport (#8710) 2025-12-29 18:47:10 +01:00
Mike Fährmann
4f535ec606 [imgpv] add 'image' extractor (#8773) 2025-12-28 21:36:06 +01:00
Mike Fährmann
b69e5a7a58 [scripts/init] use 'operator +' when building 'pattern' 2025-12-22 11:31:48 +01:00
Mike Fährmann
af475a311b [scripts/release] reduce expected executable sizes 2025-12-20 10:47:20 +01:00
Mike Fährmann
afe867361f [weebdex] add support (#8722) 2025-12-19 20:06:40 +01:00
Mike Fährmann
c268f793a4 [xenforo] support 'allthefallen.moe/forum' (#3249 #8268) 2025-12-15 17:28:16 +01:00
Mike Fährmann
4f33751c30 [koofer] add 'shared' extractor (#8700) 2025-12-14 18:05:23 +01:00
Mike Fährmann
774cb1e1a0 [comedywildlifephoto] add 'gallery' extractor (#8690) 2025-12-14 17:59:56 +01:00
Mike Fährmann
ab2c03b39e [xenforo] implement generic XenForo forum extractors 
support
- https://simpcity.cr/
- https://nudostar.com/forum/ (#8333)
 2025-12-11 21:26:34 +01:00
SpiffyChatterbox
484a15ff83 [fitnakedgirls] add support (#8671) 
* [fitnakedgirls] add extractor

Add support for fitnakedgirls.com:
- Photo galleries (/photos/gallery/)
- Category pages (/photos/gallery/category/)
- Tag pages (/photos/tag/)
- Video posts (/videos/)
- Blog posts (/fitblog/)

Handles both newer (wp-block-image) and older (size-large) templates.

* simplify & fix

- use '_extract_title' method
- move '_pagination' into base class
- update 'FitnakedgirlsTagExtractor' pattern

* update docs/supportedsites

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-12-10 21:33:17 +01:00
Mike Fährmann
f5fafd7977 [fikfap] add support (#8673) 
add 'post' & 'user' extractors
 2025-12-10 18:39:05 +01:00
Mike Fährmann
d5f4a3f5b0 [redbust] remove module (#6582) 
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3635792318
 2025-12-10 09:20:05 +01:00
Mike Fährmann
994cc34f75 [audiochan] add initial support (#8602) 2025-11-28 21:46:22 +01:00
MyFinalBellyache
d74809c538 [eporner] add support (#8581) 
* [eporner] Add support
* Line-length <80 chars
* Fix flake8 indentation
* update
    - generalize example URL
    - strip "Photo Gallery" from 'title'
    - extract 'id'
* update supportedsites
* update test results
 2025-11-20 18:03:54 +01:00
MyFinalBellyache
2bff6828e1 [scripts/init] Fix error running with default values (#8583) 
Fixes:
```sh
python scripts/init.py newsite https://newsite.com
# [INFO] gallery_dl/extractor/newsite.py
# [ERROR] TypeError: 'NoneType' object is not iterable
# Traceback (most recent call last):
#   File "/gallery-dl/scripts/init.py", line 58, in init_extractor
#     fp.write(func(args))
#              ^^^^^^^^^^
#   File "/gallery-dl/scripts/init.py", line 88, in generate_module
#     {generate_extractors(args)}\
#      ^^^^^^^^^^^^^^^^^^^^^^^^^
#   File "/gallery-dl/scripts/init.py", line 108, in generate_extractors_basic
#     for subcat in args.subcategories:
# TypeError: 'NoneType' object is not iterable
# [INFO] gallery_dl/extractor/__init__.py
# [INFO] test/results/newsite.py
```
 2025-11-20 11:00:35 +01:00
MyFinalBellyache
a9687d2928 [pornstarstube] add support (#8576) 
* Add support for pornstars.tube
* update
    - adjust pattern/example/page_url
    - extract '' & '' metadata
    - prevent unnecessary request
    - fix result URLs
    - return list comprehension result
* update supportedsites
* update test results
* fix extractor name

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-11-19 16:25:01 +01:00
MyFinalBellyache
cc7003a14c [okporn] add support (#8575) 
* Add support for ok.porn
* update
    - adjust pattern/example/page_url
    - prevent unnecessary request
    - extract 'tags'
    - text.extract -> text.extr
    - return list comprehension result
* update supportedsites name
* update test results

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-11-19 15:49:13 +01:00
SpiffyChatterbox
f6e67116f2 [cfake] add support (#707 #6021 #8549 #8430) 
* Create cfake.py
* Update __init__.py
* Create cfake.py tests
* update
    - simplify & combine code
    - adjust 'pattern': use '[^/?#]', match lines and groups
    - generalize example URLs
    - update default filenames
* update docs/supportedsites
* update test results

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-11-15 20:55:00 +01:00
Mike Fährmann
b6f855d3f3 [scripts/generate_result] don't reorder "small" dicts/lists 2025-11-11 22:02:01 +01:00
dev-KingMaster
793956d2d5 [arena] add 'channel' extractor (#5847 #8509) 
* feat: issue 5847 site support for are.na
* flake8 lint error fix
* class name error fix
* update
    - prevent unnecessary request to 'page_url'
    - fix pagination
    - simplify block extraction code
    TODO: - rewrite without GalleryExtractor
          - extractors for Blocks, Users, etc
* supportedsites
* tests
* rename to 'channel' extractor
* update site title to 'Are.na'
* prioritize attachments

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-11-11 20:44:40 +01:00
gengenson-code
a9018d1911 Added support for sex.com's feed (cookie required) (#8519) 
* Added Extractor for sex.com feed
* Removed old comment
* add test URL
* update supportedsites
* simplify & fix flake8 newlines
* warn about missing session cookie

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2025-11-09 11:57:12 +01:00
Mike Fährmann
7ed04d830e [postimg] force 'postimg.cc' domain (#8505) 2025-11-04 18:02:25 +01:00
Mike Fährmann
c0e665e3db [cyberfile] add 'shared' extractor (#8323) 
https://github.com/mikf/gallery-dl/issues/8323#issuecomment-3366857916
 2025-10-28 17:43:42 +01:00
Mike Fährmann
d708181e5f [simpcity] implement login with username & password (#8418) 2025-10-15 18:35:43 +02:00
Mike Fährmann
c1d21e8cb9 [twitter] remove login support (#4202 #6029 #6040 #8362) 
broken feature
 2025-10-07 08:32:40 +02:00
Mike Fährmann
22358b3d0a [release] update expected executable sizes (#8359) 2025-10-06 19:50:04 +02:00
Mike Fährmann
0aac99673d [poipiku] fix original file downloads (#8356) 
now requires 'POIPIKU_LK' cookie
 2025-10-06 15:33:11 +02:00
Mike Fährmann
168a6508c0 [mangareader] add support (#6529 #6868) 2025-10-02 20:09:48 +02:00
Mike Fährmann
74588e3ea9 [scripts/pyprint] use triple quotes if string value contains " 2025-10-02 20:09:48 +02:00
Mike Fährmann
ddc8c8ae09 [scripts/init] implement specifying a site's URL as 'category' 2025-10-02 11:51:32 +02:00
Mike Fährmann
bcefcd5ae1 [s3ndpics] add initial support (#8322) 2025-10-01 15:26:34 +02:00
Mike Fährmann
228fa00d47 [scripts/generate_result] implement '-l/--logging' command-line option 2025-09-29 20:24:24 +02:00
Mike Fährmann
6d4d808661 [mangafire] add support (#7064 #7701) 2025-09-25 22:27:15 +02:00
Mike Fährmann
d03572c397 [scripts] remove 'create_test_data' & 'export_tests' (#4504) 
- create_test_data.py was superseded by generate_test_result.py
- export_tests.py has done its job and is no longer needed
 2025-09-23 18:54:04 +02:00
ramoswilly
facc0e0250 [Danbooru] add 'random' extractor 
Adds a new `DanbooruRandomExtractor` to support downloading from `danbooru.donmai.us/posts/random?tags=` URLs.
 2025-09-22 21:14:34 +02:00
Mike Fährmann
96ce1926a4 [thehentaiworld] add support (#274 #8237) 2025-09-19 22:13:07 +02:00
Mike Fährmann
dbf6de2482 [mangataro] add support (#8237) 2025-09-19 10:38:34 +02:00