Commit Graph

4304 Commits

Author SHA1 Message Date
Mike Fährmann
e7522482bb merge #3463: [lynxchan] support 'bbw-chan.nl' 2023-01-01 15:50:09 +01:00
Mike Fährmann
7d6c846176 [fanbox] return 'imageMap' files in order (#2718) 2023-01-01 15:12:05 +01:00
Mike Fährmann
dc8e7ff54e [bunkr] fix URLs returned by API (#3481) 2023-01-01 14:07:15 +01:00
Mike Fährmann
1fc3845cf6 Revert "add liberapay to FUNDING.yml"
This reverts commit 2bdcd85ced.
2023-01-01 13:54:09 +01:00
Mike Fährmann
0ed5986062 document custom 'output.mode' (#2529) 2022-12-30 17:14:42 +01:00
enduser420
5fedef3a1a [fanleaks] update 'model' URL pattern 2022-12-30 19:29:55 +05:30
enduser420
5a740ef78b [fanleaks] add 'post' and 'model' extractors 2022-12-30 19:24:05 +05:30
ClosedPort22
7c8eab8d52 [twitter] implement 'syndication=extended'
to be able to fetch extended user metadata
2022-12-30 20:48:41 +08:00
ClosedPort22
be3286206a [twitter] assume 'conversation_id' when using syndication
not possible to expand replies at the momemt
2022-12-30 13:57:37 +08:00
ClosedPort22
ce8dbb1ccc [twitter] fix crash when using 'expand' and 'syndication'
caused by KeyError: 'conversation_id_str'
2022-12-30 12:45:44 +08:00
Mike Fährmann
d651d45239 implement specifying ranges in slice notation (#918, #2865)
e.g.
- '1:101'   or ':101' or ':101:'  for files 1 to 100
- '1::2'    or '::2'              for every second file
- '1:101:5' or ':101:5'           for files 1, 6, 11, ..., 91, 96

(the second argument specifies the first index NOT included)
2022-12-27 18:21:12 +01:00
ClosedPort22
38786a9593 [twitter] refactor extraction of TwitPic URLs
flattening
2022-12-27 12:23:12 +08:00
Mike Fährmann
3616adfc75 implement '--range' with Python ranges 2022-12-26 18:32:34 +01:00
enduser420
527bb2c4ab [lynxchan/bbw-chan] add 'thread' and 'board' extractors 2022-12-26 20:59:42 +05:30
pi_allen
64902f518e [docs] Update links and fix field typo 2022-12-26 07:46:41 +01:00
ClosedPort22
5d5ea58887 [twitter] update docs for 'quoted' & 'unique' 2022-12-25 15:59:26 +08:00
blankie
f82ee93676 [danbooru] extract uploader metadata (if metadata is set) 2022-12-25 11:00:59 +07:00
ClosedPort22
250d35107c [twitter] prioritize tweet type checks (#3439)
Do not consider a tweet seen before applying 'retweet', 'quote' and
'reply' checks. Otherwise the original tweets will also be skipped if
the "derivative" tweets and the original tweets are from the same user.
2022-12-24 12:50:52 +08:00
Mike Fährmann
1800bd7d14 allow '*-filter' options to be a list of expressions 2022-12-23 22:20:21 +01:00
ClosedPort22
3eb352fcb0 [twitter] force HTTPS for TwitPic URLs 2022-12-23 18:16:34 +08:00
Dante Mogrim
cb47104e3f Add Homebrew instructions to installation list
Hi! [Just noticed these instructions seemed to be missing.](https://formulae.brew.sh/formula/gallery-dl)

Have a good one!
2022-12-20 18:26:28 +01:00
Mike Fährmann
73ab5d84c0 update docs/configuration.rst 2022-12-20 17:30:46 +01:00
Mike Fährmann
2d7d80d302 release version 1.24.2 2022-12-18 14:41:03 +01:00
Mike Fährmann
bee354c264 Merge pull request #3415 from enduser420/extractor/fapello
[fapello] add 'post', 'user' and 'path' extractors
2022-12-17 20:05:46 +01:00
Mike Fährmann
8d7585534e Merge pull request #3367 from the-blank-x/deviantart-view
[deviantart] add /view URL support
2022-12-17 20:05:35 +01:00
blankie
6614d94b08 [deviantart] add /view URL support 2022-12-17 23:27:55 +07:00
Mike Fährmann
dd6eeb4336 Merge pull request #3366 from ClosedPort22/da-extra-stash
[deviantart] extract sta.sh URLs from `text_content`
2022-12-17 15:42:12 +01:00
Mike Fährmann
f36cbb3911 Merge pull request #3413 from ClosedPort22/e621-manual-pagination
[e621] implement manual pagination
2022-12-17 15:42:00 +01:00
ClosedPort22
dd4a4a3fa6 [e621] softcode the pagination threshold 2022-12-17 21:06:47 +08:00
ClosedPort22
9faa4ed738 [e621] refactor pagination control
as suggested by @mikf
2022-12-17 20:05:34 +08:00
Mike Fährmann
7851a2c520 [seiga] raise error when redirected to login page (#3401) 2022-12-17 12:52:17 +01:00
Mike Fährmann
68ce5f965d [instagram] remove unused code 2022-12-17 12:52:17 +01:00
Mike Fährmann
4063563cd7 [zerochan] update for layout v3
- remove cookie disabling v3
- fix and improve metadata extraction
2022-12-17 12:51:51 +01:00
Mike Fährmann
1e6407ca98 Merge pull request #3414 from pubak42/master
[sex.com] Download videos from cdn (#3408)
2022-12-17 12:30:34 +01:00
Mike Fährmann
a46b7dd494 Merge pull request #3419 from ClosedPort22/imgur-io
[imgur] add support for imgur.io URLs
2022-12-17 12:29:37 +01:00
ClosedPort22
bf1649dadb [imgur] add support for imgur.io URLs 2022-12-17 14:33:44 +08:00
enduser420
7e08e2d982 [fapello] set 'filename_fmt' 2022-12-16 20:57:39 +05:30
enduser420
e5076ba056 [fapello] add 'post', 'user' and 'path' extractors 2022-12-16 16:53:32 +05:30
pubak42
e7326cdf1d [sex.com] Download videos from cdn (#3408)
The format of video sources was changed recently to be a full URL with https:// in the beginning.
The original extractor code appended the video source URL to root url of the website, thus yielding
invalid url in format ...sex.comhttps... that failed to resolve.
2022-12-16 10:16:40 +01:00
ClosedPort22
d0ad6d0e67 [e621] implement manual pagination mode 2022-12-16 16:15:36 +08:00
Mike Fährmann
6f0735568c [2chen] fix file URLs 2022-12-15 18:05:32 +01:00
enduser420
a2be06d873 [2chen] add '.club' support (#3406) 2022-12-15 17:51:02 +01:00
Mike Fährmann
a6d4733e11 [pixiv] extract 'date_url' metadata (#3405)
i.e. the datetime encoded in each file URL.

https://i.pximg.net/img-master/img/2022/12/01/13/44/55/12345678_p0.jpg
->
2022-12-01 13:44:55 +09:00
->
2022-12-01 04:44:55
2022-12-15 11:40:20 +01:00
Mike Fährmann
7f99d4b3f7 link to required MSVC package for Windows executables
(#3331, #3372)
2022-12-14 20:00:48 +01:00
Mike Fährmann
1317625ec4 [webmshare] add 'video' extractor (#2410) 2022-12-14 19:59:07 +01:00
Mike Fährmann
90a9c0790f [twitter] update 'search' pagination (#544)
Only stop when list of all returned Tweets is empty
instead of when no valid Tweet was found.
2022-12-14 19:56:59 +01:00
Mike Fährmann
1cbc234819 [mangafox] extract more metadata (#3167) 2022-12-14 19:56:59 +01:00
Mike Fährmann
3082544fff misc fixes
- fix typo (#3399)
- remove double assignment
- [bunkr] update things I forgot in 6b6f886d
- [soundgasm] adjust 'archive_fmt' (#3388)
2022-12-14 13:30:27 +01:00
enduser420
41bf236d36 [lynxchan] add generic extractors for lynxchan imageboards (#3394)
* [lynxchan] add generic extractors for lynxchan imageboards

includes kohlchan.net, endchan.org:wq

* [lynxchan] set pop default to empty tuple

* Apply suggestions from code review

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2022-12-12 13:13:39 +01:00
Mike Fährmann
3c75c3bbc4 [soundgasm] add 'user' extractor (#3384)
based on code from PR #3388 by @enduser420
2022-12-12 12:23:59 +01:00