github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,280 Commits 1 Branch 0 Tags
28385bec7abd24e9d0970b85bf8288648d652e72
Commit Graph

6280 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
28385bec7a [bunkr] extract 'id_url' metadata (#6935) 
and use it as 'id' alternative instead of 'name' in default archive IDs
 2025-02-06 20:40:35 +01:00
Mike Fährmann
b9675ea764 [bunkr] update default archive ID format (#6935) 
use 'name' when there is no proper 'id' value available.
 2025-02-05 21:54:43 +01:00
Mike Fährmann
873cbf6b36 [docs] add more details to 'user-agent' and 'browser' docs (#6917) 2025-02-03 20:54:27 +01:00
Mike Fährmann
5807daa19a [issuu] unescape HTML entities 2025-02-02 18:33:18 +01:00
Mike Fährmann
6c9b20fe45 [philomena] download 'full' URLs (#6922) 
'view_url' URLs sometimes result in 404 errors
 2025-02-02 18:23:46 +01:00
Mike Fährmann
4ab9237f1d [philomena] fix 'date' values without UTC offset (#6921) 
Some instances do not include a UTC offset or 'Z' in their datetime
values, e.g. 2024-03-14T13:46:46 compared to 2024-03-14T13:46:46Z
 2025-02-02 16:32:28 +01:00
Mike Fährmann
1a9138f25e [aes] handle errors during 'Cryptodome' import (#6906) 2025-02-02 15:01:17 +01:00
Mike Fährmann
7c96c2368f [subscribestar] detect and handle redirects (#6916) 2025-02-01 21:03:24 +01:00
Mike Fährmann
52ac3a7802 [release] build 'gallery-dl.exe' on Python 3.13 (#6684) 
and rename the former Python 3.8 version to 'gallery-dl_x86.exe'.

Currently building with PyInstaller, as I wasn't able to get py2exe to
work in this environment, but the startup times are noticeably longer.

Considering switching to nuitka, maybe even for all standalone builds.
 2025-02-01 19:58:51 +01:00
Mike Fährmann
ddb2c4d69d [executables] fix SSLError when using HTTPAdapter (#6393) 
always load certifi certificates instead of relying on
'load_default_certs()', which might load no certs at all
 2025-01-31 20:36:41 +01:00
Mike Fährmann
463e123283 [twibooru] match URLs with 'www' subdomain (#6903) 2025-01-30 19:20:02 +01:00
Mike Fährmann
64dc655ed6 [twitter] revert generated CSRF token length to 32 characters (#6895) 
revert d9c4fcc7fa
 2025-01-30 19:16:10 +01:00
Mike Fährmann
ba9e4795cd release version 1.28.5 2025-01-28 15:11:23 +01:00
Mike Fährmann
de81f8e7c7 merge #6891: [vsco] fix 'JSONDecodeError' (#6887) 2025-01-28 14:46:31 +01:00
CasualYT31
a8c4665b5a VSCO: prevPageToken Bugfix 
#6887
 2025-01-28 11:50:31 +00:00
Mike Fährmann
8b0f082636 [fapachi] fix extraction (#6881) 2025-01-28 11:49:55 +01:00
Mike Fährmann
4874c8e1d1 [artstation] restore 'browser' and 'tls12' defaults 
partially revert 954796a466
 2025-01-28 11:36:06 +01:00
Mike Fährmann
1b5e0c0e87 [issuu] fix 'user' extractor 2025-01-27 21:56:11 +01:00
Mike Fährmann
fe815b9226 [danbooru][e621] use lower 'threshold' value by default (#6886) 
fixes tag searches ending prematurely when a few posts are missing
 2025-01-27 20:41:24 +01:00
Mike Fährmann
d110dfd2da [tests] update extractor results 2025-01-27 17:15:32 +01:00
Mike Fährmann
b9567ae3e0 [pornpics] fix pagination results from HTML page 
prevent URLs containing the root URL twice
 2025-01-27 16:12:19 +01:00
Mike Fährmann
b201f91d8d [pornpics] prevent redirect when retrieving gallery page 2025-01-27 16:01:34 +01:00
Mike Fährmann
65863239a0 [webtoons] fix 'username' and 'author_name' extraction 2025-01-27 12:05:40 +01:00
Mike Fährmann
44f4902544 [toyhouse] fix metadata extraction 
- date
- artists
- characters
 2025-01-27 11:33:17 +01:00
Mike Fährmann
019f9068bf [saint] fix metadata names/IDs of '/d/' URLs 2025-01-27 11:04:41 +01:00
Mike Fährmann
aec5827248 [rule34xyz] fix 'date' and 'tags' extraction 2025-01-27 10:48:11 +01:00
Mike Fährmann
db9cee20c2 [pixiv] fix 'sanity_level' workaround (#4327) 
The App API now returns
https://s.pximg.net/common/images/limit_unviewable_360.png
as URL for "Work cannot be displayed" artworks.
 2025-01-27 10:10:38 +01:00
Mike Fährmann
61d80df509 [mangaread] fix manga metadata extraction 2025-01-26 21:28:24 +01:00
Mike Fährmann
4bd9b2672c [mangahere] fix chapter extraction 2025-01-26 21:16:24 +01:00
Mike Fährmann
6b6a4942b6 [mangafox] fix chapter extraction 2025-01-26 21:11:36 +01:00
Mike Fährmann
83e50e43a8 [hiperdex] update domain to 'hiperdex.com' 2025-01-26 19:26:03 +01:00
Mike Fährmann
2cdb7e86ca [hiperdex] fix 'description' extraction 2025-01-26 19:17:10 +01:00
Mike Fährmann
21e5a84b6b [architizer] fix extraction 
switch " to '
 2025-01-26 17:28:06 +01:00
Mike Fährmann
bf361ec7d3 [urlgalleries] support new URL format 
... but the site itself is broken, i.e. image pages are empty.
 2025-01-26 17:20:28 +01:00
Mike Fährmann
254ffd3fcd [shimmie2] remove 'tentaclerape.net' 
"Site Not Found"
 2025-01-26 17:02:07 +01:00
Mike Fährmann
d2164af63d [komikcast] update domain to 'komikcast.la' 2025-01-26 16:54:14 +01:00
Mike Fährmann
804fd048ef [szurubooru] remove 'booru.foalcon.com' 
DNS record of foalcon.com no longer exists
 2025-01-26 16:42:49 +01:00
Mike Fährmann
b271a874ed [fanleaks] remove module 
DNS record of fanleaks.club no longer exists
 2025-01-26 16:35:46 +01:00
Mike Fährmann
98c068a379 [artstation] simplify '_no_cache()' 2025-01-26 16:12:25 +01:00
Mike Fährmann
954796a466 [artstation] prevent CF challenges (#5817, #5658, #5564, #5554) 2025-01-26 16:00:16 +01:00
Mike Fährmann
4003e4b688 [4archive] fix 'TypeError' when processing 'date' metadata 2025-01-26 12:55:01 +01:00
Mike Fährmann
d9f59ec56e [adultempire] bypass age confirmation check 
set 'ageConfirmed' cookie
 2025-01-26 11:24:48 +01:00
Mike Fährmann
acaae40254 [facebook] evaluate 'directory' per photo (#6874) 2025-01-24 21:37:20 +01:00
Mike Fährmann
4904186b70 [deviantart] prevent crash when accessing premium_folder_data (#6873) 
e.g. for inactive accounts
 2025-01-24 19:08:32 +01:00
Mike Fährmann
f5add4048e [nekohouse] fix pagination (#6871) 
use distinct names for URL values
 2025-01-24 10:18:52 +01:00
Mike Fährmann
783cb83258 merge #6865: [lofter] improve error handling 
- Add 'NotFoundError' when the lofter blog returns a 404
- Stop extractor if the blog is empty, because this returns
  {offset: -1} in the data which previously infinity-looped
  the extractor.
- Prevent errors when the blog is locked, the posts in
  self.posts() are None
 2025-01-21 19:23:48 +01:00
Mike Fährmann
4d609e284a merge #6833: [kemonoparty] Support /posts endpoint and Creator Tag Calls 2025-01-21 19:22:00 +01:00
BishopRed
b11434a069 [kemonoparty] Support /posts endpoint and Creator Tag Calls 
- Adding support for calling a creator with a tag selected.
    It is using a legacy endpoint but there is no other way currently
    documented to get the users post filtered by a tag.
- Fixing the User Tags feature to be paginated
    offset is not defined in the API but it is supported.
- Fixed the `/posts` endpoint not working:
    1. Added check along with metadata to make sure there is a
       creator/service information as that is a requirement
    2. Fixed the parameter from tags -> tag.
    3. Fixed the _paginate call to exit correctly when there is
       a key required for the data (it was prematurely exiting)
- Adding a type of caching mechanism for the metadata/user information.
    The current logic would work just fine if looking up for a
    singular user, however for the multiple posts via normal
    filtering would cause it to either:
    This builds a local cache during the process so it should
    only make a call for the user info once during the process.
- Updating to meet standards
    Fixes
      1. Reset formatting for unnecessary line changes
      2. Removed Type Hinting
      3.Replaced f-string with "".format
   Updates
     Renamed function creator_posts_tags -> creator_tagged_posts
     for clarity of what it does (get posts tags vs get tagged posts)
- Fixing check for the length of response:
    1. If it is list - just check len
    2. If there is a key - check that the key length is less
       than the batch.
- add test for '?tag=...' user URLs
    plus some code simplifications
 2025-01-21 19:20:22 +01:00
Mike Fährmann
cb1a75eefc [twitter] handle errors during file extraction (#6647) 2025-01-21 18:23:54 +01:00
Mike Fährmann
d9c4fcc7fa [twitter] generate longer CSRF token values 2025-01-21 18:19:25 +01:00