github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,602 Commits 1 Branch 0 Tags
ebfd87bbb04a2ff1ebdbbfd3dc90e67007972cd7
Commit Graph

59 Commits

Author SHA1 Message Date
Mike Fährmann
00c6821a3f replace 2-element f-strings with simple '+' concatenations 
Python's 'ast' module and its 'NodeVisitor' class
were incredibly helpful in identifying these
 2025-12-22 11:26:04 +01:00
Mike Fährmann
e006d26c8e Revert "use f-strings when building 'pattern'" 
revert d7c97d5a97.
 2025-12-20 22:07:37 +01:00
Mike Fährmann
73db1846e1 [furaffinity] fix 'tags' metadata (#8724) 
Co-authored-by: d-koc <141529764+d-koc@users.noreply.github.com>
 2025-12-18 08:55:27 +01:00
Mike Fährmann
968597a302 yield 3-tuples for Message.Directory 
adapt tuples to the same length and semantics as other messages
 2025-12-05 21:39:52 +01:00
Mike Fährmann
d7c97d5a97 use f-strings when building 'pattern' 2025-10-20 21:23:11 +02:00
Mike Fährmann
085616e0a8 [dt] replace 'text.parse_datetime()' & 'text.parse_timestamp()' 2025-10-17 17:43:06 +02:00
Mike Fährmann
a097a373a9 simplify if statements by using walrus operators (#7671) 2025-07-22 20:57:54 +02:00
Mike Fährmann
096bc6f784 replace more '.format(…)' calls with f-strings (#7671) 2025-07-10 21:17:17 +02:00
Mike Fährmann
a4fb9e6d33 [furaffinity] fix 'submissions' results for new layout (#7759) 2025-06-30 19:05:19 +02:00
Mike Fährmann
9dbe33b6de replace old %-formatted and .format(…) strings with f-strings (#7671) 
mostly using flynt
https://github.com/ikamensh/flynt
 2025-06-29 17:50:19 +02:00
Mike Fährmann
41191bb60a 'match.group(N)' -> 'match[N]' (#7671) 
2.5x faster
 2025-06-18 13:05:58 +02:00
Mike Fährmann
e08ec7e083 update copyright notices 2025-06-13 00:03:41 +02:00
Mike Fährmann
811b665e33 remove @staticmethod decorators 
There might have been a time when calling a static method was faster
than a regular method, but that is no longer the case. According to
micro-benchmarks, it is 70% slower in CPython 3.13 and it also makes
executing the code of a class definition slower.
 2025-06-12 22:50:52 +02:00
Mike Fährmann
e199396872 [common] simplify 'user' extractors by using 'Dispatch' mixin 2025-05-24 18:04:53 +02:00
Mike Fährmann
b81fc5c124 replace text.rextract() with rextr() 2025-05-23 18:28:58 +02:00
Mike Fährmann
e1bdcd97e1 [furaffinity] extract 'scraps' metadata (#7015) 
boolean value indicating whether a post is part of a user's Scraps
folder or the main gallery
 2025-03-12 16:29:16 +01:00
Deer-Spangle
859f1e7d04 [furaffinity] Adding a FuraffinityFolderExtractor, which extracts a single folder 
- Ensure FuraffinityGalleryExtractor doesn't detect folder links
- Fix example URL for folder extractor
- Reordering classes a bit
- Another tweak of the regex
- One more go at the regex..
- cleanup
 2025-03-12 14:00:50 +01:00
Mike Fährmann
984116ada7 [furaffinity] improve 'artist_url' extraction (#7115 #7123) 2025-03-06 14:15:11 +01:00
Mike Fährmann
b7bb40c06b [furaffinity] fix 'artist' metadata (#7115 #6582) 
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-2696619061
 2025-03-04 16:48:17 +01:00
Mike Fährmann
876169ded5 [furaffinity] use a 1s delay between requests by default (#7054) 2025-02-25 20:12:54 +01:00
Mike Fährmann
7260e236d8 [furaffinity] fix 'favorite' pagination (#6151) 2024-09-06 08:52:05 +02:00
Mike Fährmann
c07a074418 [furaffinity] remove end of system messages 2024-08-07 18:39:34 +02:00
Mike Fährmann
846512f6cd [furaffinity] add 'submissions' extractor (#5954) 2024-08-07 16:34:32 +02:00
Mike Fährmann
9a8ec95398 [furaffinity] improve 2024-07-18 01:39:58 +02:00
Nicholas Bishop
4f3be2f571 Remove leftover split() statement 
Forgot to remove this from a previous implementation attempt
 2024-07-04 16:50:49 -04:00
Nicholas Bishop
f43bccb5be [furaffinity] Add 'thumbnail' (#1284) and 'folders' properties 
Retrieve 'thumbnail' and 'folders' properties for each post.
'thumbnail' (#1284):
 - Preview image used for search results, writing posts, music, etc.
 - Filename format: <post_id>@600-<directory_containing_full_image>.jpg
'Folders' (related to #1817):
 - A list of all gallery folders containing this post
 - Folder name format: [<folder_category> - ]<folder_name>
 - Only works on new layout; old layout does not show folders, so list will be empty

A test is included for each property.
 2024-07-04 15:41:14 -04:00
Delphox
11109d5bad [furaffinity] match xfuraffinity.com 2024-05-08 12:15:47 -03:00
Mike Fährmann
5aefa9465c [furaffinity] simplify pattern, match original domain first 2024-04-25 21:58:34 +02:00
Delphox
6969963125 [furaffinity] support fxfuraffinity and fxraffinity urls 2024-04-25 13:25:39 -03:00
Mike Fährmann
2cd801232b fix --range causing crashes (#4557) 
regression caused by a383eca7
 2023-09-22 16:28:20 +02:00
Mike Fährmann
a453335a9f remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
a383eca7f6 decouple extractor initialization 
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
 2023-07-25 22:16:16 +02:00
Mike Fährmann
d97b8c2fba consistent cookie-related names 
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
 2023-07-22 01:20:50 +02:00
Mike Fährmann
b6c959744d [furaffinity] improve 'description' HTML (#4224) 
- ignore header
- include footer and closing <div> if present
 2023-06-24 20:30:40 +02:00
Mike Fährmann
5e3a1749c8 [furaffinity] simplify 'favorite_id' assignment 2023-06-09 16:35:59 +02:00
Bad Manners
952c03bc9e Add fav_id data to FuraffinityFavoriteExtractor 
An extra field is collected when paginating favorites, and saved to
a temporary cache variable. This field is identical for both the old
and the new page layouts for FurAffinity, but can only be collected
during pagination, hence the cache variable. Other FurAffinity
extractors should be unaffected by this change.
 2023-06-02 19:53:47 -03:00
Mike Fährmann
b0cb4a1b9c replace 'text.extract()' with 'text.extr()' where possible 2022-11-05 01:14:09 +01:00
Mike Fährmann
c4cc387f7d [furaffinity] fix search result pagination (fixes #2402) 2022-03-18 13:44:36 +01:00
Mike Fährmann
f1c853c6ef [furaffinity] add 'layout' option (#2277) 
to be able to force gallery-dl to parse according to a specific layout
in case its auto-detect fails
 2022-02-11 00:28:47 +01:00
Mike Fährmann
4efe56f419 [furaffinity] improve new/old layout detection (fixes #2277) 2022-02-08 18:10:52 +01:00
Mike Fährmann
87ce3fa669 [furaffinity] warn when no session cookies were found 2021-12-15 16:21:05 +01:00
Mike Fährmann
975e0a4fe0 [furaffinity] unquote search queries (#1958) 
instead of unescape
(unquote -> url params, unescape -> html entities)
 2021-10-17 20:48:08 +02:00
Mike Fährmann
6b229ac829 [furaffinity] expand URL pattern for searches (closes #1780) 2021-08-23 23:38:52 +02:00
Mike Fährmann
a5de2244d4 [furaffinity] fix using 'category-tranfer' (#1274) 2021-08-08 23:50:32 +02:00
Mike Fährmann
5b1c62bfa9 [furaffinity] add 'external' option (closes #1492) 2021-08-08 19:16:02 +02:00
Mike Fährmann
4adc44df69 [furaffinity] improve metadata extraction (fixes #1630) 
Fetch 'title' and 'artist' metadata from a different location,
since for posts with an empty title the <title> element is
completely empty and does not contain the artist's name.
 2021-06-19 03:29:00 +02:00
Mike Fährmann
2919d78bfc update extractor test results 2021-02-14 15:37:39 +01:00
Mike Fährmann
89a2bcbb2d [furaffinity] add 'descriptions' option (#1231) 2021-01-19 19:09:29 +01:00
Mike Fährmann
968d3e8465 remove '&' from URL patterns 
'/?&#' -> '/?#' and '?&#' -> '?#'

According to https://www.ietf.org/rfc/rfc3986.txt, URLs are
"organized hierarchically" by using "the slash ("/"), question
mark ("?"), and number sign ("#") characters to delimit components"
 2020-10-22 23:31:25 +02:00
Mike Fährmann
844502cad5 update extractor test results 2020-10-03 19:24:19 +02:00