Commit Graph

22 Commits

Author SHA1 Message Date
Mike Fährmann
d85fa5fb14 [facebook] do not match '/permalink' URLs (#8679) 2025-12-11 17:28:02 +01:00
Mike Fährmann
d57dc48dcd [tests/results] replace 'datetime' usage
test for 'dt:…' to catch NullDatetime instances,
which inherit from 'datetime.datetime' but are not exactly this class
2025-10-17 17:06:39 +02:00
Mike Fährmann
b79d35e5f2 [facebook] add fallback to 'biography' extraction (#8233) 2025-09-20 22:28:19 +02:00
Mike Fährmann
312f921959 [facebook] extract 'biography' metadata (#8233) 2025-09-17 21:15:16 +02:00
Mike Fährmann
4d2482ef10 [facebook] fix '/photo/?fbid=…&set=…' being matched as a set (#8181)
reorder 'photo' and 'set' extractors
2025-09-08 09:37:28 +02:00
Mike Fährmann
fa8fd260fe [comick] handle volume-only chapters (#8043) 2025-08-15 11:16:42 +02:00
Mike Fährmann
bce1678fd4 [facebook:set] fix/improve 'user_id' extraction (#7848)
https://github.com/mikf/gallery-dl/issues/7848#issuecomment-3181428631
2025-08-13 07:53:50 +02:00
Mike Fährmann
4b3b2ad111 [facebook] use main profile page for avatar & info results 2025-08-11 08:23:43 +02:00
Mike Fährmann
294557fc57 [tests/results] support testing 'post' metadata 2025-08-11 08:23:18 +02:00
Mike Fährmann
0d2b8f53cc [facebook] add 'info' extractor (#6582)
https://github.com/mikf/gallery-dl/issues/6582#issuecomment-3151899420

Currently relies on the profile having a /photos_by page.
2025-08-09 18:49:53 +02:00
Mike Fährmann
47c9e0816e [facebook] support selecting albums by title (#7848) 2025-08-08 21:24:50 +02:00
Mike Fährmann
2192a2c360 [facebook] add initial 'albums' extractor (#7848)
needs GraphQL API to fetch more than the latest 8
2025-08-08 21:22:41 +02:00
Mike Fährmann
b7a87fe2bb [facebook] raise AuthRequired for profiles requiring cookies (#7962) 2025-08-06 09:09:30 +02:00
Mike Fährmann
3c5eac29c3 [facebook] handle profiles without photos / set_id (#7962) 2025-08-05 09:27:47 +02:00
Mike Fährmann
8ee12279c7 [facebook] fix 'avatar' extraction for empty profiles (#7962) 2025-08-03 08:23:28 +02:00
Mike Fährmann
1fbc341e60 [facebook] ensure numeric 'user_id' values (#7953)
handle 'pfbid…' IDs
2025-08-02 10:28:41 +02:00
Mike Fährmann
3eb0b28d6d [facebook] implement 'include' option & add 'avatar' extractor (#7848)
rename 'profile' extractor to 'photos'
2025-07-25 18:20:05 +02:00
Mike Fährmann
456e93d489 [tests:results] enable 'extension' and ':?' in format tests 2025-05-26 13:50:42 +02:00
Mike Fährmann
ce01835995 [facebook] improve 'date' extraction (#7151)
use 'created_time' as alternative when 'publish_time' isn't available
2025-03-10 17:35:32 +01:00
Mike Fährmann
85a37ca039 [facebook] decode surrogate pairs in metadata values (#6599) 2024-12-12 20:20:30 +01:00
Luca Russo
e36cfb73ff added more tests 2024-11-28 10:55:43 +01:00
Luca Russo
e9370b7b8a merge #5626: [facebook] add support (#470, #2612)
* [facebook] add initial support

* renamed extractors & subcategories

* better stability, modularity & naming

* added single photo extractor, warnings & retries

* more metadata + extract author followups

* renamed "album" mentions to "set" for consistency

* cookies are now only used when necessary

also added author followups for singular images

* removed f-strings

* added way to continue extraction from where it left off

also fixed some bugs

* fixed bug wrong subcategory

* added individual video extraction

* extract audio + added ytdl option

* updated setextract regex

* added option to disable start warning

the extractor should be ready :)

* fixed description metadata bug

* removed cookie "safeguard" + fixed for private profiles

I have removed the cookie "safeguard" (not using cookies until they are necessary) as I've come to the conclusion that it does more harm than good. There is no way to detect whether the extractor has skipped private images, that could have been possibly extracted otherwise. Also, doing this provides little to no advantages.

* fixed a few bugs regarding profile parsing

* a few bugfixes

Fixed some metadata attributes from not decoding correctly from non-latin languages, or not showing at all.
Also improved few patterns.

* retrigger checks

* Final cleanups

-Added tests
-Fixed video extractor giving incorrect URLs
-Removed start warning
-Listed supported site correctly

* fixed regex

* trigger checks

* fixed livestream playback extraction + bugfixes

I've chosen to remove the "reactions", "comments" and "views" attributes as I've felt that they require additional maintenance even though nobody would ever actually use them to order files.
I've also removed the "title" and "caption" video attributes for their inconsistency across different videos.
Feel free to share your thoughts.

* fixed regex

* fixed filename fallback

* fixed retrying when a photo url is not found

* fixed end line

* post url fix + better naming

* fix posts

* fixed tests

* added profile.php url

* made most of the requested changes

* flake

* archive: false

* removed unnecessary url extract

* [facebook] update

- more 'Sec-Fetch-…' headers
- simplify 'text.nameext_from_url()' calls
- replace 'sorted(…)[-1]' with 'max(…)'
- fix '_interval_429' usage
- use replacement fields in logging messages

* [facebook] update URL patterns

get rid of '.*' and '.*?'

* added few remaining tests

---------

Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2024-11-26 21:49:11 +01:00