github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,778 Commits 1 Branch 0 Tags
f8cfc3b08afd9768f3a729f9291deed5b45534d7
Commit Graph

183 Commits

Author SHA1 Message Date
Mike Fährmann
d6c6c8a4a0 [twitter] improve '"replies": "self"' (#2665) 
If a username is given in the input URL,
only download from replies by that user.
 2022-06-13 19:21:32 +02:00
Mike Fährmann
9c8d895d19 [twitter] implement 'csrf' option (#2676) 2022-06-13 18:36:39 +02:00
Mike Fährmann
08db8435f1 [twitter] fix pagination for conversion tweets 
a relic from the switch to GraphQL API
 2022-06-13 16:27:30 +02:00
Mike Fährmann
1da3ccf608 [twitter] implement 'expand' option (#2665) 2022-06-12 17:26:51 +02:00
Mike Fährmann
0add1fc090 [twitter] improve strategy for user URLs (#2665) 
- use '/with_replies' when appropriate
- consider 'text-tweets'
- build search query as necessary
 2022-06-12 17:24:53 +02:00
thatfuckingbird
da0696e1f5 recognize vxtwitter URLs (#2621) 2022-05-25 17:01:58 +02:00
Mike Fährmann
dcb580240d [twitter] extract alt texts as 'description' (closes #2617) 2022-05-24 12:37:38 +02:00
Mike Fährmann
915dba8345 [twitter] improve results for regular user URLs 
- continuation of 3346f58a
- use media timeline results (or tweet timeline if retweets are enabled)
  plus search results starting from the last tweet id of the first
  timeline, similar to how Twitter Media Downloader operates
- the old behavior can be forced by appending '/tweets' to a user URL,
  like with '/media' (https://twitter.com/USER/tweets)
  although there should be no need to ever do that
 2022-05-23 18:33:52 +02:00
Mike Fährmann
9df4e0f65b [twitter] disable 'cards' by default 2022-05-21 15:39:25 +02:00
Mike Fährmann
3346f58a2a [twitter] use twMediaDownloader strategy for user URLs 
- use media timeline + search for default user URLs like
  https://twitter.com/SCREEN_NAME
- fetches all/most media for the type of twitter URL that most users
  use with gallery-dl
- can be disabled by setting 'strategy' to any truthy value,
  like "timeline"
 2022-05-02 09:03:35 +02:00
Mike Fährmann
ad5a4b1756 [twitter] fix various syndication issues 
- handle retweets
- fix videos without dimensions in URL (3e942a58)
- fix '"retweets": "self"' filter (#2499)
 2022-04-15 20:49:26 +02:00
Mike Fährmann
3e942a58be [twitter] improve syndication video selection (#2354) 
- ignore .m3u8 manifests
- always select largest format
 2022-04-11 17:06:10 +02:00
thatfuckingbird
4527a35aba [twitter] accept fxtwitter.com URLs (#2484) 2022-04-08 14:32:08 +02:00
Mike Fährmann
1171911dc3 [twitter] add 'syndication' option (#2354) 
to fetch age-restricted content using Twitter's  syndication API
 2022-04-01 16:56:47 +02:00
Mike Fährmann
2aa47e8382 [twitter] handle Tweets with "softIntervention" entries 
or other such things where the actual Tweet data is one level deeper
than usual
 2022-03-03 02:06:54 +01:00
Mike Fährmann
64bbc7969d [twitter] warn about age-restricted Tweets (#2354) 2022-03-03 02:03:27 +01:00
Mike Fährmann
e778be52bc [twitter] update query hashes 2022-03-02 23:05:31 +01:00
Mike Fährmann
4385a34e05 [twitter] fix handling of 429 responses (fixes #2339) 
Twitter doesn't return a valid JSON response for 429 errors anymore.
 2022-02-28 16:42:55 +01:00
Mike Fährmann
bc0e853d30 combine KeyError & IndexError to common base class LookupError 2022-02-11 00:42:49 +01:00
Mike Fährmann
0f1e7ff319 [twitter] fix extraction (#2275) 2022-02-07 23:18:35 +01:00
Mike Fährmann
70e6e1549e [twitter] provide fallback URLs for card images 
f2e8aedd74 (commitcomment-64057751)
 2022-02-03 23:43:18 +01:00
Mike Fährmann
492436f936 [twitter] add 'warnings' option (#2258) 
disable reporting any non-fatal errors by default
 2022-02-02 18:37:19 +01:00
Mike Fährmann
a5163e4c70 [twitter] restore 'logout' functionality (#1719) 2022-02-02 18:21:15 +01:00
Mike Fährmann
d33227fc38 [twitter] restore errors for protected timelines etc (fixes #2237) 2022-01-30 16:42:13 +01:00
Mike Fährmann
8230f31800 [twitter] update query hashes 2022-01-26 00:49:46 +01:00
Mike Fährmann
c180806cec [twitter] fix deleted/invalid retweets (#2225) 2022-01-25 23:57:13 +01:00
Mike Fährmann
2bf554a896 [twitter] fix several errors (#2212, #2216, #2225) 
- fix Tweets with deleted quotes
- fix suspended Tweets without 'legacy' entry
- fix unified_cards without 'type'
 2022-01-25 16:13:22 +01:00
Mike Fährmann
e5242b83bf [twitter] define directory format for events (#2109) 2022-01-24 17:44:17 +01:00
Mike Fährmann
5ed26e1773 [twitter] fix pinned tweets (#2216) 
caused by the changes in dffa440ede
 2022-01-23 22:52:57 +01:00
Mike Fährmann
a9f78e6527 [twitter] improve error handling 
- handle accounts without 'rest_id'
- handle timelines with empty 'instructions'
 2022-01-23 18:01:05 +01:00
Mike Fährmann
729b07c1f5 [twitter] simplify 
- use dict with common GraphQL variables
- reduce 'variables' size with custom JSON encoder instance
- centralise TwitterAPI() creation
 2022-01-23 01:44:55 +01:00
Mike Fährmann
9ca8bb2dc0 [twitter] improve error handling 2022-01-22 23:09:45 +01:00
Mike Fährmann
9a221494c3 [twitter] add 'event' extractor (closes #2109) 2022-01-22 20:55:50 +01:00
Mike Fährmann
14867dad6b [twitter] fix unified cards from search results 2022-01-22 20:25:10 +01:00
Mike Fährmann
dffa440ede [twitter] improve handling of deleted tweets (#2212) 2022-01-22 00:41:58 +01:00
Mike Fährmann
54ef874ba4 [twitter] fix retweet filter (#2212) 2022-01-21 23:53:59 +01:00
Mike Fährmann
cb43f7731b [twitter] update to GraphQL API (#2212) 
The old REST API endpoints, which were not used by Twitter since
summer 2021, are going to finally be phased out it seems, with
'/2/timeline/profile/USERID.json' being the first one.

Only Twitter's search doesn't have a GraphQL interface yet.
 2022-01-21 23:34:41 +01:00
Mike Fährmann
f2e8aedd74 [twitter] changes to 'cards' option 
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
  when 'cards' is set to "ytdl"

"cards": true   --> only download card images
"cards": "ytdl" --> download card images and
                    use youtube_dl on otherwise unsupported cards
 2022-01-15 22:02:57 +01:00
Mike Fährmann
df2f0c09bb [twitter] support "image_carousel_website" unified cards 2022-01-13 16:05:52 +01:00
Mike Fährmann
f587458a3c [twitter] include '4096x4096' as a default image fallback 
(closes #2107, closes #1881)
 2021-12-15 23:19:30 +01:00
Mike Fährmann
ab8eea1a24 [twitter] fix extractor for direct image links (fixes #2030) 2021-11-16 22:57:46 +01:00
Mike Fährmann
4377f1c284 [twitter] distinguish between fatal & nonfatal errors (#2020) 
only show a warning for nonfatal errors
and do not raise a StopExtraction exception
 2021-11-13 22:46:40 +01:00
Mike Fährmann
9156e90f1f [twitter] add 'pinned' option 2021-10-29 22:10:58 +02:00
Mike Fährmann
cd66c3c415 [twitter] add 'size' option (#1881) 2021-10-05 19:14:54 +02:00
Mike Fährmann
94143eb86c [twitter] add 'quote_by' metadata field (#1481) 
Only present for tweets quoted by another tweet.
Represents the tweet_id of said tweet quoting this one.
 2021-09-25 18:15:14 +02:00
Mike Fährmann
da16eabb82 [twitter] ensure card entries have a 'url' (#1868) 2021-09-23 18:02:19 +02:00
Mike Fährmann
0fd959a2a7 [twitter] support '/with_replies' URLs (closes #1833) 2021-09-10 20:44:26 +02:00
Mike Fährmann
6651da27e9 [twitter] fix 'url' extraction for users without 'expanded_url' 
(#1532, #1787)
 2021-08-27 18:41:16 +02:00
Mike Fährmann
ae78d95a5f [twitter] fix issue when filtering quote tweets (#1792) 
When a user quotes his own Tweet and that Tweet gets filtered by
'"quoted": false', it could also get filtered when it appeared later
as regular Tweet.
 2021-08-25 20:04:22 +02:00
Mike Fährmann
0817f468ef [twitter] expand t.co links in user descriptions (#1532, #1787) 2021-08-23 23:34:59 +02:00