github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,676 Commits 1 Branch 0 Tags
9adea93aef5221cfd0dca5ba21c2d75a5601519a
Commit Graph

174 Commits

Author SHA1 Message Date
Mike Fährmann
3346f58a2a [twitter] use twMediaDownloader strategy for user URLs 
- use media timeline + search for default user URLs like
  https://twitter.com/SCREEN_NAME
- fetches all/most media for the type of twitter URL that most users
  use with gallery-dl
- can be disabled by setting 'strategy' to any truthy value,
  like "timeline"
 2022-05-02 09:03:35 +02:00
Mike Fährmann
ad5a4b1756 [twitter] fix various syndication issues 
- handle retweets
- fix videos without dimensions in URL (3e942a58)
- fix '"retweets": "self"' filter (#2499)
 2022-04-15 20:49:26 +02:00
Mike Fährmann
3e942a58be [twitter] improve syndication video selection (#2354) 
- ignore .m3u8 manifests
- always select largest format
 2022-04-11 17:06:10 +02:00
thatfuckingbird
4527a35aba [twitter] accept fxtwitter.com URLs (#2484) 2022-04-08 14:32:08 +02:00
Mike Fährmann
1171911dc3 [twitter] add 'syndication' option (#2354) 
to fetch age-restricted content using Twitter's  syndication API
 2022-04-01 16:56:47 +02:00
Mike Fährmann
2aa47e8382 [twitter] handle Tweets with "softIntervention" entries 
or other such things where the actual Tweet data is one level deeper
than usual
 2022-03-03 02:06:54 +01:00
Mike Fährmann
64bbc7969d [twitter] warn about age-restricted Tweets (#2354) 2022-03-03 02:03:27 +01:00
Mike Fährmann
e778be52bc [twitter] update query hashes 2022-03-02 23:05:31 +01:00
Mike Fährmann
4385a34e05 [twitter] fix handling of 429 responses (fixes #2339) 
Twitter doesn't return a valid JSON response for 429 errors anymore.
 2022-02-28 16:42:55 +01:00
Mike Fährmann
bc0e853d30 combine KeyError & IndexError to common base class LookupError 2022-02-11 00:42:49 +01:00
Mike Fährmann
0f1e7ff319 [twitter] fix extraction (#2275) 2022-02-07 23:18:35 +01:00
Mike Fährmann
70e6e1549e [twitter] provide fallback URLs for card images 
f2e8aedd74 (commitcomment-64057751)
 2022-02-03 23:43:18 +01:00
Mike Fährmann
492436f936 [twitter] add 'warnings' option (#2258) 
disable reporting any non-fatal errors by default
 2022-02-02 18:37:19 +01:00
Mike Fährmann
a5163e4c70 [twitter] restore 'logout' functionality (#1719) 2022-02-02 18:21:15 +01:00
Mike Fährmann
d33227fc38 [twitter] restore errors for protected timelines etc (fixes #2237) 2022-01-30 16:42:13 +01:00
Mike Fährmann
8230f31800 [twitter] update query hashes 2022-01-26 00:49:46 +01:00
Mike Fährmann
c180806cec [twitter] fix deleted/invalid retweets (#2225) 2022-01-25 23:57:13 +01:00
Mike Fährmann
2bf554a896 [twitter] fix several errors (#2212, #2216, #2225) 
- fix Tweets with deleted quotes
- fix suspended Tweets without 'legacy' entry
- fix unified_cards without 'type'
 2022-01-25 16:13:22 +01:00
Mike Fährmann
e5242b83bf [twitter] define directory format for events (#2109) 2022-01-24 17:44:17 +01:00
Mike Fährmann
5ed26e1773 [twitter] fix pinned tweets (#2216) 
caused by the changes in dffa440ede
 2022-01-23 22:52:57 +01:00
Mike Fährmann
a9f78e6527 [twitter] improve error handling 
- handle accounts without 'rest_id'
- handle timelines with empty 'instructions'
 2022-01-23 18:01:05 +01:00
Mike Fährmann
729b07c1f5 [twitter] simplify 
- use dict with common GraphQL variables
- reduce 'variables' size with custom JSON encoder instance
- centralise TwitterAPI() creation
 2022-01-23 01:44:55 +01:00
Mike Fährmann
9ca8bb2dc0 [twitter] improve error handling 2022-01-22 23:09:45 +01:00
Mike Fährmann
9a221494c3 [twitter] add 'event' extractor (closes #2109) 2022-01-22 20:55:50 +01:00
Mike Fährmann
14867dad6b [twitter] fix unified cards from search results 2022-01-22 20:25:10 +01:00
Mike Fährmann
dffa440ede [twitter] improve handling of deleted tweets (#2212) 2022-01-22 00:41:58 +01:00
Mike Fährmann
54ef874ba4 [twitter] fix retweet filter (#2212) 2022-01-21 23:53:59 +01:00
Mike Fährmann
cb43f7731b [twitter] update to GraphQL API (#2212) 
The old REST API endpoints, which were not used by Twitter since
summer 2021, are going to finally be phased out it seems, with
'/2/timeline/profile/USERID.json' being the first one.

Only Twitter's search doesn't have a GraphQL interface yet.
 2022-01-21 23:34:41 +01:00
Mike Fährmann
f2e8aedd74 [twitter] changes to 'cards' option 
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
  when 'cards' is set to "ytdl"

"cards": true   --> only download card images
"cards": "ytdl" --> download card images and
                    use youtube_dl on otherwise unsupported cards
 2022-01-15 22:02:57 +01:00
Mike Fährmann
df2f0c09bb [twitter] support "image_carousel_website" unified cards 2022-01-13 16:05:52 +01:00
Mike Fährmann
f587458a3c [twitter] include '4096x4096' as a default image fallback 
(closes #2107, closes #1881)
 2021-12-15 23:19:30 +01:00
Mike Fährmann
ab8eea1a24 [twitter] fix extractor for direct image links (fixes #2030) 2021-11-16 22:57:46 +01:00
Mike Fährmann
4377f1c284 [twitter] distinguish between fatal & nonfatal errors (#2020) 
only show a warning for nonfatal errors
and do not raise a StopExtraction exception
 2021-11-13 22:46:40 +01:00
Mike Fährmann
9156e90f1f [twitter] add 'pinned' option 2021-10-29 22:10:58 +02:00
Mike Fährmann
cd66c3c415 [twitter] add 'size' option (#1881) 2021-10-05 19:14:54 +02:00
Mike Fährmann
94143eb86c [twitter] add 'quote_by' metadata field (#1481) 
Only present for tweets quoted by another tweet.
Represents the tweet_id of said tweet quoting this one.
 2021-09-25 18:15:14 +02:00
Mike Fährmann
da16eabb82 [twitter] ensure card entries have a 'url' (#1868) 2021-09-23 18:02:19 +02:00
Mike Fährmann
0fd959a2a7 [twitter] support '/with_replies' URLs (closes #1833) 2021-09-10 20:44:26 +02:00
Mike Fährmann
6651da27e9 [twitter] fix 'url' extraction for users without 'expanded_url' 
(#1532, #1787)
 2021-08-27 18:41:16 +02:00
Mike Fährmann
ae78d95a5f [twitter] fix issue when filtering quote tweets (#1792) 
When a user quotes his own Tweet and that Tweet gets filtered by
'"quoted": false', it could also get filtered when it appeared later
as regular Tweet.
 2021-08-25 20:04:22 +02:00
Mike Fährmann
0817f468ef [twitter] expand t.co links in user descriptions (#1532, #1787) 2021-08-23 23:34:59 +02:00
Mike Fährmann
7c0ae88185 [twitter] add 'url' to user objects (#1532, #1787) 2021-08-23 22:51:35 +02:00
Mike Fährmann
5919dc5b5a [twitter] slightly improve '_transform_user()' 2021-08-23 22:28:09 +02:00
Mike Fährmann
6b56b3ebe1 [twitter] report API errors as generic StopExtraction exceptions 
prevents duplicate logging messages for nonexistent users
(#1759)
 2021-08-21 22:46:22 +02:00
Mike Fährmann
c866fcba48 [twitter] fix 'logout' (#1719) 
delete 'auth_token' cookie and cookies.txt path
 2021-08-16 01:36:34 +02:00
Mike Fährmann
52984f7e22 [twitter] add option to log out when blocked (#1719) 2021-08-12 19:11:41 +02:00
Mike Fährmann
e5a93e113f [twitter] extend 'replies' option (#1254) 
Allow setting 'replies to '"self"' to only download from self-replies.
 2021-08-10 22:14:00 +02:00
Mike Fährmann
229498b8aa [twitter] warn about suspended accounts etc (closes #1759) 2021-08-09 02:58:27 +02:00
Mike Fährmann
414bdc95a3 [twitter] set 'retweet_id' for original retweets (#1481) 2021-07-02 21:50:37 +02:00
Mike Fährmann
5323c1c73a [twitter] ensure guest tokens are returned as string (#1665) 2021-07-01 14:35:53 +02:00