github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,510 Commits 1 Branch 0 Tags
c180806cec986cb046cad46837598a8331b7c3cd
Commit Graph

158 Commits

Author SHA1 Message Date
Mike Fährmann
c180806cec [twitter] fix deleted/invalid retweets (#2225) 2022-01-25 23:57:13 +01:00
Mike Fährmann
2bf554a896 [twitter] fix several errors (#2212, #2216, #2225) 
- fix Tweets with deleted quotes
- fix suspended Tweets without 'legacy' entry
- fix unified_cards without 'type'
 2022-01-25 16:13:22 +01:00
Mike Fährmann
e5242b83bf [twitter] define directory format for events (#2109) 2022-01-24 17:44:17 +01:00
Mike Fährmann
5ed26e1773 [twitter] fix pinned tweets (#2216) 
caused by the changes in dffa440ede
 2022-01-23 22:52:57 +01:00
Mike Fährmann
a9f78e6527 [twitter] improve error handling 
- handle accounts without 'rest_id'
- handle timelines with empty 'instructions'
 2022-01-23 18:01:05 +01:00
Mike Fährmann
729b07c1f5 [twitter] simplify 
- use dict with common GraphQL variables
- reduce 'variables' size with custom JSON encoder instance
- centralise TwitterAPI() creation
 2022-01-23 01:44:55 +01:00
Mike Fährmann
9ca8bb2dc0 [twitter] improve error handling 2022-01-22 23:09:45 +01:00
Mike Fährmann
9a221494c3 [twitter] add 'event' extractor (closes #2109) 2022-01-22 20:55:50 +01:00
Mike Fährmann
14867dad6b [twitter] fix unified cards from search results 2022-01-22 20:25:10 +01:00
Mike Fährmann
dffa440ede [twitter] improve handling of deleted tweets (#2212) 2022-01-22 00:41:58 +01:00
Mike Fährmann
54ef874ba4 [twitter] fix retweet filter (#2212) 2022-01-21 23:53:59 +01:00
Mike Fährmann
cb43f7731b [twitter] update to GraphQL API (#2212) 
The old REST API endpoints, which were not used by Twitter since
summer 2021, are going to finally be phased out it seems, with
'/2/timeline/profile/USERID.json' being the first one.

Only Twitter's search doesn't have a GraphQL interface yet.
 2022-01-21 23:34:41 +01:00
Mike Fährmann
f2e8aedd74 [twitter] changes to 'cards' option 
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
  when 'cards' is set to "ytdl"

"cards": true   --> only download card images
"cards": "ytdl" --> download card images and
                    use youtube_dl on otherwise unsupported cards
 2022-01-15 22:02:57 +01:00
Mike Fährmann
df2f0c09bb [twitter] support "image_carousel_website" unified cards 2022-01-13 16:05:52 +01:00
Mike Fährmann
f587458a3c [twitter] include '4096x4096' as a default image fallback 
(closes #2107, closes #1881)
 2021-12-15 23:19:30 +01:00
Mike Fährmann
ab8eea1a24 [twitter] fix extractor for direct image links (fixes #2030) 2021-11-16 22:57:46 +01:00
Mike Fährmann
4377f1c284 [twitter] distinguish between fatal & nonfatal errors (#2020) 
only show a warning for nonfatal errors
and do not raise a StopExtraction exception
 2021-11-13 22:46:40 +01:00
Mike Fährmann
9156e90f1f [twitter] add 'pinned' option 2021-10-29 22:10:58 +02:00
Mike Fährmann
cd66c3c415 [twitter] add 'size' option (#1881) 2021-10-05 19:14:54 +02:00
Mike Fährmann
94143eb86c [twitter] add 'quote_by' metadata field (#1481) 
Only present for tweets quoted by another tweet.
Represents the tweet_id of said tweet quoting this one.
 2021-09-25 18:15:14 +02:00
Mike Fährmann
da16eabb82 [twitter] ensure card entries have a 'url' (#1868) 2021-09-23 18:02:19 +02:00
Mike Fährmann
0fd959a2a7 [twitter] support '/with_replies' URLs (closes #1833) 2021-09-10 20:44:26 +02:00
Mike Fährmann
6651da27e9 [twitter] fix 'url' extraction for users without 'expanded_url' 
(#1532, #1787)
 2021-08-27 18:41:16 +02:00
Mike Fährmann
ae78d95a5f [twitter] fix issue when filtering quote tweets (#1792) 
When a user quotes his own Tweet and that Tweet gets filtered by
'"quoted": false', it could also get filtered when it appeared later
as regular Tweet.
 2021-08-25 20:04:22 +02:00
Mike Fährmann
0817f468ef [twitter] expand t.co links in user descriptions (#1532, #1787) 2021-08-23 23:34:59 +02:00
Mike Fährmann
7c0ae88185 [twitter] add 'url' to user objects (#1532, #1787) 2021-08-23 22:51:35 +02:00
Mike Fährmann
5919dc5b5a [twitter] slightly improve '_transform_user()' 2021-08-23 22:28:09 +02:00
Mike Fährmann
6b56b3ebe1 [twitter] report API errors as generic StopExtraction exceptions 
prevents duplicate logging messages for nonexistent users
(#1759)
 2021-08-21 22:46:22 +02:00
Mike Fährmann
c866fcba48 [twitter] fix 'logout' (#1719) 
delete 'auth_token' cookie and cookies.txt path
 2021-08-16 01:36:34 +02:00
Mike Fährmann
52984f7e22 [twitter] add option to log out when blocked (#1719) 2021-08-12 19:11:41 +02:00
Mike Fährmann
e5a93e113f [twitter] extend 'replies' option (#1254) 
Allow setting 'replies to '"self"' to only download from self-replies.
 2021-08-10 22:14:00 +02:00
Mike Fährmann
229498b8aa [twitter] warn about suspended accounts etc (closes #1759) 2021-08-09 02:58:27 +02:00
Mike Fährmann
414bdc95a3 [twitter] set 'retweet_id' for original retweets (#1481) 2021-07-02 21:50:37 +02:00
Mike Fährmann
5323c1c73a [twitter] ensure guest tokens are returned as string (#1665) 2021-07-01 14:35:53 +02:00
Mike Fährmann
035562bd11 [twitter] remove old-style URLs from image fallback lists 2021-06-28 16:25:24 +02:00
Mike Fährmann
a751afdfb3 [twitter] change some defaults 
- 'retweets' option: true -> false
- 'quoted' option  : true -> false

  i.e. disable downloading tweets from other user's timelines by default

- search directory:
    '["{category}", "Search", "{search}"]' ->
    '["{category}", "{user[name]}"]'

  i.e. change it to the same as other twitter extractors (#1308)
 2021-06-11 21:26:11 +02:00
Mike Fährmann
b5affc62aa [twitter] rename 'text-only' to 'text-tweets' (#570) 2021-05-22 21:41:12 +02:00
Mike Fährmann
724ca61f36 [twitter] add 'text-only' option (#570) 2021-05-22 17:01:49 +02:00
Mike Fährmann
394fbb5f56 [twitter] strip useless t.co links (#1532) 
The 'full_text' of Tweets with media content usually ends with a t.co
link to itself. This commit removes those.
 2021-05-17 00:20:29 +02:00
Mike Fährmann
41457dbb1b [twitter] resolve t.co URLs in 'content' (#1532) 2021-05-15 18:52:37 +02:00
Mike Fährmann
17b0ccb071 [twitter] add missing retweet media entities (fixes #1555) 
from the original tweets
 2021-05-14 22:51:01 +02:00
Mike Fährmann
fd858eed7b [twitter] add 'user_likes' metadata field for liked tweets 
i.e. the 'screen_name' of the user whose liked tweets get extracted.

Ideally this would replace 'user' or at least be in the same format,
but that would break backwards compatibility or be impossible/too
complicated thanks to API result differences.

(#1421)
 2021-04-02 03:41:41 +02:00
Mike Fährmann
8d124a3766 [twitter] rename variables 2021-04-02 02:49:53 +02:00
Mike Fährmann
105f3c9666 [twitter] add extractor for direct image links (closes #1417) 2021-04-02 02:45:23 +02:00
Mike Fährmann
ebd142e2a8 [twitter] don't use youtube-dl for cards when videos are disabled 
(#1416)
 2021-04-01 14:26:08 +02:00
Mike Fährmann
ccfa5a8694 [twitter] better error message when logging in with 2FA (#1409) 2021-03-27 18:26:37 +01:00
Mike Fährmann
2846235669 [twitter] allow specifying a custom format for user results 
(#1337)
 2021-03-21 22:26:26 +01:00
Mike Fährmann
3378b39719 [twitter] implement 'users' option (#1337) 2021-03-16 00:51:05 +01:00
Mike Fährmann
5d69e437d0 [twitter] add option to download all media from a conversation 
(fixes #1319)
 2021-02-26 13:50:46 +01:00
Mike Fährmann
de0656941b [twitter] add extractor for followed users (#1337) 
https://twitter.com/USER/following or
https://twitter.com/id:USERID/following
 2021-02-22 18:22:01 +01:00