Add support for hashtags (TagPage-s), i.e. explore/tags/<tag> URLs.
This also introduce a get_metadata() method in order to append
possible further metadata per-(sub)extractor.
Refactor and generalize _extract_profilepage() to _extract_page()
in order to be reused by _extract_profilepage() and _extract_tagpage()
simply by passing the type of page (`ProfilePage' or `TagPage') and picking up
the respective fields in shared data.
* [instagram] Add support for GraphSidecar media types
Refactor _extract_postpage() to always return a list of medias.
Fetch common keywords and gracefully handle GraphSidecar media type
by extracting each single media and adding `sidecar_media_id' and
`sidecar_shortcode' keywords to indicate the parent of sidecar
childrens.
While here join the copyright comment lines in a single one.
Closes#178.
* [instagram] Use `yield from' instead of `for ... yield' (thanks @mikf)!
* [instagram] Adjust filename for GraphSidecar medias
Add a possible leading `media_id' of the sidecar for GraphSidecar
media.
Thanks to @mikf for the suggestion!
* [instagram] Add extra metadata for youtube-dl in GraphSidecar childrens
GraphSidecar children ytdl: URLs when consumed by youtube-dl
redirects to the URL of their parent. In GraphSidecar-s with
multiple GraphVideo-s this leads to downloading the same video
multiple times.
Add a `_ytdl_index' field to indicate the index of the youtube-dl
playlist corresponding the children of the sidecar.
This will be used by the `ytdl' downloader.
- use original image if available
- support video formats
- remove user info for ImageExtractor (it is no longer possible to get
image owner information for a single image)
An URL alone isn't good enough to distinguish between a gallery or a
gallery-listing, so the new extractor decides what to do based on the
page's content.
- Sometimes an ad interfered when trying to get a download URL
- Resolving "www.hentai-foundry.com" yields an invalid(?) IPv6 address
(2607:5300:60:ca9e:feed:dead:beef:1) and urllib3 only tries to connect
to the IPv4 variant after a rather long wait time
