github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

include 'zstd' in Accept-Encoding header when supported

Browse Source 
… and slightly update optional dependency list
This commit is contained in:
Mike Fährmann
2024-07-10 00:33:35 +02:00
parent 8aca0e6970
commit eb3ef13d28
2 changed files with 13 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
README.rst
View File

@@ -25,10 +25,12 @@ Dependencies
Optional Optional
-------- --------
- yt-dlp_ or youtube-dl_: HLS/DASH video downloads, ``ytdl`` integration
- FFmpeg_: Pixiv Ugoira conversion - FFmpeg_: Pixiv Ugoira conversion
- yt-dlp_ or youtube-dl_: Video downloads - mkvmerge_: Accurate Ugoira frame timecodes
- PySocks_: SOCKS proxy support - PySocks_: SOCKS proxy support
- brotli_ or brotlicffi_: Brotli compression support - brotli_ or brotlicffi_: Brotli compression support
- zstandard_: Zstandard compression support
- PyYAML_: YAML configuration file support - PyYAML_: YAML configuration file support
- toml_: TOML configuration file support for Python<3.11 - toml_: TOML configuration file support for Python<3.11
- SecretStorage_: GNOME keyring passwords for ``--cookies-from-browser`` - SecretStorage_: GNOME keyring passwords for ``--cookies-from-browser``
@@ -417,11 +419,13 @@ To authenticate with a ``mastodon`` instance, run *gallery-dl* with
.. _pip: https://pip.pypa.io/en/stable/ .. _pip: https://pip.pypa.io/en/stable/
.. _Requests: https://requests.readthedocs.io/en/master/ .. _Requests: https://requests.readthedocs.io/en/master/
.. _FFmpeg: https://www.ffmpeg.org/ .. _FFmpeg: https://www.ffmpeg.org/
.. _mkvmerge: https://www.matroska.org/downloads/mkvtoolnix.html
.. _yt-dlp: https://github.com/yt-dlp/yt-dlp .. _yt-dlp: https://github.com/yt-dlp/yt-dlp
.. _youtube-dl: https://ytdl-org.github.io/youtube-dl/ .. _youtube-dl: https://ytdl-org.github.io/youtube-dl/
.. _PySocks: https://pypi.org/project/PySocks/ .. _PySocks: https://pypi.org/project/PySocks/
.. _brotli: https://github.com/google/brotli .. _brotli: https://github.com/google/brotli
.. _brotlicffi: https://github.com/python-hyper/brotlicffi .. _brotlicffi: https://github.com/python-hyper/brotlicffi
.. _zstandard: https://github.com/indygreg/python-zstandard
.. _PyYAML: https://pyyaml.org/ .. _PyYAML: https://pyyaml.org/
.. _toml: https://pypi.org/project/toml/ .. _toml: https://pypi.org/project/toml/
.. _SecretStorage: https://pypi.org/project/SecretStorage/ .. _SecretStorage: https://pypi.org/project/SecretStorage/

8
gallery_dl/extractor/common.py
View File

@@ -390,6 +390,8 @@ class Extractor():
headers["Accept-Encoding"] = "gzip, deflate, br" headers["Accept-Encoding"] = "gzip, deflate, br"
else: else:
headers["Accept-Encoding"] = "gzip, deflate" headers["Accept-Encoding"] = "gzip, deflate"
if ZSTD:
headers["Accept-Encoding"] += ", zstd"
referer = self.config("referer", self.referer) referer = self.config("referer", self.referer)
if referer: if referer:
@@ -990,6 +992,12 @@ try:
except AttributeError: except AttributeError:
BROTLI = False BROTLI = False
# detect zstandard support
try:
ZSTD = urllib3.response.HAS_ZSTD
except AttributeError:
ZSTD = False
# set (urllib3) warnings filter # set (urllib3) warnings filter
action = config.get((), "warnings", "default") action = config.get((), "warnings", "default")
if action: if action: