use values of 'retries' options correctly

The RE-tries option now specifies exactly that: the maximum number a
failed HTTP request is re-tried. For example a value of 2 will now
correctly stop after 3 attempts: the initial one + 2 re-tries.

The maximum wait-time now also caps at 30min and increases exponentially
for both extractor.request() and downloader.http.download().
This commit is contained in:
Mike Fährmann
2019-06-30 21:27:28 +02:00
parent 6393b47db2
commit f7b5c4c3e7
5 changed files with 16 additions and 15 deletions

View File

@@ -300,7 +300,7 @@ extractor.*.retries
-------------------
=========== =====
Type ``integer``
Default ``5``
Default ``4``
Description Number of times a failed HTTP request is retried before giving up.
=========== =====

View File

@@ -154,7 +154,7 @@
{
"mtime": true,
"rate": null,
"retries": 5,
"retries": 4,
"timeout": 30.0,
"verify": true
},
@@ -164,7 +164,7 @@
"format": null,
"mtime": true,
"rate": null,
"retries": 5,
"retries": 4,
"timeout": 30.0,
"verify": true
}