Move the parallel URL fetching from firefox.py to endoflife.py to make available parallel URL fetching for all scripts.
Also a a fix found on https://stackoverflow.com/a/44511691/374236 to avoid ChunkedEncodingError.
The purpose of this new script is to be alerted of new runtimes, while not making updates to the original product file (because release dates cannot be fetched from AWS documentation).
The main reason for doing this is to have some common code between scripts, so that it is easier to change the JSON schema globally and normalize a few things (such as release order).
The Ruby code was kept as is so we can quickly roll back if necessary.
Fetch Looker versions with their dates from the Google Cloud release notes RSS feed: https://cloud.google.com/feeds/looker-release-notes.xml.
Only version above 23.0 are retrieved, as prior version were not announced in the Google Cloud release notes.
Using tags misses all ELTS releases, which are relevant for ELTS-only release cycles.
Important changes:
- very old v3 and v4 releases completely dropped except for 3.3.0, and 3.5.0.
- We track only as far back as v7, so dropping v4 and earlier should be fine.
- 7.0.1 and 9.3.2 are missing, which is okay because both are superseded.
Fixes#92.
Recently links to web.archive.org were added for various products. Such links are very long to load. This increases the default timeout value so that such links do not make the update fail.
GraalVM for JDK versions has to be prefixed as their release cycle collide with older GraalVM release cycles.
Example: GraalVM for JDK 20 and 20.0 (will be the same with JDK 21 and 22).
The script only get versions 4.x and above, see script comments for more informations.
Note that the code related to git has been extracted to a common script so that it can be reused for the Debian script.
AWS is no longer publishing the history on GitHub, so we
instead rely on older copies of the website to track the old
release dates. This is done at 2 cutoff points right now for
1.19 and 1.20.
Older releases are available on the web archive, but those
pages didn't include release dates so not helpful.
Create a common function to write resulting JSON files to the releases directory.
It makes this task simpler to read and maintain, while making it modifiable at a central point in the future.
One example of such modification could be the sorting of the versions in a uniform way for all the scripts.
This creates a common function to fetch HTTP URLs, with enhanced capabilities (retry, use of a known User-Agent).
It makes scripts that need those capabilities simpler, while improving other scripts.
This commit also fixes some scripts that did not log properly (cos.py, eks.py, haproxy.py, palo-alto-networks.py, rhel.py, ros.py, unrealircd.py).
Generic scripts are scripts that handle multiple product based on some identifier (URL, coordinates...).
This creates a common function to list and load product configurations for those scripts.
It makes them simpler to read and maintain, while making the way they work much more consistent.
