The script only get versions 4.x and above, see script comments for more informations.
Note that the code related to git has been extracted to a common script so that it can be reused for the Debian script.
AWS is no longer publishing the history on GitHub, so we
instead rely on older copies of the website to track the old
release dates. This is done at 2 cutoff points right now for
1.19 and 1.20.
Older releases are available on the web archive, but those
pages didn't include release dates so not helpful.
Create a common function to write resulting JSON files to the releases directory.
It makes this task simpler to read and maintain, while making it modifiable at a central point in the future.
One example of such modification could be the sorting of the versions in a uniform way for all the scripts.
This creates a common function to fetch HTTP URLs, with enhanced capabilities (retry, use of a known User-Agent).
It makes scripts that need those capabilities simpler, while improving other scripts.
This commit also fixes some scripts that did not log properly (cos.py, eks.py, haproxy.py, palo-alto-networks.py, rhel.py, ros.py, unrealircd.py).
Generic scripts are scripts that handle multiple product based on some identifier (URL, coordinates...).
This creates a common function to list and load product configurations for those scripts.
It makes them simpler to read and maintain, while making the way they work much more consistent.
This uses the PHP releases API. Limitations:
1. New Major versions will needed to be added manually
The src for the API is at https://github.com/php/web-php/blob/master/releases/index.php
The data comes from https://github.com/php/web-php/blob/master/include/releases.inc
This commit keeps the old sort method for a cleaner diff. Next commits
in this PR will fix the sorting to be standardized.
The difference from using tags is:
1. One additional version shows up (4.0.4) which we'd missed previously
due to being tagged as php-4.0.4REL.
2. Slightly more accurate dates.
This script is using the GitHub CLI with the GraphQL API in order to retrieve releases. The reasons are:
- using 'gh release list' does not return all the releases
- using the API, directly or via GitHub CLI, is slow, produces a lot of 502 errors, and is harder due to pagination.
- using the GraphQL API directly is hard due to pagination.
- using a library, such as graphql-python, is sightly harder than using the GitHub CLI (and still requires a GITHUB_TOKEN).
Access to the GitHub token is required when using GitHub CLI, hence the update.yml workflow update.
Only 18.0.20.3 and later are picked up :
- format of the title for 18.0.20 and 18.0.19 are different,
- there is not entry for GA of versions 18.0.18 and older.
Co-authored-by: Marc Wrobel <marc.wrobel@gmail.com>
- Get cycles URL by parsing https://www.haproxy.org/download/:
- This way the script does not have to be updated each time a new minor version is released.
- The old algorithm was missing some versions (such as 1.6.x).
- Use a GitHub actions log group to wrap the script logs.
- Display version like the other scripts.
Fetch Java versions with their dates from https://www.java.com/releases/.
This script is using a new dependency: requests-html (https://requests-html.kennethreitz.org/). This is because https://www.java.com/releases/ needs JavaScript to render correctly. And for this to work requests-html automatically downloads Chromium because of its dependency to pyppeteer.
Chromium is downloaded into PYPPETEER_HOME, so the workflow has been modified to set this directory as a subdirectory of ~/.cache. Note that declaring the same environment variable from the python script does not work.
There were two issues with the macOS regexes:
- The regexex accepted only spaces separator. But non-breakable spaces are also used from time to time. Replacing spaces by \s fixed the issue (other product regexes were also updated - never know).
- The first regex, the one that accepted major releases with no minor version, were not useful (covered by the second regex) and partly catched valid versions (such as 13.1), who was then ignored. I removed it.
1. Drop RC and milestone releases by adopting a simple
version regex. Drops a few non-stable releases from
jreleaser and tomcat. Tested against all of the maven
automated pages
2. Fix pages with multiple auto: maven entries. In particular,
groovy was using this, but the newer 4.0 releases were not
being written correctly to disk.
