Commit Graph

126 Commits

Author SHA1 Message Date
Marc Wrobel
7bd6cc08fb Add disabled flag support for automation scripts (#553)
- Add 'disabled' field to AutoConfig class
- Skip disabled scripts in update-release-data.py unless force flag is used
2026-02-01 11:13:05 +01:00
Marc Wrobel
b9eafb3fd7 [release_table] Support non-headless mode (#552)
This may be useful for running scripts manually.
2026-02-01 11:10:15 +01:00
Marc Wrobel
a5d56e1705 Fix declare_version (#531)
Some scripts, such as declare.py, may provide dates and not datetimes.
This update make the declare_version work for both.
2025-11-15 09:20:47 +01:00
Marc Wrobel
75c08e7bd1 Prevent versions with future dates to be added to release data (#531)
When this happens, it breaks the build of the site.

Closes https://github.com/endoflife-date/endoflife.date/issues/8936.
2025-11-15 00:40:26 +01:00
Marc Wrobel
09aba7905b Handle dates with a misplaced comma
Such as in https://mxlinux.org/previous-releases/.
2025-11-11 12:39:09 +01:00
Marc Wrobel
1fb86114be [version_table] Improve date parsing (#527)
So far the script only allowed datetimes. Now full/partial dates are allowed too.
2025-11-08 13:18:14 +01:00
Marc Wrobel
9364fee53c Warn when releases have not been updated for a long time (#521)
Raise an alert during the daily auto-update about stale releases, e.g.:

- non-EOL releases with latest releases not updated in a year,
- non-EOL releases with a release date older than a year.

The threshold is configurable.
2025-09-20 09:43:39 +02:00
Marc Wrobel
d82426182c Stop auto-updating release dates (#518)
Often the actual release date does not exactly match with the first version date of for a given release.
Moreover the release date is simple to set : this is a one-time update that is not subject to change.
2025-09-14 13:54:23 +02:00
Marc Wrobel
4980f92f64 Handle dates with 'Sept' instead of 'Sep' globally (#516)
There are often failures due to the use of 'Sept' instead of 'Sep' in dates (for example https://github.com/endoflife-date/release-data/actions/runs/17707933881). Manage this globally so that it's not needed do it in each script / config.
2025-09-14 10:30:38 +02:00
Marc Wrobel
d190da8073 [discourse] Add auto method (#515)
Closes #505.
2025-09-11 12:21:20 +02:00
Marc Wrobel
20b0c8d1ff [version_table] Add auto method (#514)
Similar to release_table, but for versions.
2025-09-10 18:12:11 +02:00
Marc Wrobel
c43a24fb19 Improve string to identifier transformation (#512)
Include all spaces characters (matching \s in a regex), not just the standard space character.
2025-09-08 17:45:45 +02:00
Marc Wrobel
c7faf9545f [release_table] Add remove_if_undefined parameter (#494)
remove_if_undefined allows to exclude unreleased release more easily.
2025-08-12 22:39:39 +02:00
Nemo
44bee91053 [git] Add a user-agent to git clone (#487)
Following #474, also use the endoflife.date user agent when cloning git repositories.
2025-08-06 23:42:13 +02:00
Marc Wrobel
1735954764 [release-table] Implement render_js_click_selector parameter (#485)
This parameter allows clicking on an element of the page after it was loaded when using `render_javascript`.
2025-08-03 21:43:16 +02:00
Marc Wrobel
72b9be8955 Normalize release names (#483)
Automatically normalize release names. This may be useful in a few situations to avoid having to write a custom script.
2025-08-02 15:52:03 +02:00
Marc Wrobel
3f65ff9d9e [update-product-data] Allow disabling auto configuration (#482) 2025-08-02 15:01:47 +02:00
Marc Wrobel
7368e073dc [amazon-neptune] Fix date parsing 2025-07-27 23:34:49 +02:00
Marc Wrobel
7dfcfb2903 [citrix-vad-rss] Add auto method 2025-07-27 19:25:09 +02:00
Marc Wrobel
3b8adee264 [nvidia-driver] Add automation script (#476)
Closes #368.
2025-07-27 18:29:58 +02:00
Marc Wrobel
e1e93dc841 [motorola-security] Add auto method (#475)
Based on https://gist.github.com/ab/67c6343530c6b8e599f07cedccbcba3d.

Closes #277.
2025-07-27 12:23:20 +02:00
Marc Wrobel
c4fa5b187c [http] Improve User-Agent handling (#474)
- Simplify overriding the User-Agent by replacing the generic (unused) `headers` parameter by a new `user_agent` parameters.
- Take into account the User-Agent even in `fetch_javascript_url` : Playwright user agent is `Playwright/1.12.0` by default anyway.
- Make it possible to override the User-Agent with the `release_table` method.
- Update splunk to use Firefox's User-Agent.

This follows #470.
2025-07-26 10:24:54 +02:00
Maxim Ivanov
68c69aa3d1 [common http] improve the User-Agent identification (#470)
Properly identifying endoflife.date bot is more fair for website administrators, and give them the ability to more easily authorize the bot if they want to.

---------

Co-authored-by: Marc Wrobel <marc.wrobel@gmail.com>
2025-07-26 09:45:54 +02:00
Marc Wrobel
a985e6218f [chrome-releases] Add auto method (#469)
Based on https://chromiumdash.appspot.com API.
2025-07-15 23:37:16 +02:00
Marc Wrobel
54ff914f34 [http] Improve fetch_javascript_url error handling (#468) 2025-07-13 13:30:52 +02:00
Marc Wrobel
081d884abe Remove invalid releases automatically (#466)
Automatically remove empty releases or releases which are released in the future, regardless of the product.

Also refactored a bit releasedata.py to use better names and clarify types.
2025-07-13 11:36:13 +02:00
Marc Wrobel
ec6217b3cf [http] Improve caching management 2025-07-13 10:59:35 +02:00
Marc Wrobel
42c1189781 Copy releases from products frontmatter (#465)
Copy releases, without their properties, from product data (frontmatter) to release data.

This script is not intended to be declared in the frontmatter: it is for internal use only.
It executes before all other scripts, and helps the following scripts to work with releases.
2025-07-12 16:47:36 +02:00
Marc Wrobel
bccac7033d Fix github groups
github groups were written using logging, which was added the date as a prefix.
This were breaking the grouping of logs on GHA.
2025-07-12 11:44:08 +02:00
Marc Wrobel
e62676b5c4 [google-kubernetes-engine] Normalize script (#464)
The script used to parse different channel for historical (forgotten) reasons. This is the only script that works that way, and release data is not supposed to be consumed by other clients than endoflife.date. So aligning this script for all the other scripts.

In case other channel data is useful for someone, it would be preferable to create multiple products so that the data is available through our official API.
2025-07-12 10:29:33 +02:00
Marc Wrobel
1f7a3772d6 [http] Improve fetch_javascript_url (#462)
Replace `click_selector` by `wait_for,` which is a selector that we must wait for before considering the page loaded.

Also added `select_wait_for`, which returns the waited for element. Oddly this may be needed in some case (such as `artifactory.py`) where the `page.content()` does not contain the waited for element.
2025-07-09 23:20:39 +02:00
Marc Wrobel
08808282dd Add new declare method (#461)
For manually declare (or overwrite) missing or erroneous versions.
2025-07-08 00:16:08 +02:00
Marc Wrobel
391d65ad8a Simplify argument parsing (#459)
With the current state of automation scripts, this is not possible anymore to launch script with multiple auto configs.
2025-07-06 22:42:01 +02:00
Marc Wrobel
c78d1fe2b5 Refactor scripts arguments handling (#456)
- remove the use of environment variables to get directory paths,
- make use of arguments / argparse instead of environment variables in `update.py` and `report.py`,
- automatically guess the data directory in `latest.py` based on the script's location,
- propagate log level to auto scripts,
- move `list_configs_from_argv` from `endoflife` module to `releasedata` module,
- use `list_products` in `latest.py` to load the product's frontmatters.
2025-06-28 19:05:07 +02:00
Marc Wrobel
3beb0f6e34 Respect HTTP Cache-Control header if available (#450)
Allow stale responses in case of error in order to reduce the number of temporary errors.

Note that the cache will be reset on the first build of every week in any cases, see .github/workflows/update.yml.
2025-06-28 12:57:41 +02:00
Marc Wrobel
312ce078bb Improve HTTP handling in scripts (#449)
Provide some helper methods to hide the complexity of parsing HTML, JSON, YAML, XML or Markdown.
2025-06-28 12:26:10 +02:00
Marc Wrobel
f404274310 Align custom scripts with generic scripts (#445)
Align custom scripts with generic scripts, making them configurable. This has a few advantages:

- script code is more unified,
- no more hard-coded method names in scripts, which is less error prone and make it easier to rename scripts,
- no more hard coded product names in scripts, which is less error prone and make it easier to rename products,
- less hard-coded URLs and regexes in scripts, which makes auto-configuration more expressive / updatable,

Also added method `endoflife.list_configs_from_argv()` so that it is easier to manipulate scripts arguments.
2025-06-07 17:25:05 +02:00
Marc Wrobel
c2a22dd150 [samsung-security] Generify samsung-mobile method (#439)
- Make section and inclusions/exclusions configurable,
- Remove Galaxy Tabs from samsung-mobile data (following https://github.com/endoflife-date/endoflife.date/pull/7513),
- Initialize Galaxy Tabs dataset (following https://github.com/endoflife-date/endoflife.date/pull/7513).
2025-05-19 19:06:45 +02:00
Marc Wrobel
aa43bf6602 [virtualbox] Add automation (#438)
Using https://www.virtualbox.org/wiki/Download_Old_Builds.
2025-05-17 11:55:38 +02:00
Marc Wrobel
6091ef55fe [samsung-mobile] Add automation (#437)
Detect new models and aggregate EOL data for Samsung Mobile devices.

This script works cumulatively: when a model is not listed anymore on https://security.samsungmobile.com/workScope.smsb
it retains the date and use it as the model's EOL date.
2025-05-07 23:29:33 +02:00
Denis Fuenzalida
54ef082add [red-hat-jboss-eap] Automation version retrieval (#414)
red-hat-jboss-eap-7.py retrieves all the 7.x versions based on https://access.redhat.com/articles/2332721.

red-hat-jboss-eap-8.py retrieves only the last 8.x version based on https://maven.repository.redhat.com/ga/org/jboss/eap/channels/eap-8.0/maven-metadata.xml (the version date is approximated using the lastUpdated tag).

---------

Co-authored-by: Denis Fuenzalida 🐙 <defuenza@microsoft.com>
Co-authored-by: Marc Wrobel <marc.wrobel@gmail.com>
2025-05-02 17:31:12 +02:00
Marc Wrobel
e56b642326 Add github_tags auto method (#427)
Add a new method to fetch tags with their related commit date using GitHub GraphQL API.

For large repositories, such as https://github.com/gregkh/linux, it may be more performant than the git auto method.
2025-04-05 16:40:29 +02:00
Marc Wrobel
c79f871bc4 [eks] Update script to accept month-year dates (#408)
Some were spotted in the latest runs, see https://github.com/endoflife-date/release-data/actions/runs/13215761680.
2025-02-08 17:34:13 +01:00
Marc Wrobel
59b6e6aa7c [pan-os] Add automation for versions retrieval (#371)
Based on https://github.com/mrjcap/panos-versions/.

Closes #370.
2024-08-15 12:19:54 +02:00
Marc Wrobel
f238224681 [chef-inspec] Use GitHub releases instead of git tags (#367)
GitHub releases gives better dates than git tags.
2024-08-13 22:20:36 +02:00
Romain
cb0aa83225 [subversion] Add automation (#360)
Will be used by https://github.com/endoflife-date/endoflife.date/pull/5567.

---------

Co-authored-by: Marc Wrobel <marc.wrobel@gmail.com>
2024-08-04 19:30:09 +02:00
Marc Wrobel
0ade677365 Do not fail in update.py if a product cannot be loaded (#357)
Such errors prevent the script to run.
2024-07-19 22:58:10 +02:00
Nemo
17a5b34816 [libreoffice] Add automation (#354) 2024-06-23 13:01:20 +02:00
Marc Wrobel
08d4ea469e Do not use wait_until='networkidle' by default with fetch_javascript_url (#340)
It does not work in all situation, especially with GitHub-rendered markdown files.
2024-04-02 23:17:51 +02:00
Marc Wrobel
812565e977 Rename support fields to align with eol fields (#338)
Companion PR of https://github.com/endoflife-date/endoflife.date/pull/4931.
2024-04-02 21:41:18 +02:00