Migrate Unity scraper to use Editor Release API (#558)

Switch from HTML scraping to the official Unity Editor Release API.

- Fetch versions from paginated API endpoint
- Filter out ALPHA and BETA stream releases
- Process all available versions (not just first page)

Closes https://github.com/endoflife-date/endoflife.date/issues/9418.
This commit is contained in:
Marc Wrobel
2026-02-15 20:56:07 +01:00
committed by GitHub
parent 673837f8f7
commit d9604aa673

View File

@@ -1,27 +1,37 @@
from common import dates, http
from common.releasedata import ProductData, config_from_argv
"""Fetches the Unity LTS releases from the Unity website. Non-LTS releases are not listed there, so this automation
is only partial.
"""Fetches Unity versions from the Unity Editor Release API.
This script is cumulative, only the first page is fetched (e.g. the first ten versions). This is because:
- it is too long to fetch all (at least 30s, usually more than a minute),
- this generates too many requests to the unity.com servers,
- fetching multiple pages in parallel is raising a lot of errors and makes the overall process slower (this was tested
during https://github.com/endoflife-date/release-data/pull/194),
- and anyway oldest versions are never updated.
Note that it was assumed that:
- the script is ran regularly enough to keep the versions up to date (once a day or week looks enough),
- there is never more than 10 new LTS versions at a time.
The script will need to be updated if someday those conditions are not met."""
This script fetches stable releases from the Unity API, filtering out alpha, beta, and other pre-release versions.
The API provides paginated results with all Unity versions across different streams (TECH, LTS, BETA, ALPHA).
"""
config = config_from_argv()
with ProductData(config.product) as product_data:
html = http.fetch_html(config.url)
offset = 0
limit = 25
for release in html.find_all('div', class_='component-releases-item__show__inner-header'):
version = release.find('h4').find('span').text
date = dates.parse_datetime(release.find('time').attrs['datetime'])
while True:
url = f"{config.url}?limit={limit}&offset={offset}"
data = http.fetch_json(url)
if 'results' not in data:
break
for release in data['results']:
version = release['version']
# Skip pre-release versions (ALPHA, BETA, etc.)
stream = release.get('stream', '')
if stream in ('ALPHA', 'BETA'):
continue
date = dates.parse_datetime(release['releaseDate'])
product_data.declare_version(version, date)
# Check if we've reached the end
total = data.get('total', 0)
offset += limit
if offset >= total:
break