github/endoflife-date-release-data
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Improve HTTP handling in scripts (#449)

Browse Source 
Provide some helper methods to hide the complexity of parsing HTML, JSON, YAML, XML or Markdown.
This commit is contained in:
Marc Wrobel
2025-06-28 11:46:04 +02:00
parent fda4967c38
commit 312ce078bb
43 changed files with 103 additions and 137 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
src/distrowatch.py
View File

@@ -1,12 +1,10 @@
from bs4 import BeautifulSoup
from common import dates, endoflife, http, releasedata
for config in endoflife.list_configs_from_argv():
with releasedata.ProductData(config.product) as product_data:
response = http.fetch_url(f"https://distrowatch.com/index.php?distribution={config.url}")
soup = BeautifulSoup(response.text, features="html5lib")
html = http.fetch_html(f"https://distrowatch.com/index.php?distribution={config.url}")
for table in soup.select("td.News1>table.News"):
for table in html.select("td.News1>table.News"):
headline = table.select_one("td.NewsHeadline a[href]").get_text().strip()
versions_match = config.first_match(headline)
if not versions_match: