github/endoflife-date-release-data
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[http] Improve fetch_javascript_url (#462)

Browse Source 
Replace `click_selector` by `wait_for,` which is a selector that we must wait for before considering the page loaded.

Also added `select_wait_for`, which returns the waited for element. Oddly this may be needed in some case (such as `artifactory.py`) where the `page.content()` does not contain the waited for element.
This commit is contained in:
Marc Wrobel
2025-07-09 22:59:37 +02:00
parent 9d63ed9252
commit 1f7a3772d6
3 changed files with 20 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
src/artifactory.py
View File

@@ -7,7 +7,8 @@ needed to render the page."""
config = config_from_argv()
with ProductData(config.product) as product_data:
content = http.fetch_javascript_url(config.url, wait_until = 'networkidle')
# Oddly the full page content does not contain the versions, must use the wait_for element directly.
content = http.fetch_javascript_url(config.url, wait_for='div.informaltable', select_wait_for=True)
soup = BeautifulSoup(content, 'html.parser')
for row in soup.select('.informaltable tbody tr'):