Add Copilot instructions for repository
This commit is contained in:
144
.github/copilot-instructions.md
vendored
Normal file
144
.github/copilot-instructions.md
vendored
Normal file
@@ -0,0 +1,144 @@
|
|||||||
|
# Copilot Instructions for release-data
|
||||||
|
|
||||||
|
This repository provides release data for the [endoflife.date](https://endoflife.date) website. Scripts scrape release information from various sources and output structured JSON files.
|
||||||
|
|
||||||
|
## Build, Test, and Lint
|
||||||
|
|
||||||
|
**Install dependencies:**
|
||||||
|
```bash
|
||||||
|
pip install -r requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
**Lint code:**
|
||||||
|
```bash
|
||||||
|
pre-commit run --all-files
|
||||||
|
```
|
||||||
|
|
||||||
|
**Update all products:**
|
||||||
|
```bash
|
||||||
|
python update-release-data.py -v -p '../endoflife.date/products'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Update a single product:**
|
||||||
|
```bash
|
||||||
|
python update-release-data.py -v -p '../endoflife.date/products' python
|
||||||
|
```
|
||||||
|
|
||||||
|
**Generate automation report:**
|
||||||
|
```bash
|
||||||
|
python report.py -p '../endoflife.date/products'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Data Flow
|
||||||
|
|
||||||
|
1. **Product definitions** live in the [endoflife.date repository](https://github.com/endoflife-date/endoflife.date) (`products/*.md` files with YAML frontmatter)
|
||||||
|
2. **Scraper scripts** (`src/*.py`) fetch release data from various sources (Git tags, GitHub releases, APIs, HTML scraping)
|
||||||
|
3. **Output** goes to `releases/*.json` files with versioned release dates
|
||||||
|
4. **GitHub Actions** runs updates 4x daily and commits changes automatically
|
||||||
|
|
||||||
|
### Core Components
|
||||||
|
|
||||||
|
**`src/common/`** - Shared utilities:
|
||||||
|
- `endoflife.py`: Parses product frontmatter, handles `auto` config (regex patterns, version templates)
|
||||||
|
- `releasedata.py`: Manages `releases/*.json` files via `ProductData` context manager
|
||||||
|
- `dates.py`: Date/time utilities
|
||||||
|
- `git.py`, `github.py`, `http.py`: Source-specific helpers
|
||||||
|
|
||||||
|
**Scraper scripts** (`src/*.py`):
|
||||||
|
- Each script is standalone and rebuilds data from scratch (never relies on existing JSON)
|
||||||
|
- Uses `ProductData` context manager that auto-saves on success or raises `ProductUpdateError` on failure
|
||||||
|
- Filters versions via `config.first_match(version_str)` and renders via `config.render(match)`
|
||||||
|
|
||||||
|
**Main orchestrator** (`update-release-data.py`):
|
||||||
|
- Reads product frontmatter from website repo
|
||||||
|
- Executes relevant scripts for each product
|
||||||
|
- Tracks execution times and failures
|
||||||
|
- Outputs GitHub Actions summary
|
||||||
|
|
||||||
|
## Key Conventions
|
||||||
|
|
||||||
|
### Product Frontmatter Auto Config
|
||||||
|
|
||||||
|
Products define automation in YAML frontmatter:
|
||||||
|
```yaml
|
||||||
|
auto:
|
||||||
|
methods:
|
||||||
|
- git: https://github.com/example/repo
|
||||||
|
regex: '^v?(?P<major>[0-9]+)\.(?P<minor>[0-9]+)\.(?P<patch>[0-9]+)$'
|
||||||
|
template: '{{major}}.{{minor}}.{{patch}}'
|
||||||
|
```
|
||||||
|
|
||||||
|
- **method**: Determines which `src/{method}.py` script runs
|
||||||
|
- **regex**: Filters version strings (can be list for multiple patterns)
|
||||||
|
- **regex_exclude**: Excludes matched versions
|
||||||
|
- **template**: Liquid template to render version names from regex groups
|
||||||
|
|
||||||
|
### Script Patterns
|
||||||
|
|
||||||
|
All scripts follow this structure:
|
||||||
|
```python
|
||||||
|
from common.releasedata import ProductData, config_from_argv
|
||||||
|
|
||||||
|
config = config_from_argv()
|
||||||
|
with ProductData(config.product) as product_data:
|
||||||
|
for version_str in fetch_versions_from_source():
|
||||||
|
match = config.first_match(version_str)
|
||||||
|
if match:
|
||||||
|
version = config.render(match)
|
||||||
|
product_data.declare_version(version, release_date)
|
||||||
|
```
|
||||||
|
|
||||||
|
The `ProductData` context manager:
|
||||||
|
- Automatically loads existing data on `__enter__`
|
||||||
|
- Validates updates occurred (raises error if `updated=False`)
|
||||||
|
- Saves sorted JSON on `__exit__`
|
||||||
|
- Handles errors by reverting changes
|
||||||
|
|
||||||
|
### Version Handling
|
||||||
|
|
||||||
|
- **Stable only**: No beta, RC, nightly versions
|
||||||
|
- **Dates**: YYYY-MM-DD format, preferably in release's timezone
|
||||||
|
- **Sorting**: Versions sorted by (date, name) descending; releases sorted by name descending
|
||||||
|
- **Identifiers**: Use `endoflife.to_identifier()` for consistent naming
|
||||||
|
|
||||||
|
### JSON Output Format
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"releases": {
|
||||||
|
"1.2": {
|
||||||
|
"name": "1.2",
|
||||||
|
"releaseDate": "2024-01-15",
|
||||||
|
"releaseLabel": "optional-label",
|
||||||
|
"eol": "2025-01-15",
|
||||||
|
"eoas": "2024-07-15",
|
||||||
|
"eoes": "2024-10-15"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"versions": {
|
||||||
|
"1.2.3": {
|
||||||
|
"name": "1.2.3",
|
||||||
|
"date": "2024-01-15"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- **releases**: Lifecycle data (EOL dates) - coarser-grained
|
||||||
|
- **versions**: Point release dates - finer-grained
|
||||||
|
|
||||||
|
### Cumulative Updates
|
||||||
|
|
||||||
|
If `auto.cumulative: true` in frontmatter:
|
||||||
|
- Script should **not** delete existing data before running
|
||||||
|
- New data is merged with existing data
|
||||||
|
- Used when multiple scripts contribute to the same product
|
||||||
|
|
||||||
|
### Guiding Principles
|
||||||
|
|
||||||
|
1. Scripts must be standalone and simple
|
||||||
|
2. Never rely on existing data - rebuild from scratch each run
|
||||||
|
3. Code should handle upstream changes gracefully
|
||||||
|
4. Everything runs on GitHub Actions (no local state dependencies)
|
||||||
Reference in New Issue
Block a user