Skip to content

Add Astarte Operator compatibility matrix#3840

Open
KUANPEIEN wants to merge 2 commits into
pluralsh:masterfrom
KUANPEIEN:add-astarte-operator-compatibility
Open

Add Astarte Operator compatibility matrix#3840
KUANPEIEN wants to merge 2 commits into
pluralsh:masterfrom
KUANPEIEN:add-astarte-operator-compatibility

Conversation

@KUANPEIEN

Copy link
Copy Markdown
Contributor

Summary

Adds Astarte Kubernetes Operator to the compatibility matrix coverage.

  • Adds static/compatibilities/astarte-operator.yaml
  • Adds a scraper at utils/compatibility/scrapers/astarte-operator.py
  • Registers astarte-operator in static/compatibilities/manifest.yaml

Sources

Test Plan

  • python -m py_compile utils/compatibility/scrapers/astarte-operator.py
  • Parsed static/compatibilities/astarte-operator.yaml and static/compatibilities/manifest.yaml with yaml.safe_load
  • Dry-ran the scraper with update_compatibility_info stubbed to verify it extracts versions and chart versions from the official docs and Helm index
  • git diff --check

I did not run the full compatibility update locally because this machine does not have helm installed, and update_compatibility_info shells out to helm template to collect chart images.

Checklist

  • I have added a meaningful title and summary to convey the impact of this PR to a user.
  • If required, I have updated the Plural documentation accordingly.
  • I have added tests to cover my changes.
  • I have deployed the agent to a test environment and verified that it works as expected (required only when changing agent code).

This contribution follows the compatibility bounty guidance in the repository README for a new application plus scraper.

@soffi-ai

soffi-ai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Soffi AI Summary

This PR extends the platform's Kubernetes add-on compatibility matrix coverage by adding the Astarte Kubernetes Operator as a tracked application. The motivation is to allow Console to surface Kubernetes version compatibility information for Astarte Operator installations, consistent with how it tracks other operators and add-ons in static/compatibilities/.

Three artifacts are introduced:

  1. static/compatibilities/astarte-operator.yaml — The hand-seeded compatibility data file listing 5 Astarte Operator releases (22.11.1 through 26.5.1) with their supported Kubernetes version ranges, corresponding Helm chart versions, and Astarte platform version requirements. This file serves as the authoritative static data consumed by the Console server at runtime.

  2. utils/compatibility/scrapers/astarte-operator.py — A new Python scraper that automates future updates to the above YAML. It fetches and parses the official Astarte Operator HTML compatibility table (from the Astarte docs site) to extract operator-to-Kubernetes version mappings, and cross-references the official Helm index (helm.astarte-platform.org) to resolve the latest chart version per operator release series. Version resolution uses a two-pass strategy: prefer an exact operator version match from the docs table, falling back to a minor-series match. The scraper integrates with the existing update_compatibility_info utility used by all other scrapers in the codebase.

  3. static/compatibilities/manifest.yamlastarte-operator is registered here so the Console server discovers and loads the new compatibility file.

The second commit tightened the scraper implementation based on review feedback (improved version normalization, series-specificity logic, and error handling).

Commits

Commit Summary
c73a923 Introduces the Astarte Operator compatibility matrix: adds the static YAML data file with 5 versioned entries (Kubernetes ranges, chart versions, Astarte requirements), registers astarte-operator in manifest.yaml, and implements the initial Python scraper that parses the official Astarte docs HTML table and cross-references the Helm index to automate future updates.
8b8729b Refines the scraper based on review feedback: improves version normalization logic, adds series-specificity tracking to prefer minor-series doc table matches over more-specific entries, and hardens error handling throughout the parsing pipeline.

Deploy in Soffi


Updated: 2026-07-13 14:16 UTC

@greptile-apps

greptile-apps Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds Astarte Kubernetes Operator to the compatibility matrix, contributing a static YAML file, a scraper, and a manifest entry.

  • static/compatibilities/astarte-operator.yaml: Five operator versions (22.11.1 \u2013 26.5.1) with Kubernetes and Astarte version compatibility data; chart_version: 22.11.01 has a leading zero that will be silently normalised on the next scraper run.
  • utils/compatibility/scrapers/astarte-operator.py: Scraper that fetches and parses the official docs HTML table and the Helm index. _find_compatibility_table validates only 2 of the 3 required column headers ("Astarte Version" is not checked), so _parse_compatibility_rows will throw ValueError if the column is absent; separately, _parse_min_kube passes the return of current_kube_version() directly to expand_kube_versions without guarding against a None return when the KUBE_VERSION file is missing.
  • static/compatibilities/manifest.yaml: astarte-operator is registered as the first entry \u2014 straightforward and correct.

Confidence Score: 3/5

The static YAML and manifest registration are safe to merge; the scraper has two bugs that would cause it to crash rather than produce useful output.

The scraper will throw an uncaught ValueError whenever the docs page contains a table that matches the two validated headers but lacks the Astarte Version column. A separate crash path exists when the KUBE_VERSION file is absent and current_kube_version() returns None, which is passed straight into expand_kube_versions. Both failures happen silently without a clean error message.

utils/compatibility/scrapers/astarte-operator.py needs attention for the two crash paths; the static YAML and manifest are fine.

Important Files Changed

Filename Overview
utils/compatibility/scrapers/astarte-operator.py New scraper for Astarte Operator: _find_compatibility_table validates only 2 of the 3 required column headers, so _parse_compatibility_rows will throw ValueError if "Astarte Version" is absent; also passes potentially-None return from current_kube_version() to expand_kube_versions.
static/compatibilities/astarte-operator.yaml New compatibility matrix for Astarte Operator with 5 version entries; chart_version 22.11.01 has a leading zero in the patch field.
static/compatibilities/manifest.yaml Adds astarte-operator as the first entry in the names list; registration looks correct.

Reviews (1): Last reviewed commit: "Add Astarte Operator compatibility matri..." | Re-trigger Greptile

Comment on lines +89 to +93
for table in soup.find_all("table"):
headers = [th.get_text(" ", strip=True) for th in table.find_all("th")]
if "Astarte Operator Version" in headers and "Kubernetes Version" in headers:
return table, headers
return None, []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 _find_compatibility_table only validates that "Astarte Operator Version" and "Kubernetes Version" appear in the headers, but _parse_compatibility_rows unconditionally calls headers.index("Astarte Version"). If the docs page ever renders a table that matches the first two columns but lacks the "Astarte Version" column (renamed, merged, or split), the scraper crashes with an uncaught ValueError instead of reporting a clean error and exiting.

Suggested change
for table in soup.find_all("table"):
headers = [th.get_text(" ", strip=True) for th in table.find_all("th")]
if "Astarte Operator Version" in headers and "Kubernetes Version" in headers:
return table, headers
return None, []
for table in soup.find_all("table"):
headers = [th.get_text(" ", strip=True) for th in table.find_all("th")]
if (
"Astarte Operator Version" in headers
and "Astarte Version" in headers
and "Kubernetes Version" in headers
):
return table, headers
return None, []

Comment on lines +54 to +58
def _parse_min_kube(value: str) -> list[str]:
match = re.search(r"v?(\d+\.\d+)\+", value)
if not match:
return []
return expand_kube_versions(match.group(1), current_kube_version())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 current_kube_version() returns None when the KUBE_VERSION file is missing (e.g., on a fresh checkout). Passing None as the second argument to expand_kube_versions causes NoneType.split(".") to raise AttributeError, crashing every row parse silently before any useful error message is printed.

Suggested change
def _parse_min_kube(value: str) -> list[str]:
match = re.search(r"v?(\d+\.\d+)\+", value)
if not match:
return []
return expand_kube_versions(match.group(1), current_kube_version())
def _parse_min_kube(value: str) -> list[str]:
match = re.search(r"v?(\d+\.\d+)\+", value)
if not match:
return []
end = current_kube_version()
if not end:
print_error("KUBE_VERSION not available; cannot expand kube versions.")
return []
return expand_kube_versions(match.group(1), end)

- version: 22.11.1
kube: ['1.35', '1.34', '1.33', '1.32', '1.31', '1.30', '1.29', '1.28', '1.27', '1.26',
'1.25', '1.24', '1.23', '1.22']
chart_version: 22.11.01

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The chart_version for 22.11.1 is 22.11.01 — a non-standard semver with a leading zero in the patch field. While packaging.version normalises this to 22.11.1 when sorting, the scraper passes the raw string to helm template --version 22.11.01, and Helm also accepts it. However, sort_versions in utils.py calls Version(v["chart_version"]) which silently normalises the string, meaning the next scraper run will write back 22.11.1, losing the original 22.11.01. Normalising it now avoids the silent drift.

Suggested change
chart_version: 22.11.01
chart_version: 22.11.1

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@michaeljguarino

Copy link
Copy Markdown
Member

This seems like a cool project, but I don't think it has enough realistic usage to justify adding to our common set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants