Skip to content

Add structured validation metadata to service records and enforce ESA-accepted constraints - #638

Open
JanssenBrm with Copilot wants to merge 4 commits into
mainfrom
copilot/extend-algorithm-record-schema
Open

Add structured validation metadata to service records and enforce ESA-accepted constraints#638
JanssenBrm with Copilot wants to merge 4 commits into
mainfrom
copilot/extend-algorithm-record-schema

Conversation

Copilot AI commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

This PR introduces machine-checkable validation/certification metadata in algorithm service records so catalogue category derivation can rely on structured record data instead of ad-hoc links/keywords. It also aligns static QA checks and onboarding docs with the new metadata contract.

  • Schema: add optional properties.validation contract

    • Extended schemas/record.json with an optional properties.validation object:
      • status: pending | accepted
      • document.href: required URI
      • document.reviewed_by (ESA) and document.review_date (ISO date) required when status=accepted
    • Encodes acceptance semantics directly in schema (if/then) to prevent ambiguous “accepted” records.
  • QA: enforce malformed metadata rejection + accepted-record benchmark prerequisite

    • Updated qa/unittests/tests/test_records.py to:
      • reject invalid validation payloads via schema validation tests
      • fail records marked validation.status=accepted when no benchmark_scenarios/*.json exists for that service
  • Documentation: define contributor/reviewer workflow

    • Updated docs/catalogue.md with:
      • properties.validation shape and examples
      • PI→ESA review transition (pendingaccepted)
      • explicit ESA-validated condition: accepted ESA review metadata and benchmark scenario presence
  • Record backfill: annotate known ESA-reviewed services

    • Added accepted validation metadata to:
      • algorithm_catalog/vito/worldcereal_crop_extent/records/worldcereal_crop_extent.json
      • algorithm_catalog/vito/worldcereal_crop_type/records/worldcereal_crop_type.json
"validation": {
  "status": "accepted",
  "document": {
    "href": "https://doi.org/10.1038/s43016-023-00841-7",
    "reviewed_by": "ESA",
    "review_date": "2025-02-17"
  }
}

Copilot AI changed the title [WIP] Extend algorithm record schema with validation/certification metadata Add structured validation metadata to service records and enforce ESA-accepted constraints Aug 3, 2026
Copilot AI requested a review from JanssenBrm August 3, 2026 13:44
@JanssenBrm

Copy link
Copy Markdown
Contributor

@copilot The validation status is not really relevant in this case. As it is either validated or not.

Copilot AI commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

@copilot The validation status is not really relevant in this case. As it is either validated or not.

Updated in 85b9eff: removed validation.status from schema, tests, docs, and records. Validation is now represented as binary metadata via validation.document.reviewed_by + review_date (with href), and benchmark prerequisite checks follow that ESA-reviewed metadata.

@JanssenBrm
JanssenBrm marked this pull request as ready for review August 3, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extend algorithm record schema with validation/certification metadata

2 participants