[AIAA] Add deterministic data-collection scripts

[nate-double-u]

Adds scripts/assessment/: the deterministic inventory scripts behind
HC-5. Build step 5 (spec section 16). Contributes to #374.

What it does

• collect.mjs: file inventory (paths, sizes, sha256 hashes, totals),
  markdown link extraction (inline, reference, autolink; classified
  external, internal, anchor), optional dated site fetch and external
  link status. Data outputs are sorted, stable JSON with no
  timestamps, so identical inputs give identical bytes.
• Content-hash manifest plus collect.mjs verify: every output
  hashed, alongside the command line and pinned sources (repository
  SHAs, site retrieval dates). An edit after collection breaks
  verification (P-4, section 13); the provenance block's Data field
  reads from it (section 15).
• snapshot-agent-config.mjs: captures the agent's effective
  configuration (MCP servers, firewall state, custom allowlist) from
  the documented public-preview endpoint. On failure it writes an
  explicit error record as evidence instead of hiding it (section 13).
• assessment-test.yml workflow: runs npm run test:assessment on
  pull requests touching scripts/assessment/** or the workflow
  itself, making the tests a CI gate. This PR triggers it.

Verification

• 26 tests (npm run test:assessment, node:test): extraction shapes,
  manifest tamper and missing-file detection, injected-fetch site and
  snapshot behavior, and a CLI twice-run byte-identity check.
• Real-project demo: a clean clone of cncf/techdocs collected twice
  produced byte-identical trees, verify passed, and the manifest
  pinned the HEAD SHA.
• The workflow's ASSESSMENT tests check on this PR is the same suite
  running in CI.
• npm run check passes.

Open platform checks

Parameterized around two section 17 build-time checks, answers to be
recorded on #374: how the target project reaches the setup steps that
run collection (the CLI takes explicit --repo and --site; no
setup-workflow wiring in this PR), and what the configuration endpoint
returns from inside a session plus the credential it needs.

Done-when clause one (twice-run identity) is shown above. Clause two
(a delegated session's draft PR containing outputs its setup steps
produced) is the live check tracked on #374.

[netlify]

✅ Deploy Preview for cncf-techdocs ready!

Name	Link
🔨 Latest commit	15bfa50
🔍 Latest deploy log	https://app.netlify.com/projects/cncf-techdocs/deploys/6a7f948858fa18000801388e
😎 Deploy Preview	https://deploy-preview-382--cncf-techdocs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[Copilot]

Pull request overview

Adds deterministic assessment data collection for HC-5 and issue #374.

Changes:

• Adds repository, link, site, and agent-configuration collectors.
• Adds hash manifests and verification.
• Adds tests and a dedicated CI workflow.

Reviewed changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
.github/workflows/assessment-test.yml	Runs assessment tests in CI.
.prettierignore	Excludes frozen fixtures.
package.json	Adds assessment test command.
scripts/assessment/collect.mjs	Implements collection CLI.
scripts/assessment/snapshot-agent-config.mjs	Implements configuration snapshot CLI.
scripts/assessment/lib/git.mjs	Resolves repository HEAD SHA.
scripts/assessment/lib/inventory.mjs	Builds file inventories.
scripts/assessment/lib/links.mjs	Extracts and classifies links.
scripts/assessment/lib/manifest.mjs	Writes and verifies manifests.
scripts/assessment/lib/sitefetch.mjs	Fetches sites and checks links.
scripts/assessment/lib/snapshot.mjs	Calls the cloud-agent API.
scripts/assessment/lib/util.mjs	Provides hashing and stable JSON.
scripts/assessment/test/collect.e2e.test.mjs	Tests CLI determinism and verification.
scripts/assessment/test/fixtures/sample-project/README.md	Provides Markdown fixture content.
scripts/assessment/test/fixtures/sample-project/docs/guide.md	Provides reference-link fixtures.
scripts/assessment/test/fixtures/sample-project/assets/sample.png	Provides binary fixture content.
scripts/assessment/test/git.test.mjs	Tests SHA resolution.
scripts/assessment/test/inventory.test.mjs	Tests inventory generation.
scripts/assessment/test/links.test.mjs	Tests link extraction.
scripts/assessment/test/manifest.test.mjs	Tests manifest verification.
scripts/assessment/test/sitefetch.test.mjs	Tests site and link requests.
scripts/assessment/test/snapshot.test.mjs	Tests configuration snapshots.
scripts/assessment/test/util.test.mjs	Tests utility functions.

Suppressed comments (1)

scripts/assessment/lib/inventory.mjs:18

• In a Git worktree or submodule checkout, .git is a regular file rather than a directory, so it reaches this branch and gets inventoried. Its gitdir: content commonly contains an environment-specific absolute path, breaking deterministic output and exposing checkout metadata. Skip .git by name regardless of entry type.

    } else if (entry.isFile()) {
      paths.push(relPath);

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 24 changed files in this pull request and generated no new comments.

Suppressed comments (9)

scripts/assessment/lib/manifest.mjs:28

• Reusing an output directory after changing collection options produces a collection that immediately fails verification: files from the prior run (for example site-fetches.json, sites/*.body, or link-status.json) remain on disk while the new manifest omits them, so they are reported as unexpected. Remove the prior manifest's managed outputs before replacing the collection, or explicitly reject/clean nonempty output directories.

export function writeCollection(dir, { commands, repos, sites, outputs }) {
  const manifest = buildManifest({ commands, repos, sites, outputs });

scripts/assessment/lib/links.mjs:10

• Standard angle-bracket reference destinations are parsed incorrectly. For [ref]: <https://x.test/a>, this captures <https://x.test/a> (classified as internal) while the autolink regex also captures the bare URL, producing a bogus duplicate; destinations such as <./my page.md> are truncated. Parse and unwrap the optional angle-bracket form before classification.

const REFERENCE_DEFINITION = /^\s*\[[^\]]+\]:\s*(\S+)/gm;

scripts/assessment/lib/links.mjs:28

• A closing Markdown fence may only have trailing whitespace, but this closes on any same-marker run. A content line consisting of three backticks followed by js therefore ends the block early and causes subsequent example links to be inventoried as real links. Check that the remainder of the candidate closing line is blank.

      if (
        opener &&
        opener[1][0] === fence[0] &&
        opener[1].length >= fence.length
      ) {

scripts/assessment/collect.mjs:80

• rel.startsWith('..') also matches valid child names such as ..data. With --out <repo>/..data, the collector treats the output as external, does not exclude it, and inventories the previous outputs on the next run, breaking byte identity. Only treat .. as a parent segment.

  if (rel.startsWith('..') || path.isAbsolute(rel)) return null;

scripts/assessment/snapshot-agent-config.mjs:47

• This accepts malformed values such as /repo, owner/, and owner/repo/extra. The later split then uses empty segments or silently drops the suffix while the evidence still labels the original value, so it can query a different repository than the one recorded. Require exactly two nonempty path segments.

  if (!args.repo || !args.repo.includes('/')) {
    throw new Error('missing --repo owner/name');
  }

scripts/assessment/lib/snapshot.mjs:26

• The failure evidence discards the API response body and records only HTTP 401/403/.... GitHub's body carries the diagnostic needed to distinguish missing authentication from missing permissions, so this cannot support the stated credential/platform check. Preserve the response message (and useful fields such as documentation_url) in the error record.

    if (res.status !== 200) {
      return {
        ok: false,
        error: { status: res.status, message: `HTTP ${res.status}` },
      };

.github/workflows/assessment-test.yml:7

• The gate executes a script defined in package.json, but package.json is not included in the path filter. A PR changing only that script can disable or break the assessment test command without running this workflow. Include the package manifest in the trigger paths.

    paths:
      - scripts/assessment/**
      - .github/workflows/assessment-test.yml

scripts/assessment/lib/git.mjs:13

• Git SHA-256 repositories return 64-hex object IDs from rev-parse. This rejects those valid HEADs and records sha: null, so the manifest fails to pin that repository source. Accept both SHA-1 and SHA-256 object-ID lengths.

    return /^[0-9a-f]{40}$/i.test(out) ? out : null;

scripts/assessment/collect.mjs:92

• Joining raw arguments loses their boundaries in the provenance record. A valid repository path containing spaces, or a URL containing shell metacharacters, produces an ambiguous command that cannot reliably rerun the collection. Store the argv as structured data or apply a well-defined escaping scheme.

  return parts.join(' ');