Skip to content

Publish data/advisories.json indexed by formula name - #30

Merged
andrew merged 1 commit into
mainfrom
concat-advisories
Jul 31, 2026
Merged

Publish data/advisories.json indexed by formula name#30
andrew merged 1 commit into
mainfrom
concat-advisories

Conversation

@andrew

@andrew andrew commented Jul 28, 2026

Copy link
Copy Markdown
Member

Concatenates every advisories/*.json record into data/advisories.json, grouped by affected[0].package.name and sorted by id, so consumers can fetch the whole corpus in one request. Shape:

{
  "meta": {"count": N, "schema_version": "..."},
  "advisories": {"<formula>": [{<record>}, ...], ...}
}

rake advisories:concat builds it; regenerate.yml runs it after brew generate-vulns-advisories and the existing git add advisories/ data/ picks it up. The initial file (44 records, 10 formulae, ~150KB) is committed so https://raw.githubusercontent.com/Homebrew/advisory-database/main/data/advisories.json is live on merge.

This unblocks the generate-formula-api change in Homebrew/brew that reads this file (instead of querying OSV live per API build) and attaches a vulnerabilities field to each formula's API JSON. #29 will need the same rake advisories:concat step before its git add; happy to add that here or on #29's branch.

Copilot AI review requested due to automatic review settings July 28, 2026 15:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a generated, formula-indexed advisory corpus file (data/advisories.json) and automation to keep it up-to-date, enabling consumers (notably Homebrew/brew) to fetch advisory data in a single request rather than per-record lookups.

Changes:

  • Introduces AdvisoryIndex to concatenate advisories/*.json into data/advisories.json, grouped by formula and sorted deterministically.
  • Adds rake advisories:concat and wires it into the existing Regenerate workflow.
  • Adds a test suite covering grouping/sorting, error handling, and output formatting for the new index.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
lib/advisory_index.rb Implements the build/write logic for the new formula-indexed advisory JSON output.
Rakefile Adds advisories:concat task to generate/update data/advisories.json.
.github/workflows/regenerate.yml Runs the new concat rake task after regenerating advisory records.
test/advisory_index_test.rb Adds coverage for AdvisoryIndex behavior and output format.
data/advisories.json Commits the initial generated corpus so it’s immediately available to consumers post-merge.
Comments suppressed due to low confidence (1)

lib/advisory_index.rb:43

  • schema_versions.compact.max compares schema versions as plain strings, which can pick the wrong “highest” version lexicographically (e.g. "1.10.0" sorts before "1.9.0"). Use Gem::Version for numeric semantic version comparison and then convert back to a string for the JSON output.
    {
      "meta"       => {
        "count"          => by_formula.each_value.sum(&:size),
        "schema_version" => schema_versions.compact.max,
      },
      "advisories" => by_formula.transform_values { |records| records.sort_by { |r| r.fetch("id") } }
                                .sort.to_h,
    }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/advisory_index.rb
Comment thread test/advisory_index_test.rb Outdated
andrew added a commit that referenced this pull request Jul 28, 2026
Mirrors regenerate.yml (from #30): run rake advisories:concat after
brew advisory-match writes to advisories/, and stage data/ so the
concatenated corpus reflects the new matched records. Adds the
setup-ruby step the rake task needs. Depends on #30 for the task
itself.
@andrew
andrew requested a review from p-linnane July 28, 2026 16:24

@kmarekspartz kmarekspartz left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong opinions here, just thought I'd take a look since you've been a great feedback provider!

Comment thread lib/advisory_index.rb Outdated
Comment thread test/advisory_index_test.rb
@andrew
andrew force-pushed the concat-advisories branch from 8360f14 to e5252f1 Compare July 31, 2026 11:52
@andrew
andrew force-pushed the concat-advisories branch from e5252f1 to 8a67f1a Compare July 31, 2026 11:54
@andrew
andrew merged commit 531f4b4 into main Jul 31, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants