Warn when the CBMC on PATH does not match the pinned version - #4723
Conversation
feliperodri
left a comment
There was a problem hiding this comment.
@ivmat again, the code is massively over-narrated... version.rs is roughly half comments, and main.rs has 8-line rationale essays in front of 3-line blocks. This can be a real maintenance burden: prose comments drift from code, and this volume invites that. Let's cut them ~in half (keep the why for include_str!, --quiet, and the raw-scan; drop the paragraphs restating what the code plainly does).
|
@feliperodri addressed the comments problem and the #4466 changes so this PR only covers CBMC pin check |
`kani --version` and the startup banner now also print the CBMC version found on PATH. When it differs from the `CBMC_VERSION` pin in `kani-dependencies`, Kani prints a warning and continues: an unpinned CBMC should not block users, but its results may not match CI. The pin file is embedded with `include_str!` because release bundles do not ship `kani-dependencies`, so a runtime read would disable the check for exactly the installs that can least audit their toolchain. `cargo kani --version` now also answers when a malformed `Cargo.toml` makes `join_args` fail, via a raw-argument scan used only on that path. The check is silent under `--quiet` (tested zero-output contract) and absent when `cbmc` is not on PATH: a missing CBMC fails loudly at verification time.
7dc93d4 to
d307faa
Compare
feliperodri
left a comment
There was a problem hiding this comment.
Thanks @ivmat — this is a genuinely useful safety net, and I hit exactly this problem myself (a from-source CBMC on PATH silently diverging from the pin). The comment trimming from the last round looks great too.
I pushed two small cleanups directly to the branch to get it ready:
- Deduped the
cbmc --versionprobe —get_cbmc_infonow reusesversion::cbmc_version_on_pathinstead of a second spawn + parse. - Dropped the
--version/malformed-Cargo.tomlhandling (andrequests_version) to keep this focused on the pin check; that also removed the only merge conflict with main, so it's mergeable now.
Verified live end-to-end: a matching CBMC 6.10.0 is silent, a mismatched 6.8.0 prints the warning naming both versions, and --quiet stays silent. LGTM! 🚀
Description
kani-dependenciespins a CBMC version, but the pin is only enforced at setup/CI time (install_deps.sh,kani-regression.sh). At runtimekani-driveruses whatevercbmcresolves fromPATHwith no check, so a locally installed CBMC can silently diverge from the pin while the user believes the pinned toolchain is in use. On a machine with several CBMC versions installed this is easy to hit and invisible when it happens — results get attributed to a toolchain that never ran.This PR makes
kani-driverresolvecbmc --versionfromPATH, compare it against the pin, and warn naming both versions when they differ. Design points:include_str!ofkani-dependencies) rather than read at runtime: release bundles don't ship that file, so a runtime read would silently disable the check for exactly the users least able to audit their toolchain. rustc tracks the included file, so editing the pin still rebuilds.--versionis handled explicitly (disable_version_flag) sokani --version/cargo kani --versionactually run the check — clap's built-in flag exits during parsing, before any driver code runs. The first output line stayskani <version>/cargo-kani <version>, so scripts that parse it keep working; the CBMC lines are appended after it.cargo kani, the version flag is detected on the raw arguments before project configuration is merged, so--versioncannot be broken by a malformedCargo.toml(matching the robustness of clap's built-in, whose version action previously fired insidecargo_locate_project's early parse). The scan matches whole arguments only and stops at--or--cbmc-args; a post-parse fallback catches spellings the scan cannot see, such as-Vclustered with other short flags. Known residual: a clustered-Vcombined with a malformedCargo.tomlreports the TOML error instead of the version — loudly, not silently.--quietduring verification (preserving the zero-output contract incheck-quiet.sh), but--versionas an explicit query always reports.tools/build-kanibundling whichevercbmcis on the builder'sPATHis the build-time analogue of the same problem; left as a follow-up to keep this focused. With this check, a mismatched bundle at least reports itself at runtime.Manual testing
Unit tests cover the version parse/compare and the raw
--versionscan. Verified live both ways — a matching CBMC 6.10.0 is silent, a mismatched 6.8.0 fires naming both versions. Both script-based version regression tests (kani-version-flag-version,cargo-kani-version-flag-version) pass.cargo kani --versionverified to work from a directory whoseCargo.tomlis malformed, and a--versionafter--cbmc-argsis forwarded to CBMC, not intercepted.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.