fix(#367): read has_credentials from durable hit/risk vectors, not equations - #2376
Merged
Conversation
…uations #381 fixed the always-zero has_credentials SQLite column by reading file_data["equations"]["sec_hardcoded_secrets"], verified only against a hand-built equations dict. Population-scale validation (#1144: 718 repos, 1.2M files) showed the column still 100% zero -- file_data["equations"] is not a durable carrier, it's rebuilt/emptied/pruned across several galaxyscope.py phases before record_keeper.py reads it. Read the durable carriers instead: hit_vector["sec_hardcoded_secrets"] (raw SecurityLens count) and risk_vector["secrets_risk"] (thresholded score, also spiked to 100 for critical leaks via a path that zeroes the hit_vector -- so both slots are checked). These are the same vectors this recorder already writes as the threat_private_info / risk_secrets_risk columns, which #1144 confirms carry real nonzero data for 437 files in the same dataset. equations kept only as a last-resort fallback. Tests: positive assertion now exercises the durable path (mock's equations dict deliberately has no sec_hardcoded_secrets key); new test_has_credentials_zero_without_signal guards against an always-on read. Closes #367. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The #367 fix inserts ~24 lines above the pre-existing W291/RUF005 baseline findings in record_keeper.py; audit_check.py --regenerate confirms all 10 are pure line-shifts (same file/code/message), no new findings. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
squid-protocol
added
bug
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #367.
Background
#367 (
has_credentialsSQLite column permanently 0) was closed by #381,which read
file_data["equations"]["sec_hardcoded_secrets"]. #381's testonly ever fed a hand-built
equationsdict, so it proved the arithmeticbut not that the real pipeline delivers that dict intact.
Population-scale validation (#1144: 718 repos, 1,201,836 files) reopened
it —
has_credentialswas still 100% zero, whilerisk_secrets_riskhad genuine nonzero values for 437 files in the same dataset.
file_data["equations"]is not a durable carrier: it's rebuilt / emptied/ per-key
del'd across severalgalaxyscope.pyphases beforerecord_keeper.pyreads it.Fix
Read the durable carriers instead:
hit_vector["sec_hardcoded_secrets"]— raw SecurityLens countrisk_vector["secrets_risk"]— thresholded score; also spiked to 100for critical leaks via a path that zeroes the hit_vector
(
signal_processor.py:360), so both slots are checkedThese are the same vectors
record_keeper.pyalready writes as thethreat_private_infoandrisk_secrets_riskcolumns — the ones #1144confirms carry real data.
equationsis kept only as a last-resortfallback.
Tests
equationsdict deliberately has nosec_hardcoded_secretskey.test_has_credentials_zero_without_signal: both durable slots emptytests/tools_recorders/pass; mypy / dead-key / ast-accuracyclean; ruff baseline regen is confirmed pure line-shift (separate
commit).
Not in scope
#368 (
binary_anomaly) has the sameequations-durability bug plus itsown open question (whether
scan_binary()output reachesfile_dataforbinary files at all). Left for a separate PR.
🤖 Generated with Claude Code