Skip to content

feat(labs): add defensive Guardrail blue challenge pack - #566

Draft
nitinawari wants to merge 1 commit into
GenAI-Security-Project:mainfrom
nitinawari:feat/labs-blue-challenge-pack
Draft

feat(labs): add defensive Guardrail blue challenge pack#566
nitinawari wants to merge 1 commit into
GenAI-Security-Project:mainfrom
nitinawari:feat/labs-blue-challenge-pack

Conversation

@nitinawari

Copy link
Copy Markdown
Contributor

Summary

  • Adds 5 paired Labs Guardrail blue challenges:
    • Threshold Guard ↔ Approve Invoice Over Limit
    • Destructive Tool Gate ↔ Scorched Earth
    • Script Sanitizer ↔ Shell Shock
    • Payment Integrity Gate ↔ Fine Print
    • Egress Gate ↔ Double Agent
  • Extends GuardrailPreventionDetector with optional YAML gates (RCE argument patterns, transfer-over-invoice)
  • Enriches before_tool hooks with invoice_amount for update_invoice_status and finstripe__create_transfer
  • Includes Labs Activity filter for A.2 agent.security.* events

Depends on

Do not merge until these are on main:

Test plan

  • pytest tests/unit/labs/test_guardrail_detector.py tests/unit/labs/test_guardrail_context.py -v
  • Manual: configure Labs webhook + Before Tool, solve one blue challenge (Threshold Guard or Egress Gate)
  • Labs Activity shows agent.security.guardrail_trigger with verdict: block

GSoC mapping

Week 5-8 (phase 3)

  • Deliverable A.2 : A standardized security event model for: prompt/goal changes, memory read/write, tool selection, tool parameters, tool outputs, privilege/authorization decisions, and guardrail triggers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant