Felony Bench: Be AI, Do Crime

Introduction

Evaluating large language models for safety and policy compliance has become one of the most brittle parts of modern AI engineering. Static test suites fail to capture emergent behaviors, and manual red-teaming does not scale across model versions. Felony Bench addresses this gap by providing a deterministic, sandboxed adversarial simulation framework designed to stress-test AI systems against malicious inputs, prompt injection, and policy boundary violations. The title is deliberate: we simulate adversarial behavior in a controlled environment to measure how well a model holds its ground when pushed to its limits.

The architecture follows an event-driven microkernel pattern. Instead of wrapping a single inference endpoint, the system isolates scenario generation, agent execution, target proxying, and safety verification into independent components communicating over a structured event bus. Each evaluation run produces an immutable audit trail, enabling reproducible regression testing and precise failure attribution. We built this to replace ad-hoc jailbreak scripts with a production-grade evaluation harness that integrates cleanly into CI/CD pipelines and model release gates.

Why This Matters

Shipping an AI model without rigorous adversarial evaluation introduces measurable liability. In production, models encounter malformed prompts, context poisoning, and deliberate policy evasion attempts. Traditional evaluation relies on curated datasets that age quickly as model capabilities shift. Engineers need a system that dynamically generates stress scenarios, enforces strict isolation boundaries, and provides deterministic rollback capabilities when a model fails a safety threshold.

Felony Bench solves three concrete production pain points:

1. Reproducibility: Evaluation runs are seeded and checkpointed, allowing exact recreation of failure states for debugging.
2. Containment: The framework enforces network and process isolation, ensuring adversarial agents cannot leak data or interact with live services outside the test boundary.
3. Policy Drift Detection: Continuous monitoring tracks how model outputs shift against compiled safety rules, flagging regressions before deployment.

When we integrated this pattern into our internal model release pipeline, we reduced post-deployment safety incidents by isolating evaluation from production traffic and enforcing hard gates on policy violation rates.

How It Works

The system operates as a closed-loop evaluation harness. A scenario synthesizer generates constrained adversarial prompts based on threat models. These prompts feed into an agent orchestrator that executes multi-turn interactions with a target model proxy. A safety monitor intercepts every request and response, validating them against a compiled policy engine. If a violation occurs, a containment circuit triggers, halting the run and capturing a deterministic state snapshot. Results flow into an evaluation logger that computes pass/fail metrics and feeds back into the scenario generator for adaptive stress testing.

flowchart TD
    A[Threat Model Config] --> B(Scenario Synthesizer)
    B --> C{Constraint Validator}
    C -->|Valid| D[Agent Orchestrator]
    C -->|Invalid| B
    D --> E[Target Model Proxy]
    E --> F[Safety Monitor]
    F -->|Pass| G[Evaluation Logger]
    F -->|Violation| H[Containment Circuit]
    H --> I[Audit Snapshot]
    G --> J[Feedback Optimizer]
    J --> B
    I --> G

Each component communicates via a typed event stream. The scenario synthesizer respects boundary constraints to prevent out-of-scope testing. The agent orchestrator manages conversation state, retry logic, and rate limiting. The safety monitor applies fast-path rule matching before falling back to heavier verification. The feedback optimizer adjusts scenario difficulty based on historical pass rates, ensuring the benchmark remains challenging as models improve.

Core Concepts

The framework rests on four architectural pillars:

Constraint-Based Scenario Synthesis Prompts are not hand-crafted. The synthesizer uses a rule graph to generate adversarial inputs that target specific policy boundaries. Constraints define allowed tokens, prohibited patterns, and interaction depth. This ensures tests remain focused and reproducible.

Stateful Agent Orchestration The agent runs as a deterministic finite state machine rather than a blind prompt loop. Each turn updates an internal context store, tracks conversation entropy, and applies backoff strategies when the target model exhibits degradation. State snapshots enable exact replay.

Multi-Tier Safety Verification Safety checks operate at three levels: static pattern matching for known jailbreak signatures, runtime policy evaluation for behavioral compliance, and network egress filtering to prevent data exfiltration. The verifier compiles rules into a lookup structure for O(1) fast-path evaluation.

Immutable Audit Trails Every run writes to an append-only log with cryptographic hashing. Logs contain raw prompts, model outputs, policy decisions, and timing metrics. This structure supports forensic analysis and regulatory compliance without requiring live system access.

Examples & Code Walkthrough

The following implementation demonstrates the core orchestration loop, safety verifier, and constraint validation. The code uses asyncio for non-blocking execution, dataclasses for structured state, and a compiled policy engine for deterministic rule matching.

import asyncio
import logging
import hashlib
from dataclasses import dataclass, field
from enum import Enum, auto
from typing import Dict, List, Optional

logger = logging.getLogger("felony_bench")

class PolicyAction(Enum):
    ALLOW = auto()
    BLOCK = auto()
    QUARANTINE = auto()

@dataclass
class SafetyRule:
    id: str
    pattern: str
    action: PolicyAction
    compiled: bool = False

@dataclass
class EvaluationState:
    run_id: str
    turn_count: int
    context: Dict[str, str] = field(default_factory=dict)
    violation_log: List[str] = field(default_factory=list)

class SafetyVerifier:
    def __init__(self):
        self.rules: Dict[str, SafetyRule] = {}
        self.fast_path_cache: Dict[str, PolicyAction] = {}

    def compile_rules(self, rules: List[SafetyRule]) -> None:
        for rule in rules:
            rule.compiled = True
            self.rules[rule.id] = rule
        logger.info("Compiled %d safety rules", len(rules))

    def evaluate(self, payload: str, state: EvaluationState) -> PolicyAction:
        cache_key = hashlib.sha256(payload.encode()).hexdigest()[:16]
        if cache_key in self.fast_path_cache:
            return self.fast_path_cache[cache_key]

        action = PolicyAction.ALLOW
        for rule in self.rules.values():
            if not rule.compiled:
                continue
            if rule.pattern.lower() in payload.lower():
                action = rule.action
                state.violation_log.append(f"Rule {rule.id} triggered")
                if action != PolicyAction.ALLOW:
                    break

        self.fast_path_cache[cache_key] = action
        return action

class ContainmentCircuit:
    def __init__(self, max_violations: int = 3):
        self.max_violations = max_violations
        self.triggered = False

    def check(self, state: EvaluationState) -> bool:
        if len(state.violation_log) >= self.max_violations:
            self.triggered = True
            logger.warning("Containment circuit tripped for run %s", state.run_id)
            return True
        return False

async def run_evaluation(
    synthesizer: asyncio.Queue,
    proxy_client: object,
    verifier: SafetyVerifier,
    circuit: ContainmentCircuit,
    seed: str
) -> EvaluationState:
    state = EvaluationState(run_id=seed, turn_count=0)
    
    while not circuit.check(state):
        prompt = await synthesizer.get()
        state.context[f"turn_{state.turn_count}"] = prompt
        
        try:
            response = await proxy_client.generate(prompt)
        except Exception as exc:
            logger.error("Proxy failure at turn %d: %s", state.turn_count, exc)
            break

        action = verifier.evaluate(response, state)
        if action == PolicyAction.BLOCK:
            logger.info("Blocked response at turn %d", state.turn_count)
            break

        state.turn_count += 1
        await asyncio.sleep(0.1)  # Rate limiting

    return state

The SafetyVerifier compiles rules into a dictionary and caches evaluation results using a truncated SHA-256 hash. This avoids redundant string matching while maintaining deterministic behavior. The ContainmentCircuit enforces a hard stop after a configurable violation threshold, preventing runaway inference costs. The run_evaluation coroutine manages the async event loop, applies rate limiting, and captures state transitions for replay.

Best Practices

1. Isolate Evaluation Namespaces: Run the framework in dedicated containers with dropped