Execute a comparable experiment
Resolve versioned inputs, run an attack/detector path, and preserve the evidence needed to inspect it later.
Turnkey
Bring Your Own Detector. Turn an idea into a reproducible, inspectable jailbreak-detection experiment without rebuilding the evaluation stack.
From zero to an auditable run.
Start locally on CPU with a small built-in path. The same component boundary carries forward when you connect a real detector, model provider, dataset, or attack.
pip install turnkey
turnkey init my-detector
cd my-detector
turnkey dev ./detector.py:build0.1.0 is the functional release on PyPI. Prefer a source checkout when you want the full config library and development extras.
Run. Build. Debug. Audit.
Bring your own detector
Implement the public component boundary instead of forking the runner, metrics, and artifact pipeline.
Trace a missed case
Follow structured events and typed signals to find where selection, policy, provider, or judgment diverged.
Recompute, do not just trust
Cross-check cases, events, metrics, and run metadata to make unsupported success claims harder.
A readable path from input to evidence.
Turnkey keeps the experiment path explicit. Built-ins use registry aliases; external components use module or file entry points behind the same contract.
dataset → selection → attack → policy/provider → judge → metrics → artifacts → audit
A continuous book, backed by notebooks.
Follow the chapters in order for the D2L-style learning path, or use Tutorials, Guides, Concepts, and Reference independently like a framework documentation site.
First auditable run
Learn the command, output tree, and evidence boundary.
Build a detector
Implement a small detector without deleting the real extension boundary.
Debug a missed case
Read case and event artifacts as one causal trace.
Typed signals
Carry detector evidence through the pipeline without ambiguous dictionaries.
Course depth and reference speed.
Start with one honest run.
CPU-safe, inspectable, and small enough to understand.