Skip to content

Troubleshooting

SymptomInspect firstTypical boundary
Component cannot loadfull entry point and import tracebackmodule/file resolution or exported object
Run starts but no cases completelatest case-scoped eventprovider, detector, or judge transition
Metric looks impossiblecontributing cases.jsonl rowscase labels/decisions or metric definition
Audit rejects a runmissing/mismatched artifact named by auditserialization, interrupted write, or drift
Local profile passes, real profile failsresolved provider config and live request errorcredentials, endpoint, model identity, network
  1. Preserve the failing artifact directory.
  2. Reproduce the smallest affected case without changing revisions.
  3. Identify the earliest divergent stage.
  4. Add a focused test at that boundary.
  5. Fix and rerun the same case before expanding scope.

Avoid treating fallback/local success as proof that a configured real provider was reached. The profiles answer different questions.