Jevotron Tool Integrates Jev Anomaly Detection for Command Line Use

Command-Line Anomaly Detection

Jevotron is a new command-line tool designed for one-shot anomaly detection. Users can provide a file and guidance to receive field-level anomaly scores and a focused review queue. The tool also includes a caching mechanism to reduce the cost of subsequent runs on unchanged data.

Setup and Usage

To use Jevotron, an API key from the TypeSafe dashboard is required. The tool supports various file formats including CSV, YAML, JSON, TOML, text, OBO, and gzipped files. Users can specify fields with '--field' and provide instructions with '--guidance' or '--guidance-file'. A local Python configuration is available for custom parsing or reusable settings.

Efficiency and Review Features

Jevotron reuses assessments for unchanged input, saving successful results in SQLite. This allows users to reorder files, change thresholds, or resume failed runs without re-assessing unchanged entries. Reports include each field's probabilities, entry score, source location, and assessment date, with a warning score based on the highest field anomaly probability.

Agent Trace Performance

In a sample of 24 public traces, Jevotron matched 130 of 163 step-quality labels, achieving a 79.8% match rate. The harmful-step precision was 89.7%, with a recall of 70.3%. This evaluation used original messages and tool definitions, with human labels withheld from the model.