Azoth

Static malware detection by routed ensemble. A general LightGBM model scores every file. Per-filetype specialists score files in their domain — PE, ELF, JavaScript, PDF, and 37 more. A file is flagged when any route's score crosses its operating-point threshold.

The point of routing is that the evidence differs by format. A PE's section table is signal. A PDF's stream dictionary is signal. A shell script's token distribution is signal. One generalist trained over all of them learns averages; a specialist trained on one of them learns the format.

Thresholds were fit on a 1,720,070-row dev partition (12.5% of the labeled corpus). The numbers in this README come from a locked 1,719,220-row test partition, disjoint from training and calibration. The bundle is loaded at scan time by litmus. EMBER 2024 reference: Joyce et al., KDD'25.

Use

Input is a JSON report produced by cleave. Output is one verdict — benign or hostile — qualified by a severity level L0..L25000. Litmus reads the hostile threshold at the chosen level (consumers can label anything firing above the configured critical level as suspicious if they want a softer tier); the deployed default is L25 (0.25 FP/M). Lower levels tighten the operating point; higher levels loosen it.

Bundle layout

config.json records the deployed thresholds. Each route lives in its own subdirectory: general/, one of 7 filegroups/<name>/, or one of 41 filetypes/<name>/. A route directory carries two files: model.txt (LightGBM) and feature_spec.json (the features the model expects). Scores are the model's raw probabilities — there is no separate probability calibrator.

Further reading: ENSEMBLE_MODEL.md for routing details, GENERALIST_MODEL.md for the single-model baseline. License: Apache 2.0.

Per-filetype Ensemble Performance

Each row is the routed ensemble's intrinsic ranking quality on that filetype's slice of the locked test partition — PR AUC, ROC AUC, F1 at the F-beta-tuned threshold, and recall at the L25 (0.25 FP/M) operating point. These are properties of the model's scores; how litmus chooses to threshold those scores at each severity level is a runtime concern documented in route_policies.md.

A filetype appears here when it has at least 25 malware and 25 benign in the test slice, or at least 100 of each across the full labeled corpus. Pure archive wrappers (zip, tar, gz, ...), data, and unknown are excluded — their score reflects the container's shape, not the classifier's quality.

Optimization target. Each filetype's ensemble combiner is selected to maximize recall at L25 (0.25 FP/M) on the dev partition, with PR AUC as a tiebreak. Selection is constrained so the ensemble can never report worse than the specialist alone — when no combiner clears the specialist on dev, the ensemble falls back to specialist_priority (which equals the specialist by construction). This matches the deployment budget litmus operates at and the design intent that routing must improve, not degrade, the per-filetype model.

PR AUC summarizes recall against precision across operating points. Recall@L25 is the selection-budget headline; for filetypes whose calibration slice cannot resolve L25 (0.25 FP/M) empirically, that level shares an operating point with its neighbours (its measured ceiling). EMBER 2024 deltas are reported where Joyce et al. publish per-filetype numbers (Table 5, All files → X).

Recall by FP level (per 100M benigns)

The corpus-weighted ensemble curve weights each filetype's ensemble recall by the number of labeled files in that filetype, answering: "If I draw a random file from the labeled corpus, what fraction of malware do we catch at this FP budget?" The general curve is the single-model baseline on the full evaluated dataset. filetypes/elf is shown for comparison — a single strong route that resolves across levels, so the corpus-weighted line (dominated by large, near-flat routes like pe) can be read against it. The vertical dashed line marks the L25 deploy operating point.

Provenance

Calibration snapshot 2269663951, score-table c432e1f3d36a, model-set f94770ed508f. 1 general, 7 filegroup, 41 filetype routes.

Limits

• Strict L0..L25 (FP/100M) targets can sit below the calibration benign volume's resolution (the finest non-zero rate is 1 FP / N_benign); below that, adjacent levels share one measured-ceiling operating point. Thresholds are measured (loosest score within the level's FP budget +1 slack), not extrapolated — they can't overshoot on live traffic. More benigns sharpen the low levels.
• The split is content-deduplicated by canonical_sha256, not family-aware. Campaign-level generalization may be overstated.
• Deployment distribution may differ from the training corpus.

Sources

MalwareBazaar, VirusShare, Backstabber's Knife Collection, DataDog malicious-software-packages-dataset, VX Underground, PyPI MalRegistry, Linux Malware Samples, Tim (Wadhwa-)Brown's Linux Malware Repo, Javascript Malware Collection, ObjectiveSee macOS Malware Collection, Practical Security Analytics PE Malware ML Dataset, Ultimate RAT Collection.