Margin-reciprocal loss: enhancing robust network anomaly detection on imbalanced traffic data

Abstract

Accurate detection of network intrusions remains challenging under severe class imbalance, where rare attacks such as remote-to-local (R2L) and user-to-root (U2R) are poorly represented. Although many learning-based intrusion detection systems achieve high overall accuracy, conventional loss functions often bias training toward majority classes, leading to weak minority-class performance. This paper introduces a smooth margin-reciprocal loss (MRL), inspired by distance-weighted discrimination (DWD), which emphasizes samples with small or negative margins while rapidly attenuating penalties for well-classified instances. Unlike probability-based focal loss, MRL operates directly on the signed margin and enables stable optimization with first-order methods. Experiments conducted on the NSL-KDD benchmark using linear and shallow multilayer perceptron models show that MRL consistently improves macro-F1 and per-class precision–recall AUC compared with hinge, logistic, and focal losses, with notable gains on minority attack classes.