Anomaly Detection in Network Traffic Using Autoencoder-Based Unsupervised Learning

1. Introduction

Zero-day and novel attack variants routinely evade signature-based network intrusion detection systems, motivating unsupervised approaches that learn a model of normal traffic behaviour and flag deviations, without requiring labelled examples of every attack category in advance.

2. Methodology

A five-layer deep autoencoder was trained exclusively on benign flow records from the CICIDS2017 dataset, using 78 statistical flow features per record, with the reconstruction error threshold calibrated on a validation split of benign traffic to achieve a target 1 percent false positive rate, then evaluated on a held-out test set containing five previously unseen attack categories including port scanning and botnet traffic.

3. Results

The autoencoder achieved an F1-score of 0.93 across the five held-out attack categories at the calibrated threshold, compared with 0.85 for an isolation forest baseline and 0.81 for a one-class SVM baseline trained on the identical feature set, with the largest relative advantage observed for slow, low-volume attack patterns that departed only subtly from benign traffic statistics.

4. Conclusion

Autoencoder-based reconstruction error provides a more sensitive unsupervised anomaly signal than tree-based or kernel baselines for detecting previously unseen network attack patterns. Future work will evaluate detection latency and threshold drift under live production traffic.

References

[1] Sharafaldin I. et al., Toward generating a new intrusion detection dataset (CICIDS2017), ICISSP, 2018. [2] Sakurada M. and Yairi T., Anomaly detection using autoencoders with nonlinear dimensionality reduction, MLSDA Workshop, 2014.