This file was created by the TYPO3 extension bib --- Timezone: UTC Creation date: 2025-01-22 Creation time: 18-47-03 --- Number of references 4 article 2023_lamberts_metrics-sok Journal of Systems Research 2023 10 31 3 1 Industrial systems are increasingly threatened by cyberattacks with potentially disastrous consequences. To counter such attacks, industrial intrusion detection systems strive to timely uncover even the most sophisticated breaches. Due to its criticality for society, this fast-growing field attracts researchers from diverse backgrounds, resulting in 130 new detection approaches in 2021 alone. This huge momentum facilitates the exploration of diverse promising paths but likewise risks fragmenting the research landscape and burying promising progress. Consequently, it needs sound and comprehensible evaluations to mitigate this risk and catalyze efforts into sustainable scientific progress with real-world applicability. In this paper, we therefore systematically analyze the evaluation methodologies of this field to understand the current state of industrial intrusion detection research. Our analysis of 609 publications shows that the rapid growth of this research field has positive and negative consequences. While we observe an increased use of public datasets, publications still only evaluate 1.3 datasets on average, and frequently used benchmarking metrics are ambiguous. At the same time, the adoption of newly developed benchmarking metrics sees little advancement. Finally, our systematic analysis enables us to provide actionable recommendations for all actors involved and thus bring the entire research field forward. internet-of-production, rfc https://www.comsys.rwth-aachen.de/fileadmin/papers/2023/2023-lamberts-metrics-sok.pdf eScholarship Publishing 2770-5501 10.5070/SR33162445 1 OlavLamberts KonradWolsing EricWagner JanPennekamp JanBauer KlausWehrle MartinHenze inproceedings 2023-wagner-lcn-repel 2023 10 https://www.comsys.rwth-aachen.de/fileadmin/papers/2023/2023-wagner-repel.pdf IEEE 48th IEEE Conference on Local Computer Networks (LCN), Daytona Beach, Florida, US Daytona Beach, Florida, US IEEE Conference on Local Computer Networks (LCN) Oktober 1-5, 2023 accepted en 1 EricWagner NilsRothaug KonradWolsing LennartBader KlausWehrle MartinHenze inproceedings 2023-bader-metrics 2023 9 28 https://www.comsys.rwth-aachen.de/fileadmin/papers/2023/2023-bader-metrics.pdf Proceedings of the 9th Workshop on the Security of Industrial Control Systems & of Cyber-Physical Systems (CyberICPS '23), co-located with the the 28th European Symposium on Research in Computer Security (ESORICS '23) The Hague, The Netherlands 9th Workshop on the Security of Industrial Control Systems & of Cyber-Physical Systems (CyberICPS '23) September 28, 2023 accepted 10.1007/978-3-031-54204-6_2 1 LennartBader EricWagner MartinHenze MartinSerror inproceedings 2023_wolsing_ensemble 2023 9 25 14345 102-122 Industrial Intrusion Detection Systems (IIDSs) play a critical role in safeguarding Industrial Control Systems (ICSs) against targeted cyberattacks. Unsupervised anomaly detectors, capable of learning the expected behavior of physical processes, have proven effective in detecting even novel cyberattacks. While offering decent attack detection, these systems, however, still suffer from too many False-Positive Alarms (FPAs) that operators need to investigate, eventually leading to alarm fatigue. To address this issue, in this paper, we challenge the notion of relying on a single IIDS and explore the benefits of combining multiple IIDSs. To this end, we examine the concept of ensemble learning, where a collection of classifiers (IIDSs in our case) are combined to optimize attack detection and reduce FPAs. While training ensembles for supervised classifiers is relatively straightforward, retaining the unsupervised nature of IIDSs proves challenging. In that regard, novel time-aware ensemble methods that incorporate temporal correlations between alerts and transfer-learning to best utilize the scarce training data constitute viable solutions. By combining diverse IIDSs, the detection performance can be improved beyond the individual approaches with close to no FPAs, resulting in a promising path for strengthening ICS cybersecurity. Lecture Notes in Computer Science (LNCS), Volume 14345 Intrusion Detection; Ensemble Learning; ICS internet-of-production, rfc https://www.comsys.rwth-aachen.de/fileadmin/papers/2023/2023-wolsing-ensemble-iids.pdf Springer Proceedings of the 28th European Symposium on Research in Computer Security (ESORICS '23), September 25-29, 2023, The Hague, The Netherlands The Hague, The Netherlands 28th European Symposium on Research in Computer Security (ESORICS '23) September 25-29, 2023 978-3-031-51475-3 0302-9743 10.1007/978-3-031-51476-0_6 1 KonradWolsing DominikKus EricWagner JanPennekamp KlausWehrle MartinHenze