COMSYS @ IEEE BigData 2025
COMSYS researcher David Heye attended the IEEE International Conference on BigData 2025 in Macau, where he presented his latest work on AI-assisted artifact evaluation.
David presented the paper “Supporting Artifact Evaluation with LLMs: A Study with Published Security Research Papers,” addressing the increasing challenges of artifact evaluation in cybersecurity research particularly as submission volumes grow and manual reproducibility checks become increasingly time-consuming. The presented work introduces an innovative toolkit based on Large Language Models (LLMs) that automates critical stages of artifact evaluation. Specifically, it supports reviewers by providing text-based reproducibility ratings, autonomously setting up execution environments for submitted code artifacts, and reliably identifying common methodological pitfalls in scientific contributions.
Beyond the technical sessions, IEEE BigData 2025 provided excellent opportunities for networking and international exchange among researchers, fostering insights into emerging trends at the intersection of data science and cybersecurity.
