Enhancing and Benchmarking In-Network Computer Vision Methods

Introduction

In industrial scenarios, camera equipment used for device control, safety and quality assurance can produce huge amounts of data. Traditionally, this data is analyzed in downstream servers, so that the video streams have to be transferred over the network to the computing equipment. This puts heavy strain on the network and introduces delays to the decision loops.

One method to mitigate both the large bandwidth requirements as well as to lower latencies is to introduce in-network computation on the video streams, i.e., to partially analyze the data and react on it as it flows through the network and before it reaches the main computer vision processing servers.

In prior thesis work at our chair, a proof-of-concept implementation has already shown that significant reductions in both metrics can be obtained in simplistic scenarios with artificial video data.

Your Challenge

In your work, you will enhance and evaluate the performance of our in-network CV implementation on realistic data. To this end, you will couple a real-life industrial camera with our system, enhance the existing CV implementations to support dynamic reactions (e.g., rerouting, dropping, or frame aggregation) to the camera data, and then conduct performance evaluations of the algorithms.

The algorithms are implemented on state-of-the-art programmable SmartNICs by Netronome in Micro-C, a specific restricted variant of C. For this topic, you hence need to be familiar with C or C++ for a quick start. Importantly, knowledge of CV is good to have but not necessary as our CV pipeline must make concessions to the specific processing capabilities of network hardware so that only easy-to-understand basic CV algorithms can be run.

We are looking for a motivated student willing to spend 6- approx. 10 h/w for a period of 4 - 6 months. Working times can be freely arranged. Ideally, you could start your work by July, 2025. We at COMSYS offer a friendly, creative working atmosphere at eye level between staff members and students, a general open-door policy, three student rooms, a billiard table, occasional social gatherings, drinks and snacks sold near purchasing price, and more.

Contact

If you are interested in this topic, just contact René informally via email!