Machine learning for sustained object recognition in video streams with limited computing resources of peripheral devices

Abstract

The paper is devoted to the development and experimental verification of machine learning methods for robust object recognition in video streams on edge devices with strictly limited computational and energy resources, used in monitoring systems for construction sites and urban environments. The aim of the study is to substantiate an approach to selecting neural network architectures and optimization modes that ensures a balance between recognition accuracy, inference latency, and energy consumption in real-time scenarios. The materials include a specialized video dataset exceeding 500 hours, featuring ground-based and drone footage of construction sites annotated by classes of machinery, personal protective equipment, and structural defects, as well as a range of typical edge platforms (single-board computers and neural accelerators) used for testing. The methodology involves applying structural and unstructured pruning techniques, weight quantization to FP16 and INT8 on lightweight architectures (YOLOv8-nano, MobileNetV3, EfficientDet-D0), multifactor statistical comparison based on mAP, F1-score, latency, energy consumption, and the tracking stability coefficient, as well as constructing an integral dimensionless efficiency index. It is demonstrated that the quantized YOLOv8-nano INT8 model provides nearly a fourfold reduction in model size and more than a 1,5-fold decrease in inference time and energy consumption per frame, while accuracy drops by less than 1,5 percentage points, yielding the best accuracy/energy ratio among the considered solutions. The analysis of robustness to external disturbances (fog, rain, low illumination) revealed sensitivity of detection completeness and an increase in classification errors under challenging conditions, indicating the need for adaptive preprocessing and hybrid schemes switching between light and heavy computation branches. The results of long-term stress testing demonstrated critical modes related to temperature and frame dropping during combined detection and data transmission, forming the basis for recommendations on function distribution between edge and cloud systems and on minimum hardware platform requirements. It is concluded that deep optimization of neural network models and accounting for the energy cost of operations are key prerequisites for scalable deployment of computer vision systems in smart construction and urban infrastructure.