In view of the complex construction site environment and the problems of the YOLOv3 helmet wearing detection algorithm such as low accuracy and poor robustness, an improved YOLOv3 helmet wearing detection algorithm was proposed. The K-means algorithm was used to cluster out anchor boxes, so as to improve the network output scales. The residual block was introduced at the output to form a residual module; at the same time, the classification loss function was improved to balance the influence of positive and negative samples and difficult and easy samples on the model. In order to verify the effectiveness of the method, it was verified on the NVIDIA GTX1660Ti. The experimental results show that the mAP of the improved YOLOv3 helmet wearing detection algorithm increase by 4.84%, the detection ability of occluded targets and small targets is improved, and the model has strong robustness.