Crack detection is one of the important aspects of structural health monitoring. To achieve qualitative analysis of cracks in steel beams in complex backgrounds, a two-stage detection method based on improved Convolutional Neural Network (CNN) and digital image processing is proposed for crack damage location and crack image segmentation. The first stage uses a multi-scale convolutional neural network to identify crack images in complex backgrounds. This network consists of a multi-scale convolution module Inception and a residual module. The multi-scale convolution module Inception contains three different-sized convolution kernels (1×1, 3×3, 5×5) for multi-scale feature extraction of the image. In the residual module, convolution layers and nonlinear activation functions are introduced to enhance cross-layer fusion ability and extract deeper features. The Grad-CAM visualization analysis highlights the prediction basis of the multi-scale convolutional neural network, proving its classification performance and discrimination basis. In the second stage, for the identified crack images, a combined process of image filtering denoising, threshold segmentation to separate crack pixels, and morphological processing to optimize the segmentation result is proposed for pixel-level segmentation and extraction of cracks. The pixel marking results manually annotated are used as the true labels to evaluate the recognition effect of image segmentation. The training results on the dataset show that the multi-scale convolutional neural network has an identification accuracy of 98.8% for steel beam crack images. The proposed image processing combination process has a maximum intersection-over-union (IOU) of 0.819, which can better classify and extract cracks.