In order to improve the recognition rate of handwritten numbers, we have improved AlexNet network model in this paper. Conv3 and Conv4 were introduced to replace the model for Inception-resnet module, which improves the feature extraction capability of the model. The Batch Normalization (BN) method was used to accelerate network convergence and prevent overfitting,reducing the number of convolutional kernels and improving the training speed of the network. In this paper, training and testing are carried out on MNIST data sets. Experimental results show that the improved network model has a better detection accuracy of 0.9966, which proves the effectiveness of the algorithm.