Small sub-domain filtering method is one of common methods in processing gravity data. After processing gravity anomaly data, the cascade bands of gravity anomaly become more convergent, which could be used to reflect geological boundaries. Multiple iterations can improve the effect of convergence, but they also cause serious distortions of boundaries of gravity anomalies. In order to remedy the defect, we use an improved small sub-domain filtering method which raises calculation speed and alleviates the distortions of abnormal boundaries by improving partition mode. We conducted a model experiment to verify the feasibility of this method. The Model experiment shows that geological boundaries can be better reflected by gravity anomaly data after reusing the improved small sub-domain filtering method several times, and the distortions of abnormal boundaries are well alleviated. We reused this method to process gravity anomaly data of Hulin Basin several times. Then we got the map of Hulin Basin faults distribution by the result, and a distribution of 25 faults was obtained. The result matches well with existing geological data.