Rural banks, as the main force of the new rural financial institution, effectively solved the problems of agriculture, rural areas and farmers in the new rural construction in China to a large extent. Through the study of the current situation of rural banks in china, the internal and external problems, the application of advanced international experience of small financial institutions from Bangladesh, India, Latin America and the United States, this paper is going to explore some ways for the sustainable development of the rural banks in China.