In recent years, China has vigorously advocated green buildings. Prefabricated buildings are in line with the requirements of policy development. However, the implementation of prefabricated buildings is constrained by many factors. Among many influencing factors, the cost problem is the bottleneck of prefabricated development. Therefore, this paper identifies the influencing factors of construction cost control, and uses the Interpretative Structural Model (ISM) based on system engineering theory to study the internal relations of these influencing factors, and classify the influencing factors. Finally, according to the key factors, it proposes a cost control method to provide a reference for the contractor to carry out cost control.