In order to reveal the effect of slope gradient and surface roughness on soil erosion on the Loess Plateau, indoor simulated rainfall experiments were conducted. The five slope gradients in the experimental design were 9°, 12°, 15°, 20°, and 25°, the rainfall intensity was 1 mm/min, and the rainfall duration was 60 min. The results showed that: (1) The volume of overland runoff increased first and then decreased under all five slope gradients, with significant difference being found between the 20° and 25° slope; the drop rate of variation in surface roughness was in the order of 12°＞9°＞25°＞15°＞20°. (2) Volume of runoff was in significant exponential function relationships with slope gradient and surface roughness (ln Mw=-0.012S-0.005?Rcv+3.78, R2=0.66). (3) The yield of sediment increased first and then decreased under all five slope gradients. Sediment yield was significantly different at slope gradient of 3° slope and between the 20° and 25° slope gradient, indicating that slope gradient had a significant impact on sediment yield. (4) Yield of sediment was in significant linear relationships with slope gradient and surface roughness (Ms=0.014S-0.008Rcv+1.173, R2=0.82). (5) The regression relationship between the yield of the secondary rainfall, sediment yield and the variation of the degree of variation in roughness and the slope gradient was superior to that of relationships built using the above-mentioned parameters singly. The results provided references for revealing the mechanism of the effects of slope gradient and surface roughness on soil erosion during the process of water erosion.