The growth condition of crop trees as well as the entire forest carbon storage can be affected by thinning. Studying the long-term effects of thinning on the carbon sequestration of plantations contributes to accurately evaluate the carbon sink function and the scientific management of plantations. The Pinus tabulaeformis plantations constructed in 1962 on the Loess Plateau forest area, were thinned to 4 different densities: 800 (Heavy), 1 500 (Medium), 2 200 (Light) and 2 900 (Unthinning) stem/hm² in 1985. In August 2016, the effects of thinning on different vegetation layers, leaf litter, coarse wood debris and soil organic carbon pools were studied. The results showed that the total carbon storages of the Pinus tabulaeformis plantations under Light and Medium thinning were significantly increased by 28.54% and 21.33% respectively compared with the control, but no significant difference observed between the Heavy (154.66 t/hm²) and the control (169.26 t/hm²). The arbor layers were the main carbon pools of plantations, accounting for 64.85%~74.62% of the total carbon storage. Under different treatments, the carbon storage of stems in the arbor layers was the highest (52.05%~56.43%), followed by the roots and branches (22.27%~22.60% and 17.73%~18.32%), and the lowest in leaves (3.56%~7.01%). While Medium and Heavy thinning improved the diversity of shrubs and herbs, the total carbon storages of shrubs and herbs under those were 24.27% and 25.24% respectively higher than that of the control. Thinning significantly reduced leaf litter, which under heavy thinning declined to 48.15% of the control. The changes of soil carbon pool were mainly caused by the disturbance of topsoil 0—20 cm. The surface carbon storages under Medium and Heavy thinning fell by 17.68% and 33.76% respectively compared with the control, while the difference between Light thinning (51.23 t/hm²) and the control (50.96 t/hm²) was not significant. There was a significant negative correlation between soil surface carbon storages and plant diversities and basal respirations. In conclusion, Light and Medium thinning could increase carbon sequestration in forest ecosystems, and Light thinning was not only conducive to carbon sequestration in above-ground vegetation but also contributed to the maintenance of soil carbon pools.