The limited rainfall and low water use efficiency are the main causes of the low and unstable crop productivity in arid agricultural area of the Loess Plateau in Long Zhong. Developing conservation tillage practices is an important way to protect water and soil resources and improve water use efficiency. The aim of the study was to determine the underlying mechanisms of different tillage practices in improving yield and water use efficiency. The effect of different tillage practices on soil evaporation, field water consumption, crop transpiration, ratio of evapotranspiration to evaporation, yield and water use efficiency were studied from 2015 to 2016. The study was conducted in a long-term tillage experimental field located in the arid farmland of Long Zhong. The experiment included six treatments, which were conventional tillage with no straw (T), no-till with straw cover (NTS), no-till with no straw cover (NT), conventional tillage with straw incorporated (TS), conventional tillage with plastic mulch (TP) and no-till with plastic mulch (NTP), all the fields were used for the rotation of spring wheat and pea. The results showed that: (1) Compared with T treatment, the evaporations of spring wheat and field pea in NTS, TP and NTP treatments significantly reduced by 6.52%~50.81% during the whole growth period, and the effect of NTS to reduce the evaporation was mainly after the flowering of wheat and the pod of the pea, and the plastic mulching could significantly reduced the evaporation in each growth period. (2) NTS treatment had no significant effect on water consumption during the whole growth period, and the water consumption in NTS treatment was significantly higher than that in T treatment in wheat fields. Compared with traditional tillage, NTS treatment significantly increased the amount of water consumed and the proportion of total water consumption in the stages of wheat flowering to harvest and pea podding to harvest. (3) NTS, TP and NTP significantly increased the transpiration of spring wheat and pea, reduced effectively the proportion of evaporation to evapotranspiration in the field, and the ineffective loss of water. (4) Compared with T treatment, the yield of spring wheat and pea in NTS, TP and NTP treatment increased by 7.64%~62.79% in each year, and the water use efficiency increased by 0.43%~50.88%. Therefore, the conservation tillage measures, such as no-tillage straw mulching and plastic mulching, could improve the water use efficiency and yield of wheat and pea in the arid agricultural area of the Loess Plateau in Long Zhong. No-till with straw cover could improve the water use efficiency and yield of spring wheat and pea by reducing the evaporation and increasing the water consumption in the later period of crop growth, and reducing the ratio of evaporation to evapotranspiration. The plastic mulching was mainly to reduce the evaporation during the whole growth period, increase the total transpiration of the crop and reduce the proportion of evaporation to evapotranspiration, so as to realize efficient use of water and increase crop yield.