Climate change has resulted in increased vulnerability of smallholder farmers in marginal areas of Zimbabwe where there is limited capacity to adapt to climate change. This review analysed the history of soil and water conservation in Zimbabwe, efforts of improving water harvesting in the post-independence era, farmer-driven innovations, water harvesting technologies from other regions, and future directions of water harvesting in semi-marginal arid marginal areas. This review showed that the blanket recommendations that were made on the early conservation method were not suitable for marginal areas, as they resulted in increased losses of the much needed water.