Infrastructure is an important part in a country, so the utilization efficiency as an important index is worthy of attention. Based on the DEA-Malmquist index analysis approach in the paper, the temporal changes of urban infrastructure utilization efficiency from 2004 to 2014 in China mainland are evaluated. The laws for the temporal evolution and the reasons which caused the change of utilization efficiency are researched. The results indicate that in the past ten years the overall urban infrastructure utilization efficiency is on the rise, indeed it decreased firstly and then had a sustainable growth. The technical progress makes a great contribution to the overall efficiency, and the low technical efficiency is the main factor to restrict the infrastructure utilization efficiency. The utilization efficiency in eastern and central regions rose, but utilization efficiency in west region decreased.