Treasurer Angus Taylor says the government has to stop unnecessary spending in order to contain interest rates.