The Ministry of External Affairs on Friday said the government has cancelled the passport of self-styled godman Nithyananda and also rejected his application for a fresh one.