Rory McIlroy is still having to defend himself from "blow back," a week after playing golf with US President Donald Trump.