The Tottenham boss believes the speculation isn't fair, as his side target a UEFA Champions League place.