The man, whom former US President Barack Obama privately anointed as the successor in global diplomacy, barely survived a blackface scandal