Prince Harry's polo matches in the US might rule him out of attending Queen Elizabeth's Platinum Jubilee.