Prime Minister Sir Keir Starmer was subjected to abuse and boos by a section of the crowd during a visit to the St Leger.