Despite a recent public show of support from senior royals, scandal-hit Prince Andrew is said to be facing a lifelong ban from returning to public duties.