Many Europeans living in Britain are worried they're on borrowed time despite having families or businesses in the UK.