Potter remained firm that he has the backing of the Chelsea board, but said he understands fan frustrations