“This experience has taken a huge emotional and physical toll on both of them," a source tells PEOPLE of Lori Loughlin and Mossimo Giannulli
