"I mean, I saw it with Hannah last season, seeing her fall for someone that wasn't there for the right reasons" said Peter