"At this point, I'm starting to think I'm being attacked for who I am and no answer is the right answer," Hilaria Baldwin said