The mayor's office says the structures were illegal, but the owners insist they had the right to be there.