The FBI on Wednesday issued a public statement, expressing "grave concerns" over the memo that alleges abuse by the bureau and the Justice Department.