Iraqi forces have detained more than 500 suspected Islamic State group members trying to sneak out of the city of Fallujah by blending in with fleeing civilians, police say.