Twelve Secret Service agents were sent back to the U.S. after allegations of inappropriate conduct with prostitutes in Colombia.