Premier Chen Chien-jen has stepped down along with his Cabinet to begin the standard caretaker period before the new Lai Ching-te administration takes over on May 20.