Premier Daniel Andrews says he will watch case numbers closely this week prior to rolling back restrictions again.