Commonwealth Games Australia CEO Craig Phillips expressed his anger, and criticised the Victorian Government