Boris Johnson has declined an opportunity to apologise to the victim of Imran Ahmad Khan after the issue was raised in Parliament by Labour MP Jeff Smith.