The Business Secretary says we are all worse off due to the recession, with the decline in house prices playing its part.