Khloé Kardashian explained that the birthday bash was "family only" since they "aren’t comfortable being around too many people" during the pandemic