A source previously revealed that Charles "always envisioned working with both of his sons and their families"
