According to Jim S04 E06 Father-Daughter Dance
Jim gets upset when Ruby stops giving him a goodbye kiss outside school. Cheryl explains that Ruby is going through a phase, and Jim tries to respect that. But when Ruby starts rebelling against him, he has to try really, really hard.
