A man in the crowd approached Mr Rajoy, reportedly asking for a picture with the politician, before he suddenly lashed out with his left fist, hitting the prime minister in the side of the head and throwing him off balance.
A video of the attack shows the young man with short dark hair and wearing a black jacket.
Spain's state-owned RTVE news channel reported on its website that the impact of the blow sent Mr Rajoy's glasses flying to the ground.
A spokesman said Mr Rajoy, 60, did not suffer any serious injury, but he appeared later with a deep red mark on his face.
The attacker was 17 years old, according to Spanish news reports.
"I'm glad I did it," he said, according to newspaper El Pais, before being arrested.
The assault took place in front of a church in the northern city of Pontevedra in Galicia, where Mr Rajoy grew up.
Rajoy Assaulted During Election Walkabout In Galicia
Spain's PM Rajoy punched in face on election trail
