Bajaj Finserv is foraying into infrastructure financing through its retail financing arm Bajaj Auto Finance.