Kalyanji was born to Virji Shah, a Kutchi businessman in Kundrodi, Kutch, Gujarat, who migrated from Kutch to Mumbai to start a Kirana (provision store). His younger brother and his wife are the husband and wife duo Babla & Kanchan.
He and his brothers began to learn music from a music teacher, who actually knew no music but taught them in lieu of paying his bills to their father. One of their four grand parents was a folk musician of some eminence. They spent most of their formative years in the hamlet of Girgaum (a district in Mumbai) amidst Marathi and Gujarati environs — some eminent musical talent resided in the vicinity.[citation needed]
Kalyanji's breakthrough was with the theme entitled Been music from the film Nagin (1954).[4]