Hirsch was born on March 15, 1935 in The Bronx, New York.[1] He studied at DeWitt Clinton High School and at the City College of New York. Hirsch was married to Elissa Hirsch from 1956 until they divorced in 1958. Then he was married to Bonni Sue Chalkin from 1992 until they divorced in 2005. He has three children.