Walker grew up poor in Catskill, New York and had five older siblings. He played semi-professional football and baseball in the Hudson Valley before joining the Zulu Cannibal Giants. In 1936, he signed with the Homestead Grays for $150 per month (equivalent to $3,294 in 2023).[2] He was the starting pitcher for the East at the 1938 East–West All-Star Game. He took a year off from baseball in 1942 and worked at a Baltimore shipyard.[3] He retired from baseball in 1945 due to a sore arm and moved to Albany, New York. The baseball field at Albany's Bleecker Stadium was later named in his honor.[2]
