He was born in Johanngeorgenstadt.[2] In 1911, his family moved to Dresden. Schäfer's sporting career started in swimming. After school, he trained as a baker. He first tried rowing in 1929 when a rowing club in the suburb of Blasewitz made its club house available for a dance; he started training in March 1929.[3]