Born in Cape Town, he was the son of a prominent attorney, and he joined the family firm after qualifying at the University of Cape Town. During the Anglo-Boer War (1899–1902), he was a legal adviser to the British War Office.
In World War I (1914–1918), he was Deputy Director of War Recruiting, and served in the Cape Peninsula Garrison Regiment (a home defence unit).
Van Zyl entered politics in 1915, as a member of the Unionist Party. He was a member of the Cape Provincial Council (the provincial legislature) until 1918, and then a member of Parliament until 1942. He was Deputy Speaker from 1934 to 1942 (the Speaker being Dr Ernest George Jansen, who was later also governor-general).
From 1942 to 1945, Van Zyl was Administrator of the Cape Province. He served as Governor-General from 1945 until 1950. He was appointed to the Privy Council of the United Kingdom in 1945. In 1947, he hosted King George VI and the British Royal Family when they toured South Africa.
Van Zyl married Marie Fraser in 1900. He died in 1956.