He was born on April 29, 1871, in Lexington, Kentucky, to Franklin Waters Woolley (1845–1891) and the former Lucy McCaw (1844–1905). He married Marguerite Holmes Trenholm (1878–1936) in 1900 and had four daughters, Marguerite Trenholm Woolley (1901–1983), Lucy DeGraffenried (Woolley) List (1902–1993), Florence Trenholm Wickliffe (Woolley) McKee (1905–1997) and Frances Howard (Wolley) Robb (1914–2003). Frances was the mother of future governor of Virginia Charles S. Robb and mother-in-law of future first lady Lynda Bird Johnson Robb.
During President Wilson's 1916 reelection bid, Woolley was the chairman of the Bureau of Publicity for the Democratic National Committee and was credited with the successful slogan "He Has Kept Us Out of the War."[3]