After graduating from law school, he went on to run his own law practice. and serve as legal advisor to the Providence Civil Service Commission as well as chairman of the Providence Fair Rents Committee in 1941.
In 1942 and 1943 he served as the first Assistant City Solicitor for Providence. After a break caused by his service in the US Army he retook this post.
He was elected the lieutenant governor in November 1946 and took office in January 1947. The governor at the time was John O. Pastore, a fellow Democrat. He was re-elected lieutenant governor in 1948.
As lieutenant governor and presiding officer of the state Senate in 1949, McKiernan cast the tie-breaking vote that resulted in the adoption of new rules effectively giving Democrats control of the Senate's legislative committees for the first time in more than a decade. He was re-elected to a third term as lieutenant governor in November 1950.
After Governor Pastore left office early to become a United States senator, following a special election to replace Senator J. Howard McGrath who was appointed United States Attorney General, McKiernan became governor and served for only 15 days from December 19, 1950, to January 2, 1951. His was the shortest term as governor in the history of Rhode Island. He resumed the office of lieutenant governor on January 2, 1951, under administration of another Democrat, Governor Dennis J. Roberts.
After winning re-election in 1952 and 1954, McKiernan resigned as lieutenant governor in 1956 to accept an appointment as an associate justice of the Rhode Island Superior Court.
Personal life
A Roman Catholic, he was married to Elizabeth St. Pierre. He is survived by his four children, Mary E. McKiernan, John S. McKiernan III, William P. McKiernan and Laura E. McKiernan.