After being offered a job as assistant rector of Trinity Church, Boston, Parker was ordained deacon on February 24, 1774 and priest three days later on February 27, in London. He began as assistant rector at Trinity in November 1774, becoming rector in 1779. After the Revolution, he helped build churches with the Society for the Propagation of the Gospel.
In 1803, Parker was unanimously elected second bishop of Massachusetts. He was consecrated September 14, 1804, in Trinity Church, New York, but developed gout and never served in the post. He died in Boston on December 6, 1804.