The congregation was formed in 1828. Construction of the church began in 1832, and was completed in 1839, with its consecration on October 12, 1839. The architect William Bulgin designed the building with early Gothic Revival style and built it using limestone.[4]
The church is administered by the Episcopal Church of St. Luke and St. Mary, formed by the January 2016 merger of the former parish of St. Luke's with St. Mary's in Belvidere.