The two-and-a-half-story timber structure was refinished as Baby's residence in 1816. It is located at 221 Mill Street in Old Sandwich Town, which is the oldest part of Windsor, where settlements date back to the mid-18th century. The Duff Baby House is also one of the best-preserved and oldest Georgian-style houses in Ontario. Today the building is owned by the Ontario Heritage Trust and houses government offices.