James Crawford, who had served the community of Martinsville as justice of the peace and built the first brick church in the town, had owned the property where the house was built since 1837; he was a "wheelhorse" who helped lead the local Whig Party into competitiveness against the majority Democratic Party of Morgan County.[4] In 1862 he and his wife built the house, only to sell it later that year to Israel Gilpin, who operated one of Martinsville's first industries, a wool mill complementing the previous Martinsville industries of pork packing and shipping.[5]
In 1870 the Gilpins sold the house to a farmer named John Buckner, who in turn sold it in 1889 to the Clapper family who held it until 1926. Car dealers Howard and Doris Daily owned it from 1926 to 1930, losing it due to Howard wagering it in a poker game.[5] It then moved to the Sedwick family until 1945, when it was sold back to Doris Daily. Since Daily's death, it has changed ownership thrice, most recently in 2007.[6]
The Crawford-Gilpin House is a two-story brick and limestone Italianate structure on a brick foundation. A carriage house built with the house also still stands. The Morgan County Historical Preservation Society has awarded the current owners a grant for fixing storm shutters and porch trim.[7]