He moved to West Virginia and was elected as a Democrat to the Forty-second, Forty-third, and Forty-fourth Congresses and served from March 4, 1871, until January 31, 1877, when he resigned. As a Representative, he was chairman of the Committee on Commerce (Forty-fourth Congress).
Hereford was elected as a Democrat to the U.S. Senate on January 26, 1877, to fill the vacancy caused by the death of Allen Taylor Caperton, but didn't qualify until he resigned from the U.S. House of Representatives. He served from January 31, 1877, to March 3, 1881. As a Senator, he was chairman of the Committee on Mines and Mining (Forty-sixth Congress).
After Congress
He resumed the practice of law and died in Union, West Virginia in 1891. Interment was in Green Hill Cemetery.