Kipp was elected as a Democrat to the Sixtieth Congress. He was not a candidate for renomination in 1908, as he instead chose to run for Pennsylvania State treasurer, for which he was unsuccessful.
He resumed his former business pursuits until the 1910 congressional election when he was once again elected, serving in the Sixty-second Congress until his death, on Vancouver Island, British Columbia. He was interred in Oak Hill Cemetery, Towanda, Pennsylvania.