He was a member of first Minnesota Legislature. His term as Lieutenant Governor did not expire until a few weeks after the 2nd Minnesota State Legislative Session began. Although the Republican Party had a majority in the Minnesota Senate, Holcombe, a Democrat, presided over them until January 2, 1860 when the state officials were sworn in. Some of his rulings so frustrated the Republican majority that they asked the House to impeach him. The House responded that they had no right to interfere with the workings of the Senate and suggested they change their rules. The issue was resolved when Republican Ignatius Donnelly was finally sworn in as Lieutenant Governor.[3]