On May 2, 1906, Kelsey was appointed by Governor Frank W. Higgins to a three-year term as Superintendent of Insurance, and resigned the comptrollership. Early in 1907, Governor Charles Evans Hughes asked Kelsey to resign, but he refused. The governor then asked the New York State Senate to remove Kelsey on the ground that "while honest he utterly lacks in force and initiative", but after a lengthy hearing in the Judicial Committee, Kelsey was upheld by a vote of 27 to 24 on May 3, 1907. Then Governor Hughes appointed Matthew C. Fleming a Special Commissioner to examine the Insurance Department, and on February 2, 1908, Fleming declared Kelsey "unfit for the office" in his report to the State Senate, but Kelsey was maintained in office by an even larger majority.
Eventually, Kelsey resigned from the Insurance Department to be re-appointed First Deputy Comptroller by Charles H. Gaus on January 1, 1909, and acted as Comptroller after Gaus's death until the appointment, on November 11, of Clark Williams to fill the vacancy. A week later, Kelsey was forced to resign as First Deputy Comptroller.
Personal life
He died on August 20, 1934, in Perry, Wyoming County, New York, after complications from a fall, and was buried in Geneseo.
Sources
[1] The Rapid Transit Bill, mentioning Chairman Kelsey with wrong middle initial "C.", in NYT on April 12, 1901
[2] Speculation about Gilman's imminent resignation, in NYT on December 29, 1902
[3] Denial of Gilman's resignation in NYT on December 30, 1902
[4] The speculation about Gilman's resignation continues, in NYT on January 13, 1903