Defeated in his re-election bid, he returned to the Pennsylvania State House of Representatives (1824–1830; 1834 and 1835).
Death and legacy
He died on February 3, 1849, and is interred at the Saint Aloysius Cemetery in Littlestown, Pennsylvania.
One of his sons, James McSherry Jr. was a lawyer and writer best known for his "History of Maryland", another, William McSherry was a Pennsylvania State Senator as a Whig and Democrat. His grandson, James McSherry became chief judge of the supreme court U.S. State of Maryland Court of Appeals.