Lord Amherst married Gertrude, daughter of the Right Reverend Hugh Percy, in 1834. They had six sons and five daughters, including William Amherst and Josceline Amherst. He died at Montreal Park, Sevenoaks, Kent, in March 1886, aged 80. He was succeeded in his titles by his eldest son, William. The Countess Amherst died at Rutland Gate, London, in April 1890, aged 75.[2]