In 1694 he formally laid claim to the dormant barony and, following a favourable ruling in the House of Lords, he assumed the title of 11th Baron Willoughby de Broke and 19th Baron Latimer in 1695. He married twice, firstly to Mary Pretyman, daughter of Sir John Pretyman of Leicestershire, and secondly to Frances Dove, daughter of Thomas Dove of Upton, Northamptonshire. By his first wife he had four children (Mary, John, George and Thomas), and by his second wife a daughter (Diana).
On his death on 18 July 1711 he was buried in the chapel at Compton Verney and the title passed to his second son George (his eldest son John, MP for Leicestershire, having died childless in 1707).