On 16 May 1554, because he had made poor decisions about his properties in his youth, Lord Elphinstone put his affairs in the hands of his grandfather Lord Erskine, his father-in-law John Drummond of Innerpeffray, and his brothers-in-law Robert Drummond of Carnock and John Hamilton of Haggs. This transaction was enacted before Mary of Guise and the Privy Council of Scotland in her presence chamber at Stirling Castle.[3] Subsequently, the Elphinstones and Erskines continued a feud.
In May 1565, his brother-in-law Alexander Drummond of Midhope Castle was charged with conspiring with Lady Elphinstone to defraud Lord Elphinstone of his income over the past three years. However, there does not seem to have been a trial, or any further evidence of bad feeling between Elphinstone and his wife and the Drummonds of Midhope.[4]
In 1577 he was said to be a Catholic and still a supporter of the exiled Mary, Queen of Scots.[6] In that year he transferred his estates to his eldest son, Alexander, Master of Elphinsone.[7]
Lord Elphinstone was in London in 1592 and returned in November.[8]
Robert, Lord Elphinstone, died on 18 May 1602 and was buried in the kirk of Airth.[9]