Weatherby Castle is an Iron Agehill fort that encloses about 17+1⁄2 acres (7 hectares) on a spur of land about 3⁄4 mile (1.2 kilometres) south of the village. Its structure comprises two concentric enclosures, though parts have been damaged by cultivation and ploughing. Pieces of Roman ware were found within the site in the 19th century.[3]
There were originally two settlements within the parish: St Andrew to the south of the Dorchester-Blandford road, and Deverel to the north, though over time these coalesced into one settlement around where the road crosses the Milborne Brook. At the end of the 19th century St Andrew's ecclesiastical parish was enlarged by the addition of neighbouring Milborne Stileham to the south east (previously part of Bere Regis parish), though the civil parishes remained separate until 1933.[3]
There are a number of community facilities within the village, including an infants' school (Milborne St Andrew First School), a public house (the Royal Oak), a parish church (Church of England, dedicated to St Andrew and founded in 1069), and a sports club and pavilion, home of the Milborne Sports Football Club which was named Dorset FA Charter Standard Club of the Year in 2008.[8] Milborne Sports Football Club also supports two youth teams.