Of the original castle, all that remains is a tower, at one time known as the Prison Tower, and in the 19th century used for sheep. In the 16th or 17th century, the castle was rebuilt in the Renaissance style.[2]
The castle overhangs the village slightly and forms a square protecting a central courtyard. The western wing, the oldest, forms the gatehouse which consists of a carriage entrance and a pedestrian door and leads to the 13th century tower. The northern wing which adjoins the church is the dwelling part which is attached to the east wing by a 16th-century inside tower.
As was the custom, the castle is connected to the church by a tunnel; a private door opens into the cemetery and the church has a seigniorial chapel. The Church of St. Martin of Lagery has a tombstone of the former lords, a litre funéraire and painted blazons.