The castle was probably built in the 12th century as the home castle of the Aspermont family. The Aspermont family were ministerialis, unfree knights in service to a higher noble, in service to the Bishop of Chur. Ulrich and Schwicker von Aspermont were first mentioned in 1149.[1] The family had ties to the locally powerful Lords of Tarasp and to the Hohenstaufenkings. In 1170 two of Schwicker's sons, Ulrich and Heinrich, were two of the three witnesses when the Bishop bestowed a title on the son of the king. Members of the family also served as the vice-dominus under the bishops. Around the mid-13th century the family had managed to acquire, through marriage and carefully using the rivalry between the bishop and local nobles, titles to Jenins, Maienfeld and Prättigau. They moved to Neu-Aspermont Castle in Jenins and Fracstein Castle in Prättigau and sold Alt Aspermont to the bishop.[1]
By 1275 the castle was owned by the bishop and had been granted as a fief to Walter IV von Vaz. They held the castle for less than a decade before Walter's death in 1284. Around 1288 the Abbot of St. Gall, Wilhelm of Montfort, fled the anti-abbot, supported by Rudolf I of Germany, to Alt-Aspermont. During an uprising in 1395-96 the castle was besieged by the bishop's forces. In 1452 the castle was besieged by the League of God's House for around a year. The last bishop's bailiff was Jorg Rink von Baldenstein, who ruled over Trimmis and Zizers until the Ilanz Articles were accepted in 1526. The Articles strictly limited the bishop's worldly power and removed the need for a bailiff at the castle. It was abandoned, but remained in good condition at least until 1548. During the Bündner Wirren in the early 17th century, the castle was occupied by Austrian forces in 1622, but again abandoned shortly thereafter. On 11 April 1878 the main tower of the castle collapsed into the valley below.[2]