The palace, which was ruined and rebuilt multiple times due to wars, dates back to the 15th century. According to various historical sources, the palace was a Franciscan monastery, a powiat court, a merchants' house and a city firefighters' headquarters. In the 16th century, the palace was used as the residential building of a grand duke. According to L.Potockis, in the 16th century, the palace was used by Lithuanian and Polish heads of state while visiting Kaunas, due to floods at Kaunas Castle.
At the end of the 19th century, Russian province architect S.Gorskis was ordered to convert the building complex into a city hospital. Starting in 1951, the unique and historical building complex was used as the Kaunas psychiatric hospital with a capacity of 200 patients.