High above the gorge of the Sirmiano river, there are the ruins of Castel Payersberg.
The castle complex has been built at the beginning of the 13th century. The members of the Payr family were ministry officials of the Counts of Tyrol.
The donjon of the castle is located at the part that could be assaulted and can be accessed from the second floor via a portal. The donjon itself stretches over four floors, the third floor used to be inhabited. On the southern side of the tower there was a little palace, which has deteriorated in the course of time. In the east there was a castle chapel, dating back to the late Gothic period. Today only remnants are left.
In the course of the 16th century the castle was renovated under the knight Jakob von Boymont-Payersberg, who had appropriate fortification built. Also the entrance gate and the corner rondel in the north were added to the building. Above the gate there was an emblem illustrating the year 1560. Later on, in 1600, a certain Marx Sittich reports on the country and the fact that the castle kept deteriorating and that it was hardly inhabitable.
For this reason the Schwanburg Castle was built, in order to make living more pleasant. Only when the last count of Payersberg died in 1791, the possession was passed on. Today the Malpaga family lives in the castle, who also uses it as building for farming and farm.
Part of Payersberg Castle is also the so-called hunting tower in Sirmiano di Sopra, at the highest located farm of Nalles, the Rainerhof farm. This tower was mainly a summer residence and is also illustrated on the municipal coat of arms of Nalles.