The castle was a Scottish royal castle from the 12th century to the 16th century AD