In March, 1309, Robert the Bruce stayed there while presiding over his first Parliament. James I received his education there as did his son, James II.
Little remains of the stone castle today that was built by Bishop Roger around 1200. The present complex dates from the late 14th century.
Bishop Trail (1385-1401) rebuilt, but his residence was subsequently altered, mostly during the first half of the 16th century, a time of growing tension between church and state.
A crisis began upon the death of Archbishop Alexander, illegitimate son of King James IV, on the battlefield of Flodden in 1513. His untimely death led to a scramble. The victor, Archbishop Forman, was swiftly succeeded by Archbishop James Beaton. He transformed Bishop Trail's castle into a formidable artillery fortress with towers bristling with guns.
In 1538, James Beaton had been succeeded by his nephew, David Beaton, who later became a cardinal. He rejected Protestantism and was unyielding in his opposition to the proposed marriage of Mary Queen of Scots to the son of Henry VIII of England.
In March 1546, Beaton had the Protestant preacher burned at the stake, right in front of his castle. This act only made him more enemies. A band of Protestant lairds contrived to get into his fortress disguised as stonemasons. Once they got into the fortress, they seized and murdered the cardinal, dangling his naked corpse from the battlements.