Inverlochy Castle was built circa 1270-1280 by John "the Black" Comyn, Lord of Badenoch and Lochaber, and chief of the Clan Comyn. It may have been built on the site of an earlier Pictish fortification and settlement, which the historian Hector Boece (1465-1536) records as a "city" that was destroyed by Vikings. When Robert the Bruce succeeded to the Scottish throne in 1306, the Comyns, his rivals for the crown, were dispossessed, and the castle was unoccupied for a time. In 1431, clansmen of Alexander MacDonald, Lord of the Isles, defeated King James I larger army in the first Battle of Inverlochy, fought close by the castle.
In 1505, the partially ruined castle was granted to George Gordon, 2nd Earl of Huntly, who was charged by King James IV with repairing the castle for use as a Royal garrison. In 1645, the castle served as a stopping-off point for the royalist army of James Graham, 1st Marquess of Montrose during his campaign against the Covenanter forces of the Marquess of Argyll. This culminated in a victory for the royalists in the second Battle of Inverlochy, on February 2, 1645. In the 19th century, the estate was bought by James Scarlett, 1st Baron Abinger, who built a Scottish baronial style mansion to the north-west, which is now the Inverlochy Castle Hotel. Minor enhancements, including the restoration of loops and battlements, were carried out by Lord Abinger in advance of the visit of Queen Victoria in 1873.
Inverlochy is now a ruin, but is unusual because it has remained unaltered since it was built in the reign of King Alexander III. The castle is sited on the south bank of the River Lochy, at the strategically important entrance to the Great Glen, a key passage through the Scottish Highlands. With one side defended by the river, the castle's other three sides were originally protected by a water-filled ditch. Inverlochy is a castle of enceinte, with its main defence being a substantial curtain wall. The simple layout comprises a quadrangular courtyard, 31 by 27 metres (100 ft × 89 ft) across, surrounded by a wall up to 2.7 metres (8.9 ft) thick and up to 7.6 metres (25 ft) high, with round towers at each corner. The largest of these, known as the Comyn Tower, is 6.1 metres (20 ft) across the interior, and served as the castle's "donjon", or keep. The smaller towers are 14 feet (4.3 m), and all four have stairways curving up within the thickness of the walls. The main entrance was to the south, with a "water gate" facing the river to the north. Both entrances were defended by a portcullis, and the south door may have had an interior gatehouse.