The Old Castle at Crom was built in the early 17th-century by a Scottish planter, Michael Balfour. The Castle and the Estate passed to the Crichton family in 1655 when Abraham Crichton married the daughter of the previous tenant, the Bishop of Clogher.
The castle survived two sieges by the Jacobites in 1689, but accidentally burnt down in 1764. According to a tradition, the fire took place when Abraham Crichton was returning in his boat from a housewarming party at Florence Court. Crichton saw a worrying glow in the sky to the south, and returned home to find his castle gutted.
In the 19th-century, additional walls and towers were added to the ruins of the Old Castle for romantic effect. Today, the Old Castle ruins are a secluded monument.