The castle was built on land owned by the St Martin family. When Sir Lawrence de St Martin died in 1385 it pased to John, the fifth Baron Lovell for reasons unknown.
It was built using locally quarried Tisbury greensand, with William Wynford as the master mason, after Baron Lovell had been granted permission by Richard II in 1392.
The castle was inspired by the hexagonal castles then in fashion in parts of the Continent, particularly in France. Its six-sided design is unique in Britain, as is its inclusion of several self-contained guest suites.
After the fall of the Lovell family following Francis Lovell's support of Richard III, the castle was confiscated in 1461 and passed through several owners until being bought by Sir Thomas Arundell of Lanherne in 1544. The Arundells were of the ancient Cornish family with wide estates in Wiltshire.
The castle was confiscated when Sir Thomas, a Roman Catholic, was executed for treason in 1552. It was later acquired by his son, Sir Matthew Arundell in 1570.
In the 1570s, most of the original medieval windows and doors would have been replaced. In the centre of this interior courtyard, there would have been a well. Evidence gathered from other castles from the same era suggests that there would have been an elaborate and impressive roof over the well, carved and painted with the emblems of the Lovells and the Arundells.
The Arundells, led by Thomas Arundell, 1st Baron Arundell of Wardour were some of the most active of the Catholic landowners in England at the time of the Reformation;. They were Royalists during the English Civil War. During that conflict, Thomas Arundell, 2nd Baron Arundell of Wardour was away from home on the King’s business and had asked his wife, Lady Blanche Arundell, aged 61, and with a garrison of 25 trained fighting men to defend the castle.
On 2 May 1643 Sir Edward Hungerford, with 1,300 men of the Parliamentarian Army, demanded admittance to search for Royalists. Lady Blanche refused and laid siege to the castle, turning his guns on the walls and attempted to undermine the walls by mines. After five days the castle was threatened with complete destruction and Lady Blanche agreed to surrender. The castle was placed under the command of Colonel Edmund Ludlow.
Lord Arundell had died of his wounds after the Battle of Lansdowne. His son, Henry 3rd Lord Arundell, laid siege to his own castle, blew up much of it and caused the Parliamentary garrison to surrender in March 1644.
The family slowly recovered power , through the English Commonwealth and the Glorious Revolution, until the eighth Baron, Henry Arundell, borrowed funds to finance the rebuilding of the castle. This was done by the prominent Palladian James Paine.
Paine built Wardour New Castle, but left the Wardour Old Castle as an ornamental feature. The New Castle is not a castle at all, but a symmetrical neoclassical country house with a main block built around a central staircase hall and two flanking wings. Paine integrated the ruins of the Old Castle into the surrounding parkland, intending it to be viewed as a romantic ruin.
The castle's ground level was altered around the 18th century. In the Middle Ages, it sloped away much more steeply so that the building stood on the end of a low ridge of land. The approach to the main front door is supposed to have been protected by a wide ditch crossed by a drawbridge, with a portcullis, though there are no surviving remains of these features. Between the towers at the level of the battlements are the remains of a projecting gallery or barbican which would have been used to defend the front entrance.
Above the portal over the front entrance is the Arundell coat-of-arms and a description of the Arundell's possession of Wardour, erected by Sir Matthew Arundell in 1578 to celebrate his recovery of the property after the family lost it when Sir Thomas Arundell was executed in 1552. Above the coat of arms is the head of Christ in a niche with the inscription: Sub nomine tuo stet genus et domus.
When the south-west side of the castle was blown out in an explosion in the 17th century, the courtyard was changed from a dark to a light, spacious area. It would have been in the shape of a hexagon, and there would have been four or five storeys formed around it on all sides.
The Grotto of Old Wardour Castle was the last addition to the landscape. It was built in 1792 by Josiah Lane of Tisbury, who at the time was a well-known builder of garden ornaments and other grottos in the area. He was commissioned to build the artificial cave, complete with dripping water, fossils and ferns from brick, plaster and stone from the ruins of the castle.