The Priory for nuns was founded by Samson le Fort around 1138. He was not of a Bedfordshire family, but in right of his wife, Albreda de Blosseville, held Harrold as a life?tenant (for the Honour of Huntingdon) until the succession of her son. Samson was prompted to found Harrold Priory by his kinsman, Hilbert Pelice, a canon of Arrouaise, and so the Priory was at first a cell of Arrouaise, c.1145. The distance made the connection difficult, and in 1177 Arrouaise seems to have ceded its rights to Missenden Abbey. This also caused friction, and in 1188 Harrold's independence from Arrouaise was acknowledged and it became an Augustinian priory under the bishop.

Unfortunately the Priory did not escape the dissolution of the monasteries by King Henry VIII. In 1544 the site was granted to William Lord Parr, and again, in 1555, to John Cheney and William Duncombe. The only part of the convent building which remained was the refectory, later to become a barn and known by the name of the Hall-barn.