Protesters demanded yesterday that a Conservative MP should hand over his 621-acre sugar plantation to the people of Barbados as compensation for his family’s 200 years of slave owning and trading on the island. Richard Drax, the MP for Dorset South, has said the role of his ancestors was “deeply, deeply regrettable” but resists demands for reparations.

As part of this year’s Tolpuddle Festival, a rally organised by Stand Up to Racism, Dorset, at the gates of the Drax family estate highlighted the family’s historic role in slavery. The festival celebrates the Tolpuddle Martyrs, poorly paid farm workers, who were transported in 1834 for organising trade union activities.

This is the first time the festival has worked with reparation activists. Several hundred campaigners attended yesterday’s “It’s Time, Mr Drax” rally on the hottest day of the year so far.

Frances O’Grady, the general secretary of the TUC, told the rally that Drax had a moral duty to pay reparations to Barbados. “Mr Drax should pay up now,” she said, citing the violence, rape and murder that accompanied Caribbean slavery.

The diverse and friendly, if noisy, march went round part of the “Great Wall of Dorset”, a three-mile brick wall that surrounds Drax’s home at Charborough Park, eight miles from Tolpuddle.

Last December the Observer and Sunday Mirror revealed that Drax personally controlled the Drax Hall plantation in Barbados, which his family has owned since 1650 and was worked by slaves for 200 years. We also revealed Drax – who is said to enjoy a Downton Abbey lifestyle – is the wealthiest landowning MP, worth at least £150m, with an estate of 22.5 square miles of Dorset and 125 properties including his 17th century mansion.

Nigel Costley, secretary of the South West TUC, told protesters: “No one can be held responsible for the actions of their ancestors. But Richard Drax is still sitting on the mountain of gold gained from the horrors of slavery. It’s time he paid some back.”

Speaking from Barbados, the Labour MP Trevor Prescod said: “Drax Hall witnessed terrible pain and suffering – and people today still experience disadvantage that is the outcome of generations of slavery.

“We ask Mr Drax to show moral leadership and to discuss with us how to address the legacy of chattel slavery.”

Grafton Shaker of Dorset Unite, who grew up in Barbados, said that as children they were taken to plantations for days out, “but we were never told about the horrors of slavery”.

The rally organisers say they repeatedly asked Drax to meet them so they could hand him a statement. They have had no response.