A row has erupted in cabinet over plans to scrap a ban on the import and sale of fur and foie gras after No 10 was convinced to kill off policies that would outlaw the trade.

Lord Goldsmith, the environment minister and the ban’s main proponent, spoke out after his colleagues, including Jacob Rees-Mogg, persuaded the prime minister to ditch the proposals.

In response to those in cabinet who do not agree with the ban, Goldsmith said: “The public are strongly in favour of measures to curb the fur trade and foie gras.

“There are some who view the issue as a matter of personal choice, but no one would extend that principle to things like dog-fighting or bear-baiting, so it’s not clear why fur farming or force-feeding geese, which arguably are associated with far greater levels of cruelty, should be any different.”

There is hope among ministers who are in favour of the bill that it is not yet completely off the table, though sources say “that is the direction of travel”. The ban was supposed to be part of a raft of measures that would position post-Brexit Britain as a world leader on animal welfare.

Sources in the Department for Environment, Food and Rural Affairs (Defra) said the ban had been removed from the animals abroad bill in an attempt to scrap the policy altogether. The production of fur and foie gras products are banned in the UK.

The Guardian understands that Rees-Mogg made the intervention as he does not believe banning things to be Conservative in nature.

This is a further sign of the struggle for Boris Johnson’s ear being played out by the more libertarian side of the Tory party and the more ecologically minded. Some on the rightof the party believe such bans are against free-trade principles, and that the purchase of such goods should be a matter of choice. Others believe the ban could win over swing voters and show the Conservative party is modern and caring.

However, there are even splits on the matter within the European Research Group, often thought of as the hub for the party’s libertarian right wing. A leading member said: “I have made no representations to Boris on this matter. I personally stopped eating foie gras when I learned how it was made.”

Yesterday, a number of Conservative MPs said they would demand an explanation from the prime minister, who has previously vowed to use Brexit as an opportunity to take a stand on animal welfare.

The ban was mentioned in the government’s action plan for animal welfare and aimed to bring swing voters on side. The 2019 manifesto, which referred to the action plan, also committed to not compromising on animal welfare standards in international trade.

A Defra spokesperson said the government was still “fully committed to animal welfare”.

The shock decision to remove the ban from the animals abroad bill is likely to upset the prime minister’s wife, Carrie Johnson, who has been vocally supportive of such measures.

She has previously: “Anyone who wants to buy fur really is sick. Fur belongs on animals only. Nuts to think some of Britain’s department stores still sell fur.”

The decision is thought to have been taken while Carrie Johnson was away at the weekend, as she was seen returning to No 10 with her children and some suitcases on Monday morning.