More than six out of 10 voters believe Brexit has either gone badly or worse than they expected – a year after the UK left the EU, according to an anniversary poll for the Observer.

The Opinium survey – coming a week after the minister in charge of Brexit, Lord Frost, resigned from Boris Johnson’s government – also found that 42% of people who voted Leave in 2016 had a negative view of how Brexit had turned out so far.

26% of Leave supporters said it had gone worse than they expected, while 16% of those who voted for Brexit said they had expected it to go badly and had been proved right.

Among people who voted Remain, 86% said it had gone badly or worse than they expected. Overall, just 14% of all voters said Brexit had gone better than expected.

Adam Drummond, of Opinium, said the most striking finding was that Leavers were now more hesitant about the virtues of Brexit than previously.

“For most of the Brexit process any time you’d ask a question that could be boiled down to ‘is Brexit good or bad?’ you’d have all of the Remainers saying ‘bad’ and all of the Leavers saying ‘good’ and these would cancel each other out,” he said.

“Now what we’re seeing is a significant minority of Leavers saying that things are going badly or at least worse than they expected. While 59% of Remain voters said, ‘I expected it to go badly and think it has’, only 17% of Leave voters said, ‘I expected it to go well and think it has’.

“Only 7% of Remainers think Brexit has gone better than expected versus 26% of Leavers saying it has gone worse than expected. So instead of two uniformly opposing blocs, the Remain bloc are still mostly united on Brexit being bad while the Leave bloc are a bit more split.”

The poll comes ahead of the introduction on 1 January of full customs checks on goods being exported from the EU to the UK, which business leaders believe could deter some smaller operators – such as food exporters – from supplying UK retailers as their costs and paperwork increase. These will be followed by more checks on food imports from mid-summer.

Full customs checks on goods being exported from the EU to the UK come in next year.

Shane Brennan, chief executive of the Cold Chain Federation, said he expected many of the problems faced over the past year by small UK businesses exporting to the EU, particularly the rising cost of sending small amounts because of new charges, would now confront those sending small specialist consignments the other way, from the EU to the UK.

He warned that this could lead to less availability of specialist food products from the continent arriving in UK shops. “Small traders have a choice, find a way to send more less often, or don’t send it at all,” Brennan said. “For lots of businesses you can’t justify sending a lorry load of fresh food a day or week and so you won’t do it. The net result is less variety, less fresh, quality specialist goods on the shelf, from outside the UK anyway.”

His fears were echoed by Dominic Goudie, head of international trade at the Food and Drink Federation, who said: “As new trade barriers are introduced, it is inevitable that businesses will experience issues at the border. In some cases, this could result in significant barriers that risk blocking deliveries from EU suppliers altogether, at least temporarily, while businesses adjust to the new requirements or restructure their supply chains.

“For just-in-time supply chains, this presents a real risk which could disrupt the operation of UK supply chains where a critically important ingredient is delayed or fails to arrive. Such delays could also lead to other ingredients already at factories being unusable.”