Since then there has been a pandemic, swiftly followed by an energy crisis.

That has made it hard to decipher exactly what the impact of Brexit has been.

The latest data suggest a hit to the economy - but in some unexpected ways.

Trade

As the UK pulled out of the single market and customs union in 2021, companies trading with the EU faced new rules, new paperwork and new checks on some goods.

That prompted fears over what would happen to the £550bn of trade between the UK and its nearest trading partner.

There was an initial dip in the amount the UK exported to the EU. Once teething problems were dealt with, trade volumes, recovered to pre-pandemic levels, according to official figures. But it could be argued trade might have grown more if it hadn't been for Brexit.

When the British Chambers of Commerce surveyed 500 firms recently, more than half of them said they were still grappling with the new system. The red tape may have deterred some small exporters altogether. A study of customs classifications shows the variety of goods we export has diminished.

It is a similar story for imports - volumes have recovered to pre-pandemic levels. But academics at the London School of Economics point out that the price of food imported from the EU - the likes of tomatoes, or potatoes - rose, maybe by as much as 6% over 2020 and 2021. That was before the recent jump in inflation.

On the flipside, that has made it easier for domestic food producers to compete; economists say they may have had a £5bn boost.

It is the bigger picture, however, that is more eye-catching.

Most nations saw international trade collapse at the height of pandemic. Since then, the rest of the G7 countries have seen trade, when compared to the size of their economies, bounce back in a way that hasn't happened in the UK.

If you look at the UK's trade with the rest of the world, as well as trade with the EU, overall it has fallen relative to the size of the UK economy. Trade hasn't bounced back post-pandemic as fast as it has in other major nations, it has become less important in contributing to our prosperity. "Global Britain" has become less open. It is lagging behind.

Trade deals

What about new trade deals? They could help - but it is early days still.

In total 71 trade deals have been struck, which is swift progress, but the vast majority just replicate deals Britain had when it was part of the EU.

The UK has signed new deals with Australia and New Zealand - but they are only expected to deliver a tiny boost to trade and even that will take several years. Plus they are controversial - some UK farmers fear they will lose out.

Talks are still taking place with India and members of a trans-Pacific pact. They are taking longer than previous ministers had hoped - but analysts think that taking things more slowly may actually lead to more beneficial agreements.

Trade deals with some of the biggest players, such as the US and China, remain elusive.

Investment

How much businesses choose to spend on factories, training, equipment and technology, is also affected by our relationship with the EU. And the chancellor acknowledges that investment can turbocharge growth.

But investment has stalled since the referendum, as businesses remain wary of the outlook for the economy. Investment wasn't great even before 2016, but if it had continued its pre-referendum trend, analysis by the think tank the UK in a Changing Europe suggests it could be about 25% higher than it is now.

Economists argue about how to explain that gap. Some - including the International Monetary Fund - have suggested that uncertainty surrounding Brexit, including the unsettled issue of the Northern Ireland Protocol, have deterred at least some spending. Sir Richard Branson is among the business bosses who have suggested the cost of Brexit red tape would put them off investing in the UK.

The pro-Brexit group, Briefings for Business, claims that the numbers are misleading, and that there isn't evidence of a Brexit-related hit to investment.

Ultimately, however, a lack of investment means we are a less-efficient, lower-earning economy than we could be.

Jobs

Leaving the EU also meant changes to the rules on the free movement of labour and the introduction of a points-based immigration system That has prompted complaints from some unlikely quarters.

The chief executive of fashion chain Next, Lord Wolfson and Wetherspoons' boss Tim Martin both supported Brexit - but both have called for the UK to let in more workers.

A study by the think tanks Centre for European Reform and UK in a Changing Europe suggests that there are 330,000 fewer workers in the UK as a result of Brexit. That may only be 1% of the total workforce - but sectors such as transport, hospitality and retail have been particularly hard hit.

The boss of Wetherspoons supported Brexit but says the UK now needs to let in more workers from overseas

A lack of workers has resulted in shortages and pushed up bills for customers.

Some commentators argue these constraints will persuade businesses to boost staffs' skills and invest more.

Meanwhile, in the financial services sector, 7,000 jobs may have been lost, according to a House of Commons report, but that's far fewer than the 70,000 previously feared.

What next?

All of the above adds up to an economy that has fared less well amidst the recent upheaval than its peers. The UK is the only major rich economy that remains smaller - poorer - than prior to the pandemic and Brexit may be a factor.

Overall, the government's independent watchdog, the Office for Budget Responsibility, thinks the UK will ultimately be 4% worse off, than it would have been if we had voted no to Brexit - although for many voters, Brexit was more about sovereignty than the economy.

But there remains a lot to be settled.

It is not just the Northern Ireland protocol, but also permanent arrangements for industries like financial services, fishing and electric vehicle parts, cooperation on science and ways to reduce red tape.

There are potential gains there and realising them is a matter of political as well as economic strategy.