The business secretary, Kwasi Kwarteng, has played down the prospect of significant tax increases to start reducing the deficit in the public finances in Wednesday’s budget.

While the chancellor, Rishi Sunak, has said the country cannot not “go on spending money for ever”, Kwarteng said the priority for now was to support the economy.

“For now, what we have to do is support businesses, individuals, families, through what has been an extremely difficult time,” he told BBC Breakfast. “We have got another three years to run in the parliament and the chancellor will be looking to reduce the deficit. For now, I think the real emphasis is on trying to provide critical support.”

The chancellor is said to be considering raising corporation tax to as much as 25% from 19%. Kwarteng told BBC Breakfast that Sunak had “already indicated” that he would be extending the furlough scheme.

On Monday, the former Tory leader William Hague said taxes would “have to go up”. Writing in the Daily Telegraph, he said: “It pains me to say, after spending much of my life arguing for lower taxes, that we have reached the point where at least some business and personal taxes have to go up.” He added that those who opposed some form of tax rises in the current climate were buying into “dangerous illusions”.

Asked about whether he thought there would be higher taxes on companies that had “done well” out of the pandemic, Kwarteng said: “I am a Conservative, and I’ve always been a low-tax Conservative. I think that the best way to grow an economy, generally, is to promote enterprise, small business and larger businesses as well, but at a time like this, our focus has been on trying to keep things going and providing vital support to families and businesses, at a very difficult time.”

He told the BBC Radio 4’s Today programme he was sure in the future there would be a “debate about the right level of taxation”, but for now the focus was on providing support.

Kwarteng would not be drawn on whether businesses could expect an extension of the business rates holiday, saying only: “I think there will be continued support. I think that will be perhaps an extension of the VAT holiday.”

It comes as Sunak is expected to provide more than £400m of additional support for the culture sector to help museums, theatres and galleries in England to reopen once coronavirus restrictions start to ease in the coming months.

Sunak is expected to put an extra £300m into the £1.57bn culture recovery fund as part of the measures.

In teasers before the budget, Treasury officials also said Sunak was preparing to give a “significant chunk” of a £300m sports recovery package to help cricket fans return to stadiums this summer.

He will also announce a £150m community ownership fund to allow pubgoers to bid for up to £250,000 to save their favourite local.

National museums and cultural bodies will also receive £90m to help keep them going until they can open their doors on 17 May at the earliest. Another £18.8m will be provided for community cultural projects, while an additional £77m will be given to the devolved administrations in Scotland, Wales and Northern Ireland to provide their culture sectors with similar support.

Sunak said: “This industry is a significant driver of economic activity, employing more than 700,000 people in jobs across the UK, and I am committed to ensuring the arts are equipped to captivate audiences in the months and years to come.”