Labour called for the tax-cutting measures to be ditched after they sparked a fall in the pound and a surge in borrowing costs.
Government departments are also being asked to find spending cuts.
Treasury minister Andrew Griffith said the government's proposals were the "right plans" to grow the UK economy.
He claimed "every major economy is dealing with exactly the same issues" and said the impact of "Putin's war in Ukraine is cascading through things like the cost of energy, some of the supply side implications of that".
The value of the pound dropped to $1.05 on Wednesday, after the Bank of England stepped in to stabilise the economy, before later rallying to about $1.08.
On Wednesday, the Bank announced it would buy government bonds on a temporary basis to help "restore orderly market conditions".
Some pension funds hold a lot of government bonds, because they are normally a stable investment, but as their value dropped there were concerns over the solvency of some funds, the BBC has been told.
The Bank's moves on Wednesday followed a highly-critical intervention from the International Monetary Fund (IMF), which warned the measures were likely to fuel the cost-of-living crisis.
Meanwhile, Chief Secretary to the Treasury, Chris Philp, has confirmed he is asking government departments to find savings.
Speaking on ITV's Peston, Mr Philp said government departments are being asked to "look for efficiencies wherever they can find them".
These efficiencies will "stick to the targets" of the 2021 Comprehensive Spending Review "that set out three year spending plan", Mr Philp said.
Any savings will "enable us to target spending on things that target growth," he added.
Gerard Lyons, an economist who has spoken in support of the mini-budget, criticised the Chancellor Kwasi Kwarteng for "failing" to warn financial markets about the plans.
"Ahead of the budget, it was necessary for the chancellor to keep financial markets on-side," Mr Lyons told BBC Two's Newsnight.
"The chancellor appealed to domestic population and appealed to a domestic business community, but one key group he failed to keep on-side was the finical market."
Conservative MP Mel Stride has criticised the government for not publishing a forecast of the UK's economic outlook from the Office for Budget Responsibility (OBR) - saying it needs to do so "as soon as possible".
Forecasts from the independent OBR give an indication of the health of the nation's finances. The BBC understands the government was given a draft before announcing the tax cuts, but it has not been published.
"The OBR should have been brought in earlier," said Mr Stride, the chairman of the Treasury select committee. He said the OBR forecast could have then be used to reassure the markets.
"It is urgently important that the government and the Treasury comes forward at the earliest possible time now with that OBR forecast, with what their fiscal target is going to be, so they can turn and face the markets and say this is this is all fiscally credible."
But Mr Philp said an OBR report will not be published until 23 November, when Mr Kwarteng plans to set out a "Medium-Term Fiscal Plan".
This will allow the OBR forecasts to be done "in a way that's organised and thoughtful," Mr Philp said.
Downing Street rejected calls for Parliament to be recalled after calls from Labour leader Sir Keir Starmer to do so MPs can overturn last week's mini-budget.
Parliament is currently suspended while the two main parties hold their annual conferences. It is due to come back on 11 October.
The £45bn package of tax cuts announced last week would repair "underlying problems in the economy," Mr Griffith said.
He added the Bank of England had "done their job" by announcing it would buy government debt to stabilise the economy.
Mr Kwarteng, who unveiled the tax-cutting package last Friday, is yet to speak publicly on the growing impact of the package on markets.
The Treasury said the plans would be funded by £72bn of borrowing and there is an expectation this will surge as interest rates rise.
The pound slumped following his statement and later fell to a record low against the dollar after Mr Kwarteng hinted there were more tax cuts to come.
Julian Smith became the first former cabinet minister to call for the government to "make changes".
The former Northern Ireland Secretary, who backed Rishi Sunak to become prime minister, tweeted: "It is critical that the government is honest about the current situation and plays its part in stabilising markets.
"It can keep a growth plan but needs to make changes. Not doing so will only continue further stress and strain on UK citizens."
Speaking to broadcasters, Mr Griffith, the Financial Secretary to the Treasury, insisted the UK had a "very strong balance sheet" and one of the lowest debt to GDP ratios of major economies.
Asked whether the government had any plans to change Friday's measures, he replied: "We think they're the right plans, because those plans make our economy competitive".
"At the end of the day, that's ultimately what we've got to do," he said, saying ministers were focused on "getting on and delivering" its plans to work.
Andrew Griffith said plans in the mini-budge "make our economy competitive".