Soaring inflation has seen interest payments on government debt leap by 63% over the past year, official data reveals.
Figures published by the Office for National Statistics (ONS) also showed the UK borrowed more than expected in July as public spending outstripped taxes and other income.
Government borrowing hit £4.9bn in July, according to the ONS, dwarfing the £0.2bn forecast by its own independent financial watchdog, the Office for Budget Responsibility.
It was also significantly ahead of the £2.8bn forecast by analysts and brings the total budget deficit for the year so far to £55bn.
Back in July 2019, prior to the COVID-19 pandemic which prompted a historic surge in government borrowing, the public finances were in surplus by £0.9bn.
It came as the government's debt costs rose to £5.8bn in July up by £2.3bn from last year due to increases in Retail Price Index (RPI) inflation.
Earlier this week, the ONS revealed that RPI surged to 12.3% in July, while the broader Consumer Price Index (CPI) inflation figure hit a new 40-year-high of 10.1%.
The latest data also showed government spending increased by £3.4bn to £76.5bn in July compared with the same month year.
Responding to the figures, Chancellor Nadhim Zahawi said: "I know that rising inflation is creating challenges for families and businesses, and it is also putting pressure on the public finances by pushing up the amount we spend on debt interest.
"To help people during this difficult time, government support is continuing to arrive in the weeks and months ahead, targeted to those who need it most, like pensioners, people on low incomes, and those with disabilities.
"We are taking a balanced approach: safeguarding the public finances while providing significant help for households."
Michal Stelmach, senior economist at KPMG UK, said the latest figures will mean "tough choices" for the next chancellor following the Conservative leadership election.
He added: "The balance of risks to public finances has clearly shifted to the downside.
"The cost of living crisis will likely require further support to households, while a slowing economy will put downward pressure on receipts, making the fiscal targets ever less achievable."
The frontrunner in the Conservative Party leadership race, Foreign Secretary Liz Truss, has said she will cut taxes, while her rival, former chancellor Rishi Sunak, has warned that will risk fuelling inflation and has proposed more direct, targeted support.
Samuel Tombs, chief UK economist at Pantheon Macroeconomics, has estimated that if Ms Truss wins, Britain's budget deficit is likely to hit about £170bn in the current financial year, about three times its size before the pandemic.
Meanwhile, new research indicated consumer confidence is at an all-time low in light of "acute concerns" about the soaring cost of living and bleak economic outlook.
The Bank of England has warned escalating inflation is likely to tip the UK into recession later this year.