In a video obtained by the New Statesman, Mr Sunak tells an audience in Tunbridge Wells, Kent, he wanted to ensure "areas like this" got the funding they deserved.
Defending his remarks, Mr Sunak said he was arguing that towns and rural areas needed help not just "big urban areas".
Labour said the leadership race was revealing the Tories' "true colours".
The party's shadow levelling up secretary Lisa Nandy said: "It's scandalous that Rishi Sunak is openly boasting that he fixed the rules to funnel taxpayers' money to prosperous Tory shires."
Ms Nandy has written to Levelling Up Secretary Greg Clark asking him to "urgently investigate" the changes made by Mr Sunak to funding formulas.
In the clip - filmed on 29 July - Mr Sunak says: "I managed to start changing the funding formulas, to make sure areas like this are getting the funding they deserve.
"We inherited a bunch of formulas from Labour that shoved all the funding into deprived urban areas and that needed to be undone.
"I started the work of undoing that."
Mr Sunak's team have said his comments were in reference to the introduction of the levelling up fund and changes made to Treasury formulas on investment.
The government has changed the way it assesses projects by altering the Treasury's "green book", which guides ministers in making investment decisions.
Some submissions to a review of the "green book" in 2020 suggested that the old system relied too much on cost-benefit analysis (how much monetary value can be created for how many people) and that this skewed the system towards building more infrastructure in London and the South East of England.
But as Tunbridge Wells is in the South East of England, changing that would not necessarily help it. It is also the least deprived local authority in Kent, according to the index of multiple deprivation.
The system was changed to require projects to explain how they would promote government priorities such as levelling up or achieving net zero emissions, before considering their costs and benefits.
This means funding will not necessarily go to the "deprived urban areas" where the largest number of people might benefit, but can instead go to more sparsely populated ones that have not received much investment in the past.
During a hustings event in Eastbourne on Friday night, Mr Sunak defended his comments by suggesting that deprivation existed across the country.
Acknowledging the video clip, he explained that he wanted to "level up everywhere", saying that was not just about helping "very large urban cities".
He added: "I believe it's about investing in levelling up in small towns, in rural communities, in coastal communities like those here in the South East."
Earlier, Tees Valley Mayor Ben Houchen - a Sunak supporter - told BBC Radio 4's World at One programme the clip had been taken out of context, adding: "What he was getting across was why is it that poor urban areas receive significantly higher funding than poor rural or town areas."
He said places such as Devon, Teesside and parts of Tunbridge Wells needed fairer funding.
But Conservative MP and Truss ally Jake Berry tweeted: "In public Rishi Sunak claims he wants to level up the North, but here, he boasts about trying to funnel vital investment away from deprived areas? He says one thing and does another."
A source in the rival Liz Truss campaign said: "Levelling up isn't about pitting one area of the country against another, or laying dividing lines between urban v rural, towns v cities.
"It is about unleashing growth and making sure every individual has the chance to thrive. Liz has a plan to create an aspiration nation based on equality of opportunity, and will deliver it in a conservative way."
Rishi Sunak's comments appear to fly in the face of recent Conservative rhetoric about levelling up and securing Labour's former "red wall".
Why would the former chancellor be boasting about diverting funding away from deprived urban areas?
In short, because he was in Royal Tunbridge Wells at the time, one of his party's traditional heartlands, speaking to grassroots members.
And while the Conservative party doesn't publish data on its membership, research shows most of them live in the South East of England, and are generally older and relatively wealthy.
Mr Sunak's team insist he was talking about changes to ensure pockets of deprivation in towns and rural areas got the funding they deserved.
His opponents will seize on his comments as not just clumsy, but another example of him appearing to be out of touch.
The comments were made as part of Mr Sunak's campaign bid to replace Boris Johnson as Conservative leader and prime minister.
He is competing against Ms Truss, who polls suggest is more popular among Conservative members.
Friday night's hustings saw a brief disturbance when Ms Truss's opening remarks were interrupted by a small group of climate activists, who were believed to be from the Green New Deal Rising group.
The protesters heckled Ms Truss over climate change and energy bills, shouting "shame on you".
After the group was ejected from the event, Ms Truss took aim at what she called "unfair protests" that disrupted people's lives.
She pledged to clamp down on "militant people who try and disrupt our country" - whether from trade unions or environmental groups such as Extinction Rebellion.
Another protester was later removed from the event after interrupting Ms Truss.
Rishi Sunak: I diverted money from deprived urban areas
Watch: Protesters interrupt Liz Truss speech by delivering speeches of their own.