Downing Street later said the pledge was not to abolish current backlog of initial decisions - only claims made up to the end of June.
Under a plan unveiled by the prime minister, a dedicated unit of 400 specialists will be set up to handle claims from Albanians.
Refugee charities branded the plans "cruel" and "ineffective".
The asylum backlog has ballooned in recent years, with 117,000 outstanding claims, relating to 143,377 people, who are waiting for an initial decision on their application and unable to work. Of these, nearly 100,000 people have been waiting more than six months.
In the Commons, Mr Sunak said the government expected to "abolish the backlog of initial asylum decisions by the end of next year".
The prime minister's official spokesman later said this pledge only related to claims made before 28 June, when the Nationality and Borders Act came into force. He said this consists of 92,601 initial asylum claims.
Labour's shadow home secretary Yvette Cooper said Mr Sunak's pledge was "already falling apart just hours after he made it".
With pressure building on the Home Office to tackle the issue, Mr Sunak has labelled the small boats crisis a priority for his premiership.
More than 40,000 people have crossed the Channel in small boats so far this year - the highest number since figures began to be collected in 2018.
There has been a rapid increase in the number of Albanians crossing the Channel and this nationality now make up 35% of arrivals.
Ministers have singled out Albanian migrants, accusing many of exploiting the system by pretending to be victims of modern slavery.
Under the government's plans, asylum case workers will be given new guidance making it clear Albania is a safe country and requiring evidence of modern slavery when considering a claim.
As a result of the plans, Mr Sunak said the vast majority of claims from Albanians could be declared "clearly unfounded", which would see "thousands" returned home in the coming months.
UK border officials will also be posted at Albania's main airport, under a new deal with the country.
* a pledge to end the use of hotels for asylum seekers
* plans to house 10,000 individuals waiting on claims in disused holiday parks, former student halls, and surplus military sites
* a commitment to double the number of asylum caseworkers, who assess claims
* more staff and funding for the National Crime Agency to tackle organised immigration crime in Europe
* plans for Parliament to set an annual quota for refugees coming to the UK
* new laws, to be introduced next year, to "make unambiguously clear that if you enter the UK illegally, you should not be able to remain here"
Labour leader Sir Keir Starmer dismissed the proposals as "unworkable gimmicks".
While he welcomed the announcement of additional staff, he said there had so far been a "total failure of any co-ordinated response" to criminal trafficking gangs.
Labour MP Dame Diana Johnson, who chairs the Commons Home Affairs Committee, questioned whether there were enough staff to clear the backlog.
But in an interview with the BBC, Mr Sunak insisted tripling the productivity of staff was "realistic", by removing paperwork and streamlining asylum interviews.
Mr Sunak also pledged to "significantly raise the threshold someone has to meet" to be considered a victim of modern slavery.
But former Conservative Prime Minister Theresa May warned modern slavery was "a very real and current threat".
She urged Mr Sunak not to "diminish our world-leading protections for the victims of this terrible, horrific crime".
The prime minister denied he was watering down the definition of modern slavery and told the BBC he wanted to stop the system being "clogged up by people making spurious claims".
The UN's refugee agency welcomed measures to address the asylum backlog but said plans to limit access to asylum to those arriving through "safe, legal routes" went against the principles of the 1951 Refugee Convention.
The UNHCR said the announcements marked "a troubling step away" from the UK's "commendable humanitarian tradition".
Tim Naor Hilton, chief executive of charity Refugee Action, criticised the government for failing to commit to creating new safe routes for people to come to the UK, which he said "could end most small boat crossings overnight".
"Most of these changes are cruel, ineffective and unlawful and will do nothing to fix the real problems in the system," he said.
The Refugee Council said treating people "who come to the UK in search of safety as illegal criminals" was "deeply disturbing and flies in the face of international law".
The charity said it was "very simplistic" to label Albania safe "when in reality it has serious problems with criminal and sexual exploitation of women and children".
Chris Mason challenges Rishi Sunak over plans on processing asylum claims