The Tory manifesto before the general election in 2019 promised that "overall numbers will come down".
Net migration - the number of people moving to the UK minus the number who leave - was 226,000 in the year to March 2019.
In the year to June 2022, net migration hit an all-time high of 504,000.
The latest number, for the year to December 2022, will be published next week - and is widely expected to be higher still.
Speaking to reporters en route to the G7 Summit in Japan, Rishi Sunak said: "I've inherited some numbers, I want to bring the numbers down."
As the numbers climb, he is maintaining a desire that they fall, but not explicitly below the level they were at when the initial promise was made.
"When it comes to legal migration, the key thing for people to know is we're in control of why people are here, the circumstances and the terms on which they are here, making sure they contribute, to public services like the NHS for example," Mr Sunak said.
"Those are all now part of our migration system and they weren't before," he added, in reference to Brexit - which means immigration policy is now decided solely at Westminster.
There is some evidence that the importance attached to cutting immigration has fallen for some people since the UK left the European Union, suggesting control over it, as well as how much of it there is, does really matter to some.
As I have written about here, there is quite a discussion going on within government about how to respond to next week's new net migration figure.
I am told the prime minister has not yet looked at it in detail. But he is likely to in the coming days, ahead of a government announcement which is expected to include a restriction on the dependents some foreign students can bring with them when they come to the UK to study.
Strikingly, when pressed on his instincts on legal migration, the prime minister repeatedly changed the subject to talk about illegal immigration instead - small boat crossings.
"I do think most people's number one priority when it comes to migration is illegal migration, that is crystal clear to me," Mr Sunak said.
"That's why one of my five priorities is to stop the boats, that's why recently we have moved to a Covid-style committee structure where I am meeting twice a week with ministers to drive the implementation of the new bill," he added, referring to the Illegal Migration Bill.
He also pointed out what he believes is a significant breakthrough in helping to limit the number of small boats crossing the Channel.
At a meeting of the Council of Europe in Iceland on Tuesday, the European Union agreed to begin conversations about the UK having a cooperation agreement with Frontex, the European Border and Coastguard Agency.
There has been no such agreement since Brexit.
Such an agreement, Mr Sunak said, was "of practical value to us in stopping illegal migration - sharing intelligence, operational cooperation will make a difference to our ability to stop the boats."