Keir Starmer has used prime minister’s questions to attack the tax affairs of Tory leadership candidates, while Boris Johnson insisted that while he had not wanted to depart No 10, he was nonetheless doing so “with my head held high”.

In what is likely to be Johnson’s penultimate PMQs appearance – it is then recess, and his replacement will be in post in September – Starmer focused his questions on the financial affairs of would-be successors.

He asked Johnson if the government would abolish non-domiciled tax status, as used by the wife of Rishi Sunak, the former chancellor, and about offshore tax schemes, targeting Nadhim Zahawi, the chancellor, who has ties to investments based in Gibraltar.

The Labour leader also ridiculed the lavish tax cut pledges made by candidates, saying these totalled £330bn, and the fact that many of them had been in a Johnson government, which had raised taxes.

“Sadly they haven’t found time to explain how they are paying for it, even though one of them is the chancellor and another one was chancellor until a week ago,” Starmer said. “They all backed 15 tax rises. Now they’re acting as if they’ve just arrived from the moon, saying it should never have happened.”

Starmer also derided the claim by Zahawi – who he called “the member for Stratford and Gibraltar – that he could pay for his own promised tax cuts, estimated by some at £36bn a year, by cutting government department headcounts by 20%.

He also took aim at Sunak for saying that if he became prime minister he would rebuild the UK economy.

“Even the prime minister must be impressed by that Johnsonian brass neck-ery,” Starmer said. “Can the prime minister think of any jobs his former chancellor may have had that means he bears some responsibility for an economy that he now claims is broken?”

For his part, Johnson vigorously defended the eight remaining candidates, saying any one of them would “wipe the floor” with Starmer.

He said “every one of the eight candidates” would continue with his flagship levelling up programme of infrastructure investments, even though several of them have talked about wanting to slash public spending and the role of the state.

Castigating Starmer for having “no idea, plan or vision”, Johnson also gave a defence of his time in office, noting that if one of the two final candidates drops out, avoiding the need for a run-off vote among Tory members, this could could be his last PMQs.

Citing his Brexit deal, the UK’s initial vaccine rollout and its support for Ukraine, Johnson insisted he was proud of his record.

“It’s perfectly true that I leave not at a time of my choosing. But I am proud of the fantastic teamwork that has been involved in all of those projects, both nationally and internationally. And I am also proud of the leadership I have given,” he said.

“And I will be leaving soon with my head held high.”

The start of the session was delayed after the Speaker, Lindsay Hoyle, ordered two Alba party MPs, Neale Hanvey and Kenny MacAskill, to leave the chamber after they stood up seeking to make a point about a Scottish independence referendum.

MPs will next week debate a confidence motion in Johnson’s government – but one proposed by the government itself, after No 10 rejected a version by Labour that directly criticised the prime minister.

The unexpected move will force Conservative MPs to back the government but will make the vote more straightforward as it will not constitute an endorsement of the prime minister.

“Labour were given the option to table a straightforward vote of no confidence in the government in keeping with convention. However, they chose not to,” a government spokesperson said.