Rishi Sunak’s local council leader is among a growing number of senior Conservatives calling for Boris Johnson to step down as party members lose patience with his leadership.

Carl Les, the Conservative leader of North Yorkshire county council, said he thought it was time for a leadership election, blaming Boris Johnson squarely for heavy losses in the local election.

“I am very disappointed that the strong majority we had in North Yorkshire has diminished down to a working majority, but only just, and a lot of the comment we were getting on the doorstep was about the impact of Partygate,” said Les.

The North Yorkshire Tories had gone from 55 seats to 47, going from 75% of the total seats to just 52%, he said. The heaviest losses were in affluent metropolitan areas such as Harrogate, where the Liberal Democrats are now the biggest party.

Asked how local members viewed Johnson, he said: “There are a number of people who think the PM should resign and we should have a leadership election and personally I would support that.” Sunak, his local MP, was his choice for leader, he added.

In Sunderland, where the Conservatives were said to have “thrown the kitchen sink” at winning seats off Labour in the local elections but made no gains, the local Tory group leader said Johnson had become an electoral liability and that it was now “almost guaranteed that he will lose the next general election”.

Antony Mullen said he would not be surprised if Johnson survived the most serious test of his leadership to date, given his ability to wriggle out of difficulties in the past.

But he added: “If Conservative MPs are happy to defend what he’s done and lose their seat as a result of it, then that’s fair enough.

“But I think a lot of them will come quickly to the conclusion that they quite like being MPs and would like to have a future in parliament – and that isn’t possible with him as leader.”

Andy Morgan, a senior Conservative councillor and cabinet member in Bolton, said he didn’t think it was the right time for Johnson to go but that public support was clearly waning.

“The polls are suggesting that [the public] aren’t happy at the moment, so he’s either going to have to pull something out of the bag and make them happy or he’s going to have to think again. As in, he has to do something to get the trust back into politics,” he said.

“There’s fundamentally something going wrong at the centre and it needs to be sorted. He has suggested he is doing that and has sorted his staff out and the culture within No 10 but he now has to demonstrate that is the case to the public.”

Morgan has little enthusiasm for policies such as bringing back pounds and ounces, which he sees as “an irrelevant distraction”. He wants the Conservatives to “knuckle down and do what they have been elected to do by the public”, particularly on the cost of living crisis.

Mike Bird, the Conservative leader of Walsall council, said it would be a mistake to get rid of Johnson given the war in Ukraine and the soaring cost of living.

Bird said people were “sick and tired” of Partygate and that he wanted the government to provide more help for people struggling financially.

“People are more concerned about how they pay their electricity bills, gas bills and shopping bills. People are quite genuinely seriously concerned, and so am I,” he said.

“Both Rishi Sunak and Boris Johnson, and all MPs, ought to realise that they don’t live in a cocoon down in Westminster. They need to address the issues facing the cost of living crisis … that’s more important than tea and cakes. I hope [last week’s mini-budget] was the first of the steps that are required to assist those people.”