Boris Johnson ends a bruising week with a vacuum at the heart of Downing Street and cabinet ministers beginning to distance themselves from his leadership.

The loss of his communications director, Jack Doyle, his chief of staff, Dan Rosenfield, his principal private secretary, Martin Reynolds, and his policy chief, Munira Mirza, all in one day leaves the Tory party unsure who is now running the Johnson project in No 10, what its aims are and who will carry out their pursuit.

While the first three have notionally agreed to stay on until replacements are found, many MPs are beginning to doubt whether a team can be stitched together in time to stave off a confidence vote.

Boris Johnson started the week aiming to overshadow partygate.

With every day that passes, Tory MPs agree among themselves that Johnson’s premiership has an “end of days” feel to it – and yet the threshold of 54 letters signalling a loss of support in the prime minister has not been reached, leaving the party and government in a paralysed limbo.

Johnson started the week with a schedule of announcements designed to overshadow the damaging revelations of the Sue Gray report and the Met police investigation into the Downing Street parties.

A day trip to Ukraine was set up to make him look like an involved international leader, and there was the launch of his flagship policy on levelling up. MPs appeared temporarily boosted by the news that Sir Lynton Crosby, his elections guru, would be back giving him advice.

But on closer inspection, each of these events only served to make him look weaker, as he ran shy of the press by taking just a Sun journalist with him to Kyiv, and failed to take the lead on the white paper, sending out the cabinet minister Michael Gove instead to do a statement and media interviews. It also became increasingly clear that Crosby’s involvement would be at arm’s length only.

A bellicose performance at prime minister’s questions on Wednesday also triggered at least one of the resignations – that of policy chief Munira Mirza as he failed to apologise for his slur two days earlier against Keir Starmer. Johnson had falsely claimed that Starmer had failed to prosecute the serial child abuser Jimmy Savile when director of public prosecutions..

For all the furore around the parties, it is the prime minister’s flailing response and lack of contrition over the Savile slur that appears to have been the last straw for Mirza, along with some other MPs, such as Gary Streeter, who was one of at least five to put in letters of no confidence this week.

And all the while, his cabinet ministers are watching from the sidelines, wondering whether the moment will come soon for a leadership contest.

Rishi Sunak returning to Downing Street on Friday.

Rishi Sunak, the chancellor and the frontrunner to succeed Johnson, was the first to draw a line between himself and the beleaguered prime minister, saying he would not have made the Savile comment about Starmer.

And then Sajid Javid, the health secretary, another possible rival, also laid down a marker that he did not approve of the remarks, saying Starmer as director of public prosecutions did a “good job and he should be respected for it”.

Several of the cabinet – Liz Truss, Nadhim Zahawi and Grant Shapps – are in isolation with Covid, but the collective role of Johnson’s top team will become increasingly important in the days to come.

Should Johnson manage to cobble together only a weak and insubstantial team over the weekend to replace his losses in No 10, then MPs may begin to lose the faith even before the conclusion of the Met police investigation and full publication of the Gray report.

In this situation, they will increasingly be looking to the cabinet heavyweights and potential leadership challengers to give them the nod that now is the time to start submitting letters in droves. That moment has not yet come – but MPs and Downing Street are on tenterhooks.