Hours after he stepped off stage with his political future in the balance, Keir Starmer decompressed with some of his closest friends and advisers for beers and prosecco in the garden of his former chief of staff, Jenny Chapman.

“He looked better than he had in ages,” one said. “Sometimes you just have to let go.”

The get-together was the old guard: Chapman and her husband, the MP Nick Smith; the former Labour MPs Gloria De Piero and Heidi Alexander; his former communications chief Ben Nunn; and campaign director Morgan McSweeney, one of the few of the original gang who is still at the heart of the Labour leader’s operation.

Since he won the leadership, Starmer’s team has been almost completely overhauled after internal clashes and slow electoral progress. Many now say there is major improvement – shown in a professionalised local elections campaign – but the fallout from the Beergate scandal has been traumatising for some of the current team.

“I think the new operation have had an easy ride so far,” a close ally of Starmer said. “There has been a honeymoon period of progress in the polls, the Tories in the doldrums. This is the first real test and frankly it’s not been good enough.”

The new investigation by Durham police was a body blow after local elections results that were at the top end of expectations, despite a lukewarm write-up. In detailed internal and external analysis since the results, a far more positive picture has emerged than in the immediate aftermath.

In a granular analysis, the Oxford professor Stephen Fisher found not only had Labour radically improved its electoral performance in London, but actually bounced back even further in communities that voted leave.

Any celebration was impossible. Last Friday, Starmer had made up his mind to pledge to resign if fined, a decision that sources said blindsided some of those who he confided in, including his chief of staff Sam White and his director of strategy Deborah Mattinson. Most were shattered by the elections, sources admitted.

By the end of the weekend, the Labour leader told his team they had not been able to give him “one single reason” why he shouldn’t make that promise. He had already sought guidance externally from his former advisers, frustrated with the impasse.

Tensions exploded at the morning meeting on Monday – with recriminations between aides over why the party had not prepared more of a fightback against the tabloid campaign. “It’s not easy to tell your boss, ’Yes I think you should offer to resign,’” one official said. “But the job’s not meant to be easy.”

Although the operation is running more smoothly since the chaotic botched reshuffle that split the team last year, there is tension between the “big brains” who want to plot a detailed strategic advance and those who have a more natural political instinct for opportunism, such as McSweeney and the MPs in charge of campaign strategy, Conor McGinn and Shabana Mahmood.

They are also the group that has been the loudest arguing for giving no quarter to the party’s left – which make some shadow cabinet ministers uneasy – although Starmer takes very little convincing to be hardline.

Frontbenchers have been privately urging the Labour leader to take more risks, shut down opponents’ attack lines quickly and focus relentlessly on the cost of living. “If this moment of courage ends up just being a one-off then we’ll be back to square one,” a senior shadow minister said.

“His foot’s in the door now, so assuming that he doesn’t get an FPN [fixed penalty notice] then he has to keep going, double down on the strong leadership narrative, and keep convincing people that he has what it takes.”

His defenders reject the caricature of him as a cautious lawyer and say he often edits his PMQs briefings to be more brutal. “Keir has got way more of a killer instinct than he is given credit for,” one senior aide said.

The coming summer campaign is regarded by many as one of the most important periods for Labour to capitalise on its polling advantage and an opportunity for airtime to be used to emphasise some key new policies.

New selections for both target and safe seats left by departing MPs will ramp up. In a 51-page report on the party’s future, Tony Blair urged Starmer to make a virtue of the selections, highlighting a high calibre of candidate backed by the leadership, though that is likely to reignite tensions with local parties wary of stitched-up selections.

The former leader also urged Starmer to draw a line under “woke” issues within the party – another source of internal anguish. But the extensive polling in the report suggested a significant opportunity for Labour to recapture many new voters who voted Conservative by large margins in 2019 despite not being strong leave voters.

Those were defined in two groups: aspirational working class voters who voted leave but by a slim margin; and professionals worried about issues such as tax and economic competence – the majority of whom voted remain.

Both groups voted Conservative by large margins in 2019 but Blair said they represent Labour’s strongest opportunities to gain more votes.

Blair also urged Starmer to be radical in laying out a political programme. The party has succeeded in putting across plans for a windfall tax – which look increasingly likely to be pinched by the Tories.

But when it comes to the next manifesto, Labour-leaning policy wonks who have held discussions with Starmer’s advisers warn that they are still a very long way away from having the bold ideas they will need.

The next few weeks are likely to bring another key advance, with a predicted victory in the Wakefield byelection. That timing is also likely to coincide closely with the verdict of Durham police, who have the Labour leader’s fate in their hands.