Boris Johnson is under growing pressure to confirm whether parties were held in his Downing Street flat in defiance of Covid rules during last year’s lockdowns.

The prime minister’s former adviser, Dominic Cummings, claimed on Friday that photos could come to light proving such gatherings were held. On the day that Cummings dramatically left his job in Downing Street in November last year, blaring music and aides chatting were heard in the No 11 residence where Johnson lives with his wife, Carrie, sources have said.

Separately, the Guardian has been told a “wine and pizza” party attended by Tory advisers was held in Downing Street after a Covid press conference during the first lockdown, in spring 2020. No 10 said the prime minister was in his flat “solely with his family” on the evening in question and the spokesperson denied knowledge of any gathering.

Another insider suggested there were “various little drinks things” in breach of rules preventing social gatherings during 2020, with press office staff in particular tending to open a bottle of wine on Friday evenings, especially if they were working late. They said it would have been impossible for Johnson not to notice such gatherings as he had to walk past on his way up to the 11 Downing Street flat. “There’s literally no other way he could get to his flat,” the source said.

The prime minister has consistently denied breaches of Covid rules, but this week his aides were exposed as having laughed and joked about a party on 18 December, which is now under investigation.

Asked on Friday whether Johnson had been present at any social gatherings in his residence during lockdowns or restrictions on contact, his official spokesperson said: “No.” The spokesperson declined to say whether Johnson had known about the 18 December event, saying it would be covered by the independent investigation by Simon Case, the cabinet secretary.

The Guardian also understands that Jack Doyle, Johnson’s chief spin doctor, tried to quit his post after it emerged he attended the party to thank staff and handed out mock awards, including for the “best dressed”. The December event went on late into the night, with dozens of staff drinking, swapping secret Santa presents and playing party games, it is understood.

Doyle’s resignation was rejected by the prime minister, two sources said, contrasting with the approach taken earlier this week when the government’s Cop26 spokesperson, Allegra Stratton, was forced to stand down in a tearful address outside her home. “The PM wouldn’t have it,” a source said.

Asked whether Doyle had tried to step down, Johnson’s spokesperson said: “Not that I’m aware of.” Doyle did not comment. No 10 said the prime minister still had full confidence in him.

Insiders believe Johnson is resisting pressure for any more resignations. However, Stratton said in leaked footage four days after the 18 December party that she “went home”, leading to some Tory MPs calling for those who were allegedly there – such as Doyle and No 10’s head of broadcast, Ed Oldfield – to face the sack.

Charlie Falconer, Labour’s former shadow attorney general, asked how Johnson could possibly retain confidence in Doyle after he presided over “days of No 10 briefing” that there was no party and no rules were broken.

Cummings said there were “lots of pictures of the parties which will inevitably get out” and that invitations to the 18 December event were sent to staff in other Whitehall departments.

He added that Johnson knew about it and would have had to walk past the area where it was held to go upstairs to his own flat. Cummings claimed the prime minister had “tried to lie his way out” by denying any knowledge of the event but was boxed into a corner due to the footage unearthed this week.

On Friday, the Labour leader, Keir Starmer, said Johnson was unfit for office, accusing the government of being “up to its neck in dishonesty”. “We’ve had lie upon lie in relation to the parties going on in Downing Street,” he said. “The question really is for Tory members of the cabinet, Tory MPs, to ask themselves: ‘Are they prepared to put up with this?’

“Are they prepared to go through the degrading of themselves and their party, to go out to the media, have to defend the indefensible, for months to come, or are they going to actually have the courage to challenge him and say: ‘You’re not fit for office.’”

There are three gatherings under investigation by Case: the 18 December party; another believed to be a leaving do for an aide, Cleo Watson, on 27 November; and “thank you” drinks for staff working in the then education secretary, Gavin Williamson’s department on 10 December.

Case said that “if during the course of the work any evidence emerges of behaviour that is potentially a criminal offence, the matter will be referred to the police”. However, other events took place that are not being looked into, including a party at Conservative campaign headquarters on 14 December – the same day it was announced that London would be going into tier 3, with social mixing between households already banned.