Experienced Tory MPs with a “neutral” attitude to Boris Johnson are being sought to sit on the committee judging whether he misled parliament over Partygate, following concerns that its current membership is too aligned to the government.

Johnson is facing an investigation by the privileges committee into whether he lied to the Commons when he said he did not know about any parties in Downing Street during lockdown.

It was thought the current membership of seven – four Tories, two Labour, and one SNP MP – would conduct the investigation. However, two of the Tories are government aides and the Labour chair, Chris Bryant, has recused himself from the investigation for having already said he thought Johnson should quit.

With huge pressure on the MPs conducting the inquiry, the Tories have accepted that the inquiry needs to be undertaken by parliamentarians who are not on the government payroll, and are experienced enough not to be seeking preferment from No 10. Parliamentary rules state that ministerial aides should not be allowed to sit on committees, but this has been ignored in recent years.

Some of the senior Tories being mooted include former attorney general Sir Geoffrey Cox, former justice secretary Sir Robert Buckland, who was previously a member of the committee, former culture secretary Maria Miller and Jeremy Wright, another former attorney general. From Labour, Harriet Harman, the former deputy leader, is believed to have been tapped up.

Sir Bernard Jenkin, a current senior member of the committee, may stay on the body. However, Laura Farris and Alberto Costa, two government aides, will be dropped.

However, Tory MPs in particular are proving reluctant to sit on the committee given the sensitivity and impact of their decision, which could lead to Johnson’s resignation if he is found to have knowingly misled the House of Commons.

There is also an issue with finding candidates to sit on the committee who have not expressed a view on the prime minister’s conduct over Partygate. Wright, for example, has publicly given his opinion that Johnson should go if he knowingly broke the law and therefore misled the Commons, as well as saying more recently that he can “see how a single mistake can be forgiven, even when it is made by a prime minister”.

The privileges committee will have a Conservative majority with candidates picked by the party’s authorities. However, the membership will have to be approved by the House of Commons, so the chosen MPs will need to command the confidence of a wide range of Tory backbenchers – including the prime minister’s critics.

One source with knowledge of the process said Tory MPs were trying to dodge the job of having to sit in judgment over the prime minister, knowing it to be a “poisoned chalice”.

The committee will not begin examining evidence against Johnson until both the conclusion of the Metropolitan police’s inquiries into lockdown breaking in Number 10 and Whitehall – which have already yielded one fine for the prime minister – and a Whitehall report by the civil servant Sue Gray, which will be published after the Met’s case is closed.

Some Conservatives have expressed scepticism that the committee is likely to report before the autumn or even before the end of the year – given the length of time for the Met to finish its investigation.

Bryant told the BBC he believed Johnson would be forced to quit by the end of May, but Number 10 has said he has no intention of resigning if he receives more fines from the Met or if he is criticised in Gray’s report.

In an exchange with Keir Starmer in January, the prime minister has acknowledged it is precedent for ministers to resign if they are found to have misled parliament.