Paul Dacre has pulled out of the running to be the next chair of the media regulator, Ofcom, after concerns were raised about the transparency of the recruitment process.

After failing in his first attempt when an interview panel decided the former editor of the Daily Mail did not fulfil the required criteria, ministers then cleared the way for him to be given another shot.

But when the Guardian revealed a lobbyist at a company with close connections to the Conservative party was picked to help select which candidates should be approved, Dacre announced he would not proceed with an application again despite being urged to “by many senior members of the government”.

He said he had been judged inappropriate to head Ofcom the first time because his “strong convictions” were not compatible with the role.

One government figure told the Guardian they were “genuinely disappointed” by Dacre’s decision to pull out and ministers had expected to back him to become chair of Ofcom. They insisted the former Daily Mail editor had taken the decision not to proceed after not enjoying the original interview process and reflecting on how it could be second time around. The growing public scrutiny of appointments and government sleaze following the Owen Paterson affair may have also played a role, although the source insisted ministers did not tell Dacre to step aside.

The government had spent more than a year trying to ease Dacre’s application for the job. After he failed the first interview process the government spent the summer trying to find people willing to sit on a fresh interview panel for the job, with many individuals reluctant to put their name to the process. The job description was also rewritten to favour a more confrontational candidate, in another move seen as favouring Dacre.

The government now has just 10 days to find another preferred candidate, with applications due by 29 November.

Announcing his decision to drop out of the Ofcom race, Dacre described the process of civil service recruitment as an “infelicitous dalliance with the blob”, and suggested it was Whitehall workers rather than politicians “who really run this country”.

Writing in a letter in the Times, he said: “To anyone from the private sector, who, God forbid, has convictions, and is thinking of applying for a public appointment, I say the following: the civil service will control (and leak) everything; the process could take a year in which your life will be put on hold; and if you are possessed of an independent mind and are unassociated with the liberal/left, you will have more chance of winning the lottery than getting the job.”

Dacre said he wished Ofcom “all the luck in the world” as it prepared to grapple with “trying to regulate the omnipotent, ruthless and, as we’ve learnt, amoral tech giants without damaging freedom of expression”.

In a final attack on the civil service, Dacre said he was taking up “an exciting new job” in the private sector that “struggles to create the wealth to pay for all those senior civil servants working from home so they can spend more time exercising on their Peloton bikes and polishing their political correctness”.

Following the suspicion that senior politicians had interfered in an attempt to install Dacre despite his apparent unsuitability for the role, Boris Johnson was urged to rule him out of the running.

The Conservative MP Julian Knight, the chair of the culture select committee, said earlier this autumn: “Where a previous candidate has been deemed to be unappointable for a post, they should be ruled out of reapplying.”

A spokesperson for Department for Digital, Culture, Media and Sport said: “The recruitment process is fair and open and in line with the governance code for public appointments, which clearly sets out that assessment panels must be objective when deciding which candidates meet the criteria for a role.

“The process is regulated by the Commissioner for Public Appointments, who is responsible for ensuring that the appointment is made in accordance with strict guidelines.”

It came as the Scottish and Welsh governments raised concerns about interference in the recruitment of Ofcom’s next chair, saying it could “adversely affect the standing of the public service broadcasting system”.

In a letter to the culture secretary, Nadine Dorries, ministers for both the devolved administrations said “delays, changes and an apparent lack of transparency have caused us to lose further confidence in the process”.

They questioned the level of Downing Street’s interference, saying the appointment process had been “tarnished by delays and questions about the real independence of those involved”.

Accusing Dorries and her predecessor as culture secretary Oliver Dowden of overseeing a selection that was “falling below the standards we expect”, the Welsh and Scottish government demanded to be brought in on the process.

They wrote: “Given the importance of public service broadcasting to our nations and the real impact for our nations of any decision on selecting the Ofcom chair which is not transparent or impartial, we urge you to involve us fully in the process as is right to protect a system which is so important to the public in Scotland and Wales and all the UK.”