MI5’s partially secret policy of allowing agents to participate in serious crimes in pursuit of intelligence was legal, three court of appeal judges have concluded.

The judges held on Tuesday that MI5 was “not above the law” because the long-established power did not equate to an immunity from prosecution, in the latest step in a long-running legal case brought by four human rights groups.

At a hearing in the case in January government lawyers told a court that MI5 officers could in theory authorise an informer to carry out a murder if they were “an extremely hostile individual”.

But in its judgment the appeal court concluded any authorisation given by MI5 officers to informants would have been acceptable because the spy agency’s internal guidance said it could only apply proportionately.

The guidance, the judges stated, “stipulates that authorisation may only be given” where “the potential harm to the public interest from the criminal activity is outweighed by the benefit to the public interest derived from the anticipated information the agent may provide”.

As a result there was “a limit to what criminality may be authorised”.

Human rights groups indicated they would seek to appeal to the supreme court. Maya Foa, the director of Reprieve, said: “The idea that the government can authorise undercover agents to commit the most serious crimes, including torture and murder, is deeply troubling and must be challenged.”

Critics say agents operating in Northern Ireland have repeatedly been accused of colluding in murder, although bringing cases to court has proved complex. Last summer, prosecutors decided not to charge two former MI5 officers, among others, relating to cases involving an IRA informer, codenamed Stakeknife.

Priti Patel, the home secretary, said she was pleased the court of appeal had recognised “the vital role that undercover agents play in preventing and safeguarding victims from serious crimes”.

Home Office sources cited the case of Naa’imur Zakariyah Rahman, who was jailed for life in 2018 for plotting to kill the former prime minister Theresa May. He was caught following an undercover operation in which he was provided with what he thought was a jacket and rucksack packed with explosives.

The court battle applied to MI5’s previous policies, which date back to at least the 1950s. The agency says it is often necessary to allow informants to commit some crimes so their cover is not blown.

Fifteen months ago, a lower tribunal concluded that MI5’s informants policy was legal, but only narrowly, by three to two.

Ministers, concerned about the outcome of future legal challenges, introduced the covert human intelligence sources bill to put MI5’s policy on a statutory footing. Labour split on whether to oppose it, but the Conservatives’ Commons majority ensured it passed into law on 1 March.

However, the Scottish parliament formally rejected the bill, meaning the judgment of the court of appeal is potentially more significant there. Without the legislation, the spy agency is able to rely on the powers in the guidance upheld by the court of appeal as legal.