Labour and senior legal figures have raised concerns that Dominic Raab was appointed as justice secretary in order to enact wholesale changes to the Human Rights Act.

Labour has unearthed footage of the former foreign secretary saying he did not support the act, which he will now be expected to enforce or overhaul. In messages sent to ministers earlier this year, Raab urged the government to be more ambitious as it sought to reform human rights law and judicial reviews.

The government launched the independent Human Rights Act review, expected to report later this year, which is reconsidering the duty on UK courts to “take into account” judgments from the European court of human rights, and their ability to declare British laws “incompatible” with human rights.

Footage of Raab uncovered by the office of the shadow justice secretary, David Lammy, from 2009 shows Raab, then a backbench MP, looking into the camera and saying: “I don’t support the Human Rights Act and I don’t believe in economic and social rights.”

In a book entitled The Assault on Liberty: What Went Wrong with Rights, authored by Raab in the same year, he argued that the Human Rights Act – introduced by Labour in 1998 – had led to a slew of new claims in the courts.

“The spread of rights has become contagious and, since the Human Rights Act, opened the door to vast new categories of claims, which can be judicially enforced against the government through the courts,” he wrote.

The act had allowed UK law to be trumped by the European courts, Raab claimed, while the boundaries between parliament, the judiciary and the executive had been blurred.

“The very enactment of the Human Rights Act has served as a trigger for the formulation of claims by lawyers and judicial reasoning by courts, using human rights arguments that would never have been dared before,” he said.

On Wednesday, Raab was removed as foreign secretary and accepted the roles of justice secretary, deputy prime minister and lord chancellor, and he is now also responsible for the independence of the judiciary. The previous justice secretary, Robert Buckland, has returned to the backbenches.

Informed sources said Raab commented earlier this year on the government’s proposals for possible changes to the Human Rights Act and judicial review. After receiving a “write-round” – a note sent out to cabinet ministers about proposed policy – from Buckland’s office, Raab suggested that ministers could be “more ambitious”, a source said.

The legal blogger David Allen Green said Raab was not a popular choice for the position among lawyers because of his fixation with the act. “One would not be surprised that one stipulation made by Raab in accepting the position as lord chancellor is that he get another crack at repealing the Human Rights Act,” he said.

There has been concern about the high turnover in the role of lord chancellor over what has been a turbulent period for the justice system, with huge spending cuts. Derek Sweeting QC, the chair of the Bar Council, said: “As we welcome the eighth justice secretary in the last 10 years to play this vital role, the need for a consistent and strong voice in government for our justice system could not be greater.”

Sir Bob Neill, the Conservative chair of the justice select committee, said he would work with Raab but lamented the way Buckland had been sacked by Boris Johnson to “make way”.

“The position of lord chancellor is crucial. It is not some sort of sweetie to be handed out by the PM,” he said.

A Ministry of Justice spokesperson said: “The new secretary of state will carefully consider the independent review’s findings when the panel reports back later this year.”