Thousands of children in care are increasingly being placed in homes that are illegal or unregulated, in what critics have labelled a national scandal, a Guardian investigation has found.

A lack of places to house vulnerable children in the UK is prompting a surge in placements that are less safe. These include those that are unregulated or not registered with Ofsted.

MPs, the police, charities and the children’s commissioner warn that children accommodated in these homes are at risk of exploitation from sexual predators and drug gangs.

An investigation has found:

• The number of times children were housed in unregulated accommodation – which means they faced no checks by Ofsted, which typically inspects homes – rose by 22% between 2016-17 and 2018-19. The number increased from 4,814 to 5,874, according to data obtained from 112 councils through freedom of information requests.

• The number of times children were placed in unregistered homes rose from 129 to 212 in the same period.

• The average annual cost of placements was between £9,714 a year and £364,980, meaning the total bill across all authorities is likely to run into the millions.

• In Northamptonshire, where a recent Ofsted report found the Conservative-run county council was failing to keep children safe, 360 young people were placed in semi-independent-living accommodation not subject to children’s homes regulations in 2018-19, up from 348 two years previously. Of the 360, 36 placements were outside of the local authority area.

The children’s commissioner for England, Anne Longfield, said placing children in unregulated homes was “simply unacceptable”.

Referring to unregistered placements, she said despite some providers having good intentions, all “unregistered provision is … not acceptable as a care package for vulnerable children”.

She said: “I understand that there is a serious shortage of accommodation for older children in care but on their behalf we should never accept anything less than genuinely high-quality caring places that would pass appropriate levels of scrutiny. This is another example of managing a crisis not finding a solution to a crisis. These children deserve better from us all.”

Her words were echoed by Ann Coffey, who chairs the all-party parliamentary group (APPG) for runaway and missing children. She said: “The government must act and make it unlawful for local authorities to place children in these illegal settings and Ofsted should take immediate action to close down these homes.

“It cannot be right that they are being placed in care settings with minimum checks on who is providing the accommodation. It’s a national scandal.”

Unregulated provision is essentially when children, usually over the age of 16, need support to live independently rather than needing full-time care. It is often called a “semi-independent placement” and is allowed in law, although Ofsted does not regulate it.

Unregistered provision that claims to offer any form of “care” as opposed to just support is illegal. However, there is no legal definition of what care means, which provides a loophole that can be exploited.

In some instances, children in these kinds of placements have been placed by councils in accommodation including on boats or in caravans.

A recent parliamentary inquiry into the issue of children being used to traffic drugs found that 80% of 41 police forces in England and Wales expressed concern about unregulated accommodation.

Issues raised included children being left isolated and targeted by those wishing to exploit them for sex or to sell drugs. One girl who had been sexually exploited was housed with a perpetrator of child sexual exploitation.

Iryna Pona, the policy manager from the Children’s Society, said cuts to social services had resulted in many councils not having enough placements available locally.

“And there is an increasing number of children with complex needs being taken into care,” she said. “Vulnerable teenagers placed in unregulated accommodation are at risk of being groomed for sexual and criminal exploitation. Our research has shown that teenagers may be housed with vulnerable adults and be exposed to or groomed with drugs and alcohol, they may also go missing and be subject to physical violence and verbal abuse.

“The quality of accommodation and support provided in these types of placements varies greatly from one place to another and from one area to another. Staff often aren’t sufficiently trained in protecting young people from criminal and sexual exploitation.”

Katharine Sacks-Jones, the chief executive of Become, the national charity for children in care and young care leavers, said: “The rise in children being placed in these homes is deeply concerning … They can be magnets for criminals, leaving vulnerable children at heightened risk of sexual and criminal exploitation such as county lines.”

Sacks-Jones said a comprehensive review of the care system was needed to understand and address why more children were being placed in these homes.

Andrew Parry, the portfolio holder for children, education and learning at Dorset council, said it used unregistered provision when there were not sufficient registered placements to support children and young people with very complex needs.

He said the placements were generally time-limited, with regular visits made by the authority to monitor care . “We don’t want to have to use unregistered placements and are currently in the process of registering all the provision we’re using.”

Of the 109 councils that responded to the Guardian’s freedom of information requests, Kent county council said it used unregulated accommodation the most frequently. It recorded 452 placements in 2016-17, 312 the following year and 326 in 2018-19.

Sarah Hammond, the director of children’s social services at Kent county council, said its commissioning team undertook its own inspections of accommodation for looked-after children before contracts were signed.

“We pay for quite a substantial amount of support – some young people in these places are having upwards of 35 hours of support each week,” she said.

In Northamptonshire, where the number of unregulated placements was similarly high, the council said the figures could include those aged 18 waiting for permanent housing.

“Northamptonshire does receive a relatively high number of unaccompanied asylum seeking children because of its location,” it said. “The county council carries out thorough quality checks on providers.”

Yvette Stanley, Ofsted’s national director for social care, said: “Over the last 12 months we’ve investigated 150 places that were not registered with us and when we went out and looked, only 30 need not register with us – the rest should have done.

She added that councils were less likely to run thorough checks on provisions when a child needed to be urgently placed elsewhere, but emphasised that Ofsted had recently improved its system for notifying them about unregistered homes.

Stanley also noted that there had been an increase in the use of supported accommodation because of progressive policies – including that fewer child criminals were being placed in secure units and those who were previously inpatients in mental health hospitals were being cared for in the community.

“The strategic issue at the heart of this very challenging problem is with the lack of supply, some people with good intentions some with bad intentions are opening provisions to meet that need,” she said.

“We don’t see councils doing this to save money, we see councils doing this because they have tried. They are really caught between a rock and a hard place. But children’s homes should be registered and supported lodgings should be of a standard that we would expect for young people moving to independence.”