Boris Johnson has said he does not “think that biological males should be competing in female sporting events”, amid the fallout from his decision not to ban conversion practices for people questioning their gender.

Speaking on Wednesday, the prime minister said the issue “wasn’t something I thought that I would have to consider in great detail”. Johnson also said that women should have spaces in hospitals, prisons and changing rooms which were “dedicated to women”.

“That’s as far as my thinking has developed on this issue,” Johnson told a journalist during a visit to a hospital in Hertfordshire. “And if that puts me in conflict with some others, then we have got to work it all out.”

He added: “That doesn’t mean that I’m not immensely sympathetic to people who want to change gender, to transition, and it’s vital that we give people the maximum possible love and support in making those decisions.

“But these are complex issues. And I don’t think they can be solved with one swift, easy piece of legislation … It takes a lot of thought to get this right.”

He also insisted it was right to exclude people who were questioning their gender from a long-promised ban on so-called conversion “therapy”.

After the resignation of a senior equality adviser and cancellation of the government’s flagship international LGBT conference in the face of a mass withdrawal of support from stakeholders, the prime minister said he was “sad” at their response.

Despite some Tories pressing for No 10 to reverse its decision to take forward a ban on conversion practices only for those questioning their sexuality – not their gender – Johnson appeared to confirm that the decision taken last week was final.

Johnson said: “I suppose I’d just make a few points: I don’t think that it’s reasonable for kids to be deemed so-called Gillick competent to take decisions about their gender or irreversible treatments that they may have. I think there should be parental involvement at the very least. That’s the first thing.

“Second thing, I don’t think that biological males should be competing in female sporting events. And maybe that’s a controversial thing, but it just seems to me to be sensible.”

The prime minister added: “We will have a ban on gay conversion therapy, which to me is utterly abhorrent. But there are complexities and sensitivities when you move from the area of sexuality to the question of gender. There, I’m afraid, there are things that I think still need to be worked out.

“I’m sorry we haven’t been able to reach agreement with the organisations concerned but that will in no way diminish our determination to tackle prejudice wherever we can.”

Johnson, and the previous government led by Theresa May, had promised for years to institute a ban on conversion practices, which critics say subject those questioning their sexuality or gender to undue pressure or, in the most extreme cases, torture.

On Tuesday, Iain Anderson, the government’s LGBT+ adviser, stepped down and accused ministers of trying to drive a wedge between those who were gay, lesbian or bisexual and people who identified as transgender.

He said: “Britain needs a strategy for trans people and I can’t see one at the moment. We have a tabloid debate going on about people’s lives. It’s not a respectful debate; it’s turned into a woke war.”

The Tory MP Dehenna Davison also expressed concern at the cancellation of the Safe To Be Me conference, due to be held in June to put pressure on countries to enact stronger rights for LGBT+ people.

“We had such a huge opportunity to prove the UK (and the Conservative party) is a defender of freedom,” she posted on Twitter. “As a Conservative member of the LGBT+ community, it is so wrong it has come to this.”

The Scottish Conservatives on Wednesday diverged from the Westminster party as they committed to voting for a ban on all conversion practices – including a ban on trans conversion practices – in forthcoming Holyrood legislation.

But the Scottish party also announced that “supporting women’s spaces in council-run venues including schools, parks and swimming pools” would be a key plank of its manifesto for the May council elections.