Plans to introduce a nationwide age verification system for online pornography have been abandoned by the government after years of technical troubles and concerns from privacy campaigners.

The climbdown follows countless difficulties with implementing the policy, which would have required all pornography websites to ensure users were over 18. Methods would have included checking credit cards or allowing people to buy a “porn pass” age verification document from a newsagent.

Websites that refused to comply with the policy – one of the first of its kind in the world – faced being blocked by internet service providers or having their access to payment services restricted.

The culture secretary, Nicky Morgan, told parliament the policy would be abandoned. Instead, the government would instead focus on measures to protect children in the much broader online harms white paper. This is expected to introduce a new internet regulator, which will impose a duty of care on all websites and social media outlets – not just pornography sites.

She said: “This course of action will give the regulator discretion on the most effective means for companies to meet their duty of care.”

Despite abandoning the proposals, Morgan said the government remained open to using age verification tools in future, saying: “The government’s commitment to protecting children online is unwavering. Adult content is too easily accessed online and more needs to be done to protect children from harm.”

The decision will disappoint a number of British businesses that had invested substantial time and money developing verification products. They had been hoping to capitalise on the large amount of Britons expected to verify their age in order to view legal pornography. One age verification provider estimated the potential market was as many as 25 million people.

Although the age verification policy was first proposed by the Conservatives during the 2015 general election, it took years to develop and make it into law. Its implementation date was then repeatedly delayed amid difficulties with implementing the policy.

The British Board of Film Classification was tasked with overseeing the system, which would be run and funded by private companies, despite the organisation’s lack of historical expertise in the world of technical internet regulation. Some of the age verification sites had close links to existing pornography providers.

Concerns over the system grew as the public became increasingly aware of the approaching implementation date.

Despite repeated reassurances from pornography websites and age verification sites that personal details would be kept separate from information about what users had watched, privacy campaigners continued to raise concerns about data security.

In addition, earlier this year the Guardian showed how one age verification system could be sidestepped in minutes. Proponents of the policy privately accepted it would not block a persistent teenager from accessing adult material but said it could stop younger children from stumbling across images they found deeply disturbing.

The policy had the backing of charities such as the NSPCC that were concerned about the impact of pornography on children.

The final blow to the porn block came from an unlikely source: the European Union. Just weeks before the policy was due to be finally implemented in July, the government realised it had failed to inform the EU of its plans.

This administrative error was initially announced as requiring a six-month delay – but Morgan’s announcement, made on a day when media attention was focused on the Brexit negotiations, means the age verification system has now been abandoned in its current form.