The scale of credit card, identity and cyber-fraud makes it the most prevalent crime, costing up to £190bn a year.
UK intelligence agencies should play a greater role in responding, the RUSI argues in a report.
Policing should be better resourced, working more closely with the private sector, it adds.
The report argues that the scale of fraud against the private sector has an impact on the reputation of the UK as a place to do business.
Meanwhile, the amount lost by the government in fraudulent claims represents a "heist" on the public purse, undermining faith and trust, it says.
It is the crime UK citizens are most likely to fall victim to, but the failures in responding risk undermining public confidence in the rule of law.
The Crime Survey for England and Wales found 3.7 million reported incidents in 2019-20 of members of the public being targeted by credit card, identity and cyber-fraud.
The private sector takes the biggest financial losses. One estimate from 2017 put the cost of fraud to businesses at £140bn.
Fraud against the public sector, including benefit, tax credit and student loan fraud, is estimated to cost £31-48bn a year, the upper figure larger than the UK's annual defence budget.
The losses go beyond the financial, the authors say.
"Fraud has the potential to disrupt society in multiple ways, by psychologically impacting individuals, undermining the viability of businesses, putting pressure on public services, fuelling organised crime and funding terrorism," they add.
The report cites evidence that terrorist groups and lone actors turn to fraud in order to finance their activities.
In one case, eight supporters of the Islamic State group were convicted of defrauding UK pensioners out of more than £1m, which was alleged to be used in part to fund travel from the UK to Syria.
The men carried out a type of courier fraud in which they pretended to be police officers, telling victims that their bank accounts had been compromised and needed to be transferred.
But despite the growing scale of the problem, there is no national strategy for tackling the issue, while the police response is underfunded and lacking focus.
This makes fraud "everyone's problem but no-one's priority", according to the report, written by RUSI experts Helena Wood, Tom Keatinge, Keith Ditcham and Ardi Janjev.
The digitisation of everyday life - accelerated by Covid - has only increased the risks, with organised crime groups showing increased sophistication in their tactics.
"The UK has become a target destination for global fraudsters," the RUSI argues.
But the extent to which international criminals focus on the UK is hard to gauge, because intelligence agencies have not traditionally focused on the issue.
One senior fraud professional interviewed by the researchers said that despite 30 years of investigating fraud, they still had no idea what proportion of the threat emanated from overseas.
Classifying fraud as a national security issue would help ensure the right level of resourcing and prioritisation, the authors argue.
They also recommend more focused intelligence direction from the National Security Council, including greater tasking for GCHQ as well as the National Crime Agency to understand the issue.
They call for better information-sharing and use of data analytics, as well as more money and attention from police forces to address what they call a "responsibility vacuum".
Individual rights are not subject to a public vote; a majority has no right to vote away the rights of a minority.