The Prince of Wales has called for small family farmers in the UK and across the world to come together in a cooperative movement using sustainable farming methods, and for their plight to be at the centre of environmental action.

Small farmers, in the UK and EU, are facing their biggest upheavals in more than a generation, with the loss of farm subsidies and new post-Brexit trade deals in the UK, and sweeping reforms to the EU’s common agricultural policy to be announced this week in Brussels.

Writing for the Guardian, Prince Charles has urged small farmers to band together to cope with the coming shocks and shift to a low-carbon economy: “There are small farms the world over which could come together in a global cooperative committed to producing food based on high environmental standards … With the skills of ethical entrepreneurs and a determination from the farmers to make it work, I would like to think it could provide a very real and hopeful future.”

Farming is undergoing a “massive transition”, and the needs of family farmers must be taken into account, the prince said.

“To me, it is essential the contribution of the small-scale family farmer is properly recognised – they must be a key part in any fair, inclusive, equitable and just transition to a sustainable future. To do this, we must ensure that Britain’s family farmers have the tools and the confidence to meet the rapid transition to regenerative farming systems that our planet demands,” he said.

Analysis of farming data for the Guardian has shown that small farmers were already facing an increasingly difficult future, before the shocks of Brexit and the Covid-19 pandemic. The EU has lost vast numbers of livestock farms in particular, with 3.4m gone between 2005 and 2016, the latest year for which full data was available.

At the same time, the number of livestock on farms has increased on average, a clear sign of intensification in the sector.

John and Mary Atkinson on their small-scale farm in Nibthwaite Grange, the Lake District, an area where many farming families have been forced to leave.

In the UK, a quarter of livestock farms, amounting to 45,500 farms, were lost in 12 years from 2005 to 2016. That loss was part of a longer-term trend for all farms, with more than 110,000 farms gone from the 319,000 farms in 1990.

Many farmers have warned that Brexit could hasten the loss of smaller farms, as the UK’s markets open up to lower-cost imports from outside the EU that previously faced tariffs and other barriers. The government has been phasing in new support for farmers, based on payments for providing public goods such as tree planting, wildlife protection and nurturing the soil, but it is still unclear how these will work in practice.

Ministers also announced a consultation last week on giving lump sums to farmers who want to retire, accompanied by support for people who want to enter farming, but cannot afford to. However, some farmers concerned that the scheme will encourage a further exodus of small farmers.

The Prince of Wales has long been a supporter of sustainable farming, and earlier this year launched Terra Carta, a roadmap to 2030 for businesses to move towards a low-carbon and environmentally sustainable future. He said this could provide a template for farmers coming together in cooperatives to reach consumers who are increasingly interested in buying locally and from small-scale producers.

“These [small] farmers are some of the most hard-working and innovative small businesses and, in so many ways, we depend on them far more than most of us will ever know,” the prince wrote.

Minette Batters, president of the National Farmers’ Union, echoed his views: “We would not want to see a loss of the traditional family farm. We would lose the culture and heritage of this country, where 70% of the land is farmed and the expectation is that at the end of every farm track is a family. Our national identity is built on this.”

Small farmers in the UK are increasingly worried about what Brexit will mean for them, with the government’s approaching trade deal with Australia provoking widespread unease. Many farmers told the Guardian they were alarmed at the impact an influx of cheap meat and other produce could have on small farmers and the British countryside.

Liz Lewis, who with her husband, David, farms 650 hectares (1,600 acres) in north Wiltshire, including a beef cattle herd producing 100 calves a year, said: “This will be curtains for small livestock farmers. There’s a lot fewer already. It won’t happen overnight, it will be a slow burn, but in a few years you will see it.”

She warned that the UK’s landscape would be transformed, as has happened in the US, into widespread factory farming with a green veneer. “It’s sad for the younger generation – it’s really hard to see how they can make money unless they intensify, by going for big feedlots [of the kind common in the US],” she said. “You have to imagine what the countryside will be like in 10 or 20 years’ time – it will not be what we’re used to. There will be more trees, but behind those trees there will be industrialised farms.”

Tim Ashton, in north Shropshire, said the impact of the loss of small farms was already evident, even in the most rural areas. “What we’re seeing around us is social cleansing. Local people can’t afford to live here and small farms are disappearing. There is a lot of gentrification happening – you expect to see that in Cornwall or the Cotswolds but now it’s even coming here.”

Consumers would also lose out from the decline of small farms, said Ruth Hancock, a self-described “new entrant” who faced many barriers to setting up as a small farmer. She said: “We are at great risk of creating a two-tier food system akin to that in the US, where we may end up having a few smaller or organic producers supplying the concerned and better-off citizen. Meanwhile, the vast majority have to make do with the lowest common denominator imports, because they can’t afford to buy better or don’t understand the difference.”

A spokesperson for the Department for Environment, Food and Rural Affairs said: “Our landmark plans for a renewed agricultural sector will transform the way we produce food and support farmers in England. Through these changes, rather than paying farmers and landowners for the amount of land they manage, we will pay them to provide environmental and animal health and welfare outcomes. We are phasing out the current subsidy, the basic payment scheme, in a phased and progressive way. This means that initially those who claim smaller amounts of subsidy will receive a smaller reduction in their payments when we make the first reduction later this year.”