Unions representing museum staff fear that the culture secretary, Oliver Dowden, is seeking to “airbrush” Britain’s colonialist and racist past following a meeting with the heads of some of the UK’s most prestigious cultural institutions.

Prospect, the FDA union and PCS union wrote that their members were “deeply worried” that the government was challenging the independence of museums and galleries to provoke an unnecessary “culture war” over the portrayal of historical figures.

Dowden held a virtual meeting on Tuesday with leading heritage organisations after right-leaning media were briefed that he planned to intervene in changes to the presentation of Britain’s imperial history.

In a tweet after the meeting, he wrote that heritage organisations had agreed to draw up new national guidelines to enforce the government’s policy to “retain and explain” controversial objects, art or statues.

Union figures said Dowden’s statement will not have soothed members’ fears that the government was seeking to launch a false “war on woke” by interfering in arms-length bodies.

Prospect’s general secretary, Mike Clancy, said: “Ministers need to respect the independence and professionalism of institutions, and our members who work there, to get on with the job, and that means listening to their expertise over any fringe clamour.”

In their first unified response to the row, the unions have sent a joint letter to the National Museum Directors’ Council (NMDC), which represents the leaders of the UK’s national collections and major regional museums, urging them to stand firm against government interference.

“Our members are deeply worried that government policy and a seeming desire to ride roughshod over the arm’s-length principle will lead heritage bodies to row back from some of the important work that has been carried out to develop an understanding of collections and properties that form our cultural heritage,” the unions’ letter, seen by the Guardian, says.

The letter praises museum projects such as the National Trust’s recent report on slavery and the Tate’s statement that emphasises the importance of acknowledging “the uncomfortable and inappropriate images ideas and histories” in its collections.

“In our view any backward steps in these areas will be tantamount to airbrushing history in the way that is apparently disapproved of by the secretary of state,” the letter says.

In October, Dowden wrote to national museums and arm’s-length heritage bodies warning that they should be consistent with the government’s position and notify the department of actions or public statements they were planning to make regarding contested heritage. The letter implied that failure to comply could put government funding for the sector at risk.

Dowden is under pressure from Conservative MPs from the Common Sense Group who argue that there is a culture war raging inside the UK’s museums over how to present the country’s colonial past, accelerated by the Black Lives Matter movement.

Last month, Robert Jenrick, the communities secretary, drew criticism after claiming there was an “attempt to impose a single, often negative narrative” of British history and criticised “town hall militants and woke worthies” who sought to remove statues at the behest of “a baying mob”.

Last year, MPs criticised the National Trust after it highlighted Winston Churchill’s home, Chartwell, in a report about properties linked to slavery and colonialism. The report noted Churchill’s opposition to Indian independence in the 1930s and his leadership at the time of the Bengal famine of 1943. The trust defended its report as being factual and not representing a “judgment” on him.

After the meeting, Dowden said he and the heads of heritage organisations agreed to set up a working group which would seek to “retain and explain” contentious ideas and artefacts from Britain’s past.

A spokesperson from the Department for Digital, Culture, Media and Sport said: “We will now set up a working group to produce national guidelines on how culture and heritage bodies can put the government’s ‘retain and explain’ policy into practice, so that more people can engage in our shared past.”