Sajid Javid has apologised for saying it was time to stop “cowering” from Covid, after an outcry from families bereaved by the virus.

On Saturday, the health secretary said on Twitter that he had recovered from coronavirus – which he contracted despite having had two doses of the vaccine. “Please – if you haven’t yet – get your jab, as we learn to live with, rather than cower from, this virus,” he said.

His use of the word “cower” led some to accuse him of failing to understand the concerns of those whose underlying conditions make them particularly vulnerable to the virus – or who have lost loved ones.

On Sunday, after the campaign group Covid-19 Bereaved Families for Justice called the remark “insensitive” and urged Javid to meet them at the Covid memorial wall, he issued an apology and said he had deleted the tweet.

“I was expressing gratitude that the vaccines help us fight back as a society, but it was a poor choice of word and I sincerely apologise,” Javid said. “Like many, I have lost loved ones to this awful virus and would never minimise its impact.”

Covid-19 Bereaved Families for Justice had written to the health secretary, saying the flippancy of his message “betrays the sacrifice so many of our loved ones have made”.

Labour joined condemnation of Javid’s language. The shadow disabilities minister, Vicky Foxcroft, said: “In the first and second waves more than 3 million people shielded at the request of the government. Most have been happy to do this as we know this has kept us safe.

“These people aren’t cowering away from the virus, they are just trying to stay safe.”

Javid’s arrival as health secretary appeared to mark a more bullish phase in the government’s management of the disease, in which ministers have pressed ahead with lifting almost all legal restrictions despite rapidly increasing cases.

Within days of arriving in his post, after the departure of Matt Hancock, Javid claimed the lifting of restrictions would be “irreversible” and there was “no going back”.

The word “irreversible” has since disappeared from the government’s messaging, however, amid careful monitoring of the impact of the “exit wave” as restrictions are lifted.

New cases of the virus have declined, with Sunday’s figure of 29,173 down almost 18% from the same day last week – though the impact of “freedom day”, when nightclubs reopened and mass events went ahead – appears unlikely to have shown up in the figures yet.

Javid tested positive for Covid on 17 July, prompting the prime minister and the chancellor to have to self-isolate.

The pair initially sought to avoid quarantine, announcing they had been enrolled in a pilot programme allowing participants to be tested each day instead of staying at home. But that decision was reversed within hours after an outcry, including from Conservative backbenchers.

Javid was meeting Rishi Sunak and Boris Johnson to discuss issues including how to upgrade the creaking social care system, something the prime minister promised to “fix” when he arrived in Downing Street two years ago.

The trio appeared to be closing in on a proposal that would mean a £50,000 cap on care costs for individual households would be paid for with a 1p increase in national insurance for both employers and their staff.

But the announcement was pulled after the three key figures went into isolation, with a final decision expected to be made by the prime minister over the summer. Some cabinet ministers are wary of the idea of a national insurance increase, fearing it could be described as a “tax on jobs”.

The former permanent secretary to the Treasury Nick Macpherson told Times Radio on Sunday: “National insurance is only paid by people who work and so this is effectively a tax on employment. It’s a tax on jobs. And if you tax something more, generally, you’ll get less of it.

“But there’s also a distributional issue. National insurance is paid only by people who are below pension age. It’s paid by young people who have taken a pretty big hit in recent years, what with tuition fees and the general stagnation in wages.”