From Thursday in England, staff who are neither double vaccinated nor exempt cannot enter a care home.
The warning suggests some care homes could lose large numbers of staff, worsening an existing staffing crisis.
Health Secretary Sajid Javid says the rule was announced several months ago, and would make care homes safer.
The figure, from the National Care Forum (NCF) is based on research conducted early in November with not-for-profit member organisations running 300 services, supporting 11,000 people and employing 14,000 staff.
These providers say 3.5% of their staff have already left because of the vaccination mandate, with 4.4% more who could yet leave, including those who are seeking medical exemption or have self-certified.
Self-certification of a medical exemption to being vaccinated allows care staff to continue in their jobs until 24 December.
After that, they must get a formal medical exemption, and employers are worried that some will not succeed.
Recent reports have suggested there is already an exodus among England's 1.54 million care workers, across the entire sector, from residential homes to providers of care in clients' own homes.
"The true cost of implementing this policy has been very high," says NCF chief executive Vic Rayner.
"It has absorbed a huge amount of time and energy of staff, which could have been better devoted to recruitment and the wellbeing of existing staff," she added.
"They have instead had to focus on working through dismissal procedures."
Daniela Bell is one of those care workers who have decided against vaccination.
Her last shift at the care home near Winchester where she has worked for more than three years, ended on Wednesday morning. "Heartbreaking," she says.
She herself was off work with Covid for six weeks earlier this year, and believes antibodies from the infection will protect her.
She says she does not know enough about what the vaccine contains, and is concerned about side-effects.
"I am not an anti-vaxxer and have had every vaccine until this one."
Daniela is supported by the lobby group Christian Concern, whose chief executive Andrea Williams believes: "This is an issue of personal autonomy and freedom of conscience...
"If this legislation causes the care home sector to collapse over winter, it will bring far more suffering than it prevents."
At Hill House Nursing Home near Croydon, owner Pat Robshaw says all but three of her staff have been vaccinated.
These three are self-certifying and can work until 24 December.
"After that period... we may have to say goodbye to them."
Ms Robshaw is concerned that finding replacements for them could be "exceptionally difficult".
She believes making the vaccines compulsory has backfired.
"I think as people have felt forced into having it, this has also made people U-turn and not want to have it and make their voice heard."
Alice Rogers and Francisca Gamalopes are healthcare assistants at Hill House and despite their initial fears, were eventually vaccinated.
Francisca says to begin with, she was scared by social media claims that taking the vaccine could mean you "won't have kids in the future or because we are young, problems, health problems will come".
Alice says she read similar social media warnings and "wasn't ready to take it".
However, she too eventually went ahead with both doses and is now pregnant, despite earlier worries about the vaccine's effect on her fertility: "I got pregnant after having my double dose vaccines."
She says deciding to take it was "not really about keeping my job...I took it to get safe and my residents as well."
Paul Najsarek, chief executive of the Society of Local Authority Chief Executives and Senior Managers, is confident that "the large majority of care homes will absolutely get through this successfully with support from their local authority."
He says the latest information suggests more than 90% of care staff are now double vaccinated, with almost 95% having had one jab.
He says "a small number of care homes" in areas where vaccine uptake has been lower could struggle, and some will have to limit the number of new residents they take in.
"But overall this has been well managed and the sector will come through it."
Speaking on BBC Radio 4's Today programme, Mr Javid said: "Whilst this is challenging for the sector... it is something that is manageable by the sector".
Mr Javid warned that without the policy, thousands of care workers were "more likely than otherwise to be infected with this virus because it is still out there" and could infect "people that are so vulnerable it could be fatal and I think that cannot be accepted".