Five White House staffers have been fired a result of prior marijuana use, weeks after the Biden administration said it would loosen these policies in an effort to modernize and expand the pool of people who can work there.

Officials pushed back on Friday on a report from the Daily Beast that “dozens” of staffers had lost their jobs or been forced to work remotely due to their past weed use, saying the number was actually about a dozen. The officials added that there were additional security issues for some of the fired staffers, such as hard drug use.

“Of the hundreds of people hired, only five people who had started working at the White House are no longer employed as a result of this policy,” White House press secretary Jen Psaki tweeted on Friday.

The roles and identities of those let go were not made public.

The White House has not published any official guidance on the new policy, but in an NBC article from last month, an official said that past marijuana use would not automatically disqualify staffers.

Normally, White House staffers must receive top secret security clearance, for which drug use can be disqualifying.

To get around this, staffers who have used marijuana only on a “limited” basis and whose jobs don’t require this security clearance would be given a waiver to work without the clearance.

The policy does not actually allow staffers to use marijuana or even to have previously used it more than once or twice a year. According to the Associated Press, “limited” use of marijuana means fewer than 15 times in the past year.

Staffers with clearance waivers face additional rules and scrutiny, officials said. They must take a pledge to stop using marijuana completely while working for the government, agree to random drug testing, and may be directed to work from home until they meet clearance standards.

Though the Biden administration has been more permissive than previous presidential administrations, the terminations have faced widespread criticism, particularly because marijuana has been legal in DC for over five years. It remains illegal at the federal level.

Many local and national advocates for legalization are denouncing the White House’s actions.

Adam Eidinger, cofounder of DC Marijuana Justice, told BuzzFeed News he found the firings “outrageous,” saying there’s no evidence that staffers using marijuana in their free time would affect how they perform in their jobs.

“You should only be looking at someone’s job performance to determine whether they are qualified to work in a place — not whether they used marijuana in the past or are currently using it,” he said.

Disciplining employees because they’ve used marijuana outside of working hours is a harmful and hypocritical precedent to reinforce, particularly from the nation’s highest office, National Cannabis Industry Association spokesperson Morgan Fox told BuzzFeed News.

“It’s definitely sending the wrong message, not just to federal employers, but to policy makers and private sector employers around the country,” Fox said. “It kind of reinforces the false notion that cannabis is a detractor from somebody’s ability to do difficult jobs with a lot of responsibility.”

“Just look at the fact that three of the five last presidents have admitted to past cannabis use,” Fox said, “including Biden’s former boss.”