Getting people back into the labor force and into a job is now key to keep the recovery going and stop the economy from overheating, according to Minneapolis Federal Reserve President Neel Kashkari.

"We have to find a way to bring [women] back to work. This is about our economic potential," Kashkari told CNN's Poppy Harlow on Tuesday. "It is certainly about fairness about women and families. But it is also about our economic potential as a nation."

The US Chamber of Commerce said Tuesday that the nation's worker shortage is getting worse by the day.

Experts are struggling to disentangle the biggest reasons why people might choose to stay home, including access to child care, persistent virus risks and higher-than-usual unemployment benefits. Women have been hit particularly hard during the pandemic, as child care responsibilities still tend to fall on them and they were more likely to be forced to leave the workforce.

"I believe we're going to bring women back into the labor force and workers who have been displaced," said Kashkari.

Failing to do so would make the higher inflation readings more concerning, as it would call into question the efficacy of all the government stimulus.

Kashkari believes higher inflation numbers aren't here to stay and are just a temporary effect from the economy getting back to normal. That has been the Fed's go-to line for months now. Economists predict that prices might continue to rise faster than normal during the summer months.

Since the recovery picked up steam in the face of the full reopening, economists, investors and lawmakers have been concerned about the economy overheating.

With plenty of fiscal and monetary stimulus in the system, some worried that too much of a good thing would turn inflation up so high that people might stop spending. That would be very bad news for the consumer spending-dependent US economy.

Worrying about this might not be a bad thing.

"There was a lot of complacency in the years following the Great Financial Crisis when millions of Americans were on the sidelines. So it took 10 years to rebuild the labor market," Kashkari said.

"We cannot have another 10-year recovery. So while I appreciate the fact that some people are worried about inflation, they didn't seem that worried when millions of Americans were on the sidelines in the last recovery," he added.

As of April, America was still down more than 8 million jobs compared with February 2020, before the pandemic lockdown. Economists expect to see 664,000 jobs added back in May in next week's jobs report.

While that would be a great number in normal times, it hardly moves the needle in the pandemic economy.