However, his ad-libbed line at the end of what was billed as a "major speech" in Poland on Saturday - seemingly calling for President Vladimir Putin to be removed from power - may have landed the hardest.

In his speech to a crowd of assembled Polish government officials and dignitaries at the Royal Castle in Warsaw, the US president once again warned that the world was in the midst of an era-defining conflict between democracies and autocracies.

He pledged that Nato would defend "every inch" of its member states' soil. He also promised continued support to Ukraine, although he noted that the US military would not engage with Russian forces there.

It was a confrontational, but measured, speech - well in line with what US officials, from Secretary of State Antony Blinken on down, have been saying for months.

Then, right before the "thank yous" and "goodbyes", Mr Biden added of his Russian counterpart: "For God's sake, this man cannot remain in power."

Cue the fireworks.

"This speech - and the passages which concern Russia - is astounding, to use polite words," Russia spokesperson Dmitry Peskov said. "He doesn't understand that the world is not limited to the United States and most of Europe."

And the US walk-backs.

"The president's point was that Putin cannot be allowed to exercise power over his neighbours or the region," a Biden administration official said on background. "He was not discussing Putin's power in Russia, or regime change."

The speed with which the US issued its "clarification" - later echoed by Mr Blinken - suggests the US understands the danger inherent in Mr Biden's words.

Earlier in the day, the US president had called Mr Putin a "butcher". Last week, he seemed to get ahead of his own administration's diplomatic process by accusing the Russian leader of war crimes. In both cases, Mr Biden's remarks prompted condemnations and warnings from Moscow that US-Russia diplomatic relations were being frayed to the point of breaking.

There's a line between condemning a nation's leader - the sometimes overheated rhetoric of diplomacy - and calling for his removal. It was a line both the Americans and the Soviets respected even at the height of the Cold War. And it is a line that Mr Biden had apparently crossed.

"Regime change" is something powerful countries are accused of imposing on weaker ones - not what one nuclear-armed nation demands of another.

By Sunday, even some US allies were attempting to distance themselves from Mr Biden's remarks.

French President Emmanuel Macron warned that the US president was putting ceasefire negotiations between Russia and Ukraine in jeopardy.

"We want to stop the war that Russia has launched in Ukraine without escalation," he said. "If this is what we want to do, we should not escalate things - neither with words or actions."

In Washington, congressional leaders were also expressing concern. Idaho Republican Jim Risch, the senior Republican on the Senate foreign relations committee, called Mr Biden's remarks a "horrendous gaffe".

"My gosh, I wish they would keep him on script," he said. "Any time you say or even, as he did, suggest that the policy was regime change, it's going to cause a huge problem. This administration has done everything they can to stop escalating. There's not a whole lot more you can do to escalate than to call for regime change."

A history of off-the-cuff remarks

Mr Biden has a well-known propensity for making off-the-cuff remarks that can land him in difficult situations. They've derailed past presidential bids and occasionally frustrated Obama administration officials when he was vice-president. Those lines are not entirely accidental, however. They can reveal where Mr Biden's heart lies, even if his head - and the heads of those around him - might prefer he hold his tongue.

That, at times, is a political strength, writes The Atlantic's Tom Nichols, allowing Mr Biden to authentically connect with the emotions of the American people. But, in this current moment of diplomatic crisis - when poorly chosen words can lead to consequential actions - they're also a weakness.

"It is hard to blame Biden for giving in to his famous temper after talking to the people who have suffered from Putin's barbarism," Mr Nichols writes. "But the words of every world leader matter right now, and none more than those of the president of the United States."

Mr Biden may believe that US-Russia relations are damaged to the point that they will not be repaired while Mr Putin is in power. Explicitly saying so, however, could make the immediate US goal - ending the war in Ukraine while preserving the nation's territorial integrity - more difficult.

The war in Ukraine is not going the way Mr Putin intended so far. His army is bogged down in bitter fighting and casualties are growing. His economy is straining under the weight of economic sanctions. Russia is increasingly isolated from the rest of the world.

The situation has potential to lead to the de-escalation the US and its allies are calling for - but it also could make Mr Putin more desperate. And if the Russian leader believes his power is at stake, and believes the US is not-so-secretly making that an objective of its efforts, the turn the crisis takes from here may not be one toward peace.