Police were called to the blaze on the 85ft (26m) vessel at Princess Pier in Torquay, Devon, at about 12:10 BST on Saturday.
Members of the public described hearing a bang, before seeing a huge plume of black smoke. No injuries have been reported, Devon & Cornwall Police said.
Emergency services declared a major incident and officers evacuated a nearby beach and roads.
The boat, which police said broke from its mooring, was secured by the fire service near the marina's pier before it sank.
The Environment Agency (EA) said it was believed to have had about nine tonnes of diesel on board.
A police spokesman said nearby residents should keep their doors and windows shut "due to the levels of smoke and fumes".
He added that the cause of the fire was currently being treated as unexplained.
Joseph Barlow said he heard a loud noise and then saw the fire as walking to the nearby beach.
"I heard a bang like a gun shot from the harbour, followed by big plume of black smoke," he said.
Mr Barlow said he then saw "a fire coming from the boat in the harbour" and that shortly after firefighters arrived and the area was cordoned off by police.
BBC Devon reporter Louise Mitchell said it looked like the fire "was at the end of a pier".
"You could certainly see massive flames reaching up into the air and the plume of smoke was just enormous," she said.
She added that coastguards had been evacuating the area around Torre Abbey Meadows.
In a tweet, the EA said as the vessel had now sunk, the focus was "now on pollution".
An EA spokesman added that the response to that threat would be led by the harbourmaster, while the agency would provide "advice and guidance on bathing water quality".
Torquay-based musician David Balmer, 58, said the sinking of the yacht had raised fresh concerns about the fuel on the boat potentially leaking into the water.
"That could create another hazard in the harbour," he said.
He said there were "masses of people around" when the yacht caught fire, adding: "It was engulfed in flames and there was this huge cloud of black smoke."
"It was quite a spectacle, it was just an unbelievable sight," he said.
He added that onlookers had been told to leave the area because "there might be an explosion" and the crowds were moved a safe distance away.