Rishi Sunak told Sky News "there is always the risk of disruption" to holidays during the coronavirus pandemic.
Arrivals to the UK from Belgium, the Bahamas and Andorra have been told they will need to self-isolate for 14 days.
It comes amid a surge in cases in France.
The Foreign Office is also warning against "all but essential travel" to Belgium, Andorra and the Bahamas.
Belgium has one of the highest coronavirus case rates in Europe at 49.2 per 100,000 people, compared with 14.3 per 100,000 in the UK.
For comparison, Spain's rate was 27.4 per 100,000 around the time it was removed from the UK's travel corridor list of countries exempt from quarantine restrictions.
According to the European Centre for Disease Prevention and Control, France recorded 23.4 cases per 100,000 people in the last two weeks.
Meanwhile, new infections in France rose by more than 40% on Friday, with 2,288 further cases over the past 24 hours. Some 1,604 new cases were reported on Thursday.
On Friday, the UK reported a further 98 people had died with the virus, taking the total to 46,511, with a further 871 people testing positive for Covid-19.
The total includes all deaths in people who have ever tested positive and the actual cause of death may not be Covid-19 in all cases.
Asked whether tourist destinations such as France could be next to be added to the quarantine list, Mr Sunak told the BBC the government was keeping the situation "under constant review" and that people should "be mindful" of the risk of disruption to travel given the global pandemic.
He told holidaymakers they should constantly look at government guidance and "make the best decisions they can, knowing we live in uncertain times".
He told Sky News No 10 was doing "the right thing", keeping the situation "under review on a constant basis" and consulting with scientists and medical advisers. If action was needed, the government would "not hesitate to do that", he said, in order "to protect people's health".
The new rules for Belgium, the Bahamas and Andorra have already come into force in Wales and will take effect in the rest of the UK from 04:00 BST on Saturday.
Continental rail service Eurostar said there was an increase in passengers travelling on its trains from Brussels to London on Friday, beating the deadline.
Quarantines have already been re-imposed for travellers from Spain and Luxembourg.
But people arriving in England and Wales from Brunei and Malaysia will no longer need to self-isolate, after a decrease in confirmed new coronavirus cases there.
People who do not self-isolate when required to after being abroad can be fined up to £1,000 in England, Wales, and Northern Ireland. Those returning to Scotland could be fined £480, with fines up to £5,000 for persistent offenders.
Joshua Holloway, who works at the University of Ghent, in northwest Belgium, says the "crushing" quarantine news means he will have to abandon plans to visit his family in Shropshire next week.
He says they cannot visit him, because they can't afford to miss two weeks of work when they return to the UK, adding: "I also cannot go to the UK for an additional period of two weeks prior to when I had planned to travel in order to be able to enjoy a long weekend there."
Mr Holloway called on No 10 to provide "clearer guidelines" on the number of cases a country would need to hit to trigger its removal from the quarantine exemptions list.
"With the current system, there is no indication at all as to when Belgium will reappear on the quarantine-free list," he said.
The Department of Transport has confirmed people travelling through Belgium by car from Germany or the Netherlands to the UK do not need to isolate on arrival so long as all the travellers remained in the car throughout their time in Belgium - including not getting out at service stations.
However, it is permitted to drop a passenger off in Belgium without the need to isolate, so long as they don't get back in the car.
"My husband is Dutch and he was going to go with my son to The Netherlands to see family. They hadn't seen them since before lockdown.
Ms Dalton said they would have cancelled their trip had they needed to self-isolate, due to her husband's work commitments and not wanting her son to be stuck indoors.
Meanwhile, the Department for Transport has said that rail travellers arriving in the UK on journeys which include a stop in Belgium will need to quarantine unless no new passengers boarded the train and no-one left it before getting back on.
This means Eurostar passengers travelling from Amsterdam to London will need to self-isolate, as the journey involves a change of trains in Brussels.
Eurotunnel have drafted in extra staff to help with customer queries, as people call to rearrange their travel plans between the UK and Belgium.
There are two types of people who will tell you that you cannot make a difference in this world: those who are afraid to try and those who are afraid you will succeed.