They have been added to the list of "travel corridor" countries.
But travellers coming from Slovenia and Guadeloupe will have to self-isolate for two weeks.
Both have also been added to Wales' quarantine list, while arrivals there from Gibraltar and Thailand will not need to self-isolate.
The changes come into force at 04:00 BST on Saturday.
The Department for Transport (DfT) said there had been "a significant change in both the level and pace of confirmed cases of coronavirus" in both Slovenia and Guadeloupe.
Data from Slovenia shows that its seven-day rate of cases is 29.1 per 100,000 people, up from 14.4 in the previous seven days.
The rate for Guadeloupe has risen more than six-fold in the past four weeks, the DfT said.
When a country's rate rises above 20, the UK government considers imposing quarantine restrictions.
Travellers who do not self-isolate when they are supposed to can be fined £1,000 in England, Wales and Northern Ireland, or £480 in Scotland.
Transport Secretary Grant Shapps reminded passengers they were required by law to fill in a passenger locator form.
The form asks travellers to provide their contact details and UK address. Passengers can be fined up to £3,200 in England if they do not provide accurate contact details, or £1,920 in Wales.
"This is vital in protecting public health and ensuring those who need to are complying with self-isolation rules," Mr Shapps said.
The decision to remove quarantine restrictions for arrivals from Thailand and Singapore is unlikely to lead in a surge of people from England visiting as both countries are only allowing people to enter for a limited number of reasons, such as if they have a work permit or are the spouse or child of a resident.
Denmark retained its quarantine exemption, despite its seven-day case rate being 33.8.
The DfT urged employers to be "understanding" of people returning from Slovenia and Guadeloupe, as they will need to self-isolate.
The statement gave no update on the possible introduction of testing at airports as a way of reducing quarantine requirements.
The travel industry has demanded this should take place urgently to avoid further job losses.
Earlier this week, British Airways boss Alex Cruz called for trials to be held for passengers flying between London and New York.
He said "this is imperative", adding that the airline is "still fighting for our own survival".
Last week, Sweden was made exempt from quarantine for Wales, England and Scotland.
At the same time, Portugal was placed back on England's quarantine list after a rise in infections.
It comes as coronavirus cases in the UK rose by 3,395 on Thursday, government figures showed, while deaths within 28 days of a positive coronavirus test increased by 21.
Don’t worry about people stealing your design work. Worry about the day they stop.