His comments came after the government announced quarantines on travellers from some African countries following the emergence of a new strain.
Prof Whitty said he questioned whether "we could take people with us" if restrictions had to be imposed.
England has been through three national lockdowns since Covid first struck.
There have also been many local restrictions imposed at various points during the past 20 months.
Prof Whitty told a panel discussion hosted by the Local Government Association: "My greatest worry at the moment is that people... if we need to do something more muscular at some point, whether it's for the current new variant or at some later stage, can we still take people with us?"
But he added that, despite previous restrictions being "very destructive" to society and the economy, the public had shown an "extraordinary" ability to "just accept that there are things we collectively have to do to protect one another and do collectively".
Prof Whitty added, however, that it had become harder to ensure compliance over time, saying: "It's easier to be confident of people's response right at the beginning than it is after people put up with two years of their lives being interfered with."
There have been a further 50,091 Covid cases in the UK and another 160 deaths within 28 days of a positive test, the latest government figures show.
Both deaths and the number of people admitted to hospital continue to trend lower despite the rise in cases. It is likely we're seeing the impact of the booster campaign protecting the most vulnerable people and keeping them out of hospital.
On Friday, Health Secretary Sajid Javid told the House of Commons it was "highly likely" the B.1.1.529 variant - which has been named Omicron by the World Health Organization - had already spread from South Africa and Botswana. No cases have yet been detected in the UK.
Travellers from these countries, as well as Namibia, Zimbabwe, Lesotho and Eswatini, will have to self-isolate for 10 days, with those arriving in England after 04:00 GMT on Sunday having to quarantine in a hotel.
Former Health Secretary Matt Hancock told BBC Radio 4's PM programme: "The government are right to be cautious and they've moved very swiftly to bring these travel restrictions in, and that's the right thing to do, especially as there's so much uncertainty about this new variant."
He added: "I hope very much that we can avoid further restrictions here, not least because of the breadth of the vaccination programme."
But there are fears the new variant might be more transmissible, that vaccines might be less effective against it and that it might affect one of the UK's major treatments, Ronapreve.
Belgium has become the first European country to confirm a case of the new variant. The European Union is already struggling to cope with fresh waves of coronavirus, and several countries have re-imposed lockdowns and restrictions.
Since Thursday a number of countries have placed restrictions on travel from southern Africa amid concerns about the variant.
The EU has agreed to ban flights from Botswana, Eswatini, Lesotho, Mozambique, Namibia, South Africa and Zimbabwe, following a call to do so from the European Commission on Friday.
The US has also banned visitors from those countries, and from Malawi.