People "just have to have sense", said the UK government's chief medical adviser, Prof Chris Whitty.
Coronavirus rules announced this week mean three households can form a bubble and mix for five days over Christmas.
From 23 to 27 December, three households in can mix indoors in homes, at a place of worship or outdoors.
The rules apply to the whole of the UK, although in Scotland the number of people who can be in the Christmas bubble is limited to eight.
And in Northern Ireland, the rules are relaxed from 22 to 28 December, to allow time to travel between the nations.
Speaking at a Downing Street press conference on Thursday, Prof Whitty - who revealed he would be "on the wards" over Christmas - said: "Would I want someone to see their family? Of course, that's what Christmas is about.
"But would I encourage someone to hug and kiss their elderly relatives? No, I would not.
"It's not against the law - and that's the whole point. You can do it within the rules that are there, but it does not make sense because you could be carrying the virus and if you've got an elderly relative, that would not be the thing you'd want to do in the period where we are running up to a point where we actually might be able to protect older people.
"So I think people just have to have sense. The fact that you can do something - this is true across so many other areas of life - doesn't mean you should."
Sir Patrick Vallance, the government's chief scientific adviser, added: "It's not going to be a normal Christmas but if you want to make those connections with family, it has to be done in a way where you try and make sure that you don't increase the risk.
"I think hugging elderly relatives is not something to go out and do. It will increase the spread to a vulnerable population."
Prof Whitty added: "If you want them to survive to be hugged again."
Prime Minister Boris Johnson also responded to the question about hugging elderly relatives, urging people to be "common sensical".
"Until the vaccine comes on stream, we are not out of the woods yet and we have to be very, very vigilant."
Prof Whitty also said it was "not a secret" that Christmas would increase the risk of transmission.
"Take it really seriously during Christmas. Don't do stupid things. Don't do unnecessary things just because the rules say you can. Think sensibly."
Scotland's First Minister Nicola Sturgeon has that said the "default advice" and "safest position" was still that people should avoid contact.
"Just because we are allowing people to meet up in a limited way does not of course mean people have to do so, and people should not feel under pressure to do so," she said.
The government's official guidance on Christmas bubbles advises people with loved-ones who are vulnerable advises to take personal responsibility to limit the spread of the virus.
The NHS considers anyone 70 and over as "clinically vulnerable" and at moderate risk from coronavirus.
The government guidance also has specific advice for people considered extremely vulnerable, as well as care home residents.
It suggests forming a Christmas bubble is "a personal choice" for extremely vulnerable people, while those in care homes should only visit families if they are of working age.
Under the government's rules, the three households must be fixed, so you will not be able to mix with two households on Christmas Day and two different ones on Boxing Day. Households in your Christmas bubble can't bubble with anyone else.
Scotland has announced that the bubbles of three households should contain no more than eight people - but children under 12 are exempt.
People who are self-isolating should not join a Christmas bubble. If someone tests positive, or develops coronavirus symptoms up to 48 hours after the Christmas bubble last met, everyone will have to self-isolate.
The cure for boredom is curiosity. There is no cure for curiosity.