Mr Sunak told the BBC's Sunday with Laura Kuenssberg programme that his healthcare was "a personal choice".
Nursing union leader Pat Cullen said the PM "needed to come clean as a public servant".
And when asked the same question, shadow health secretary Wes Streeting said he did not use private healthcare.
In the interview, Laura Kuenssberg suggested there was huge public interest in Mr Sunak's decisions and that former Conservative prime minister Margaret Thatcher was open about her choice to use a private GP.
Mr Sunak said healthcare was "something that is private", adding he "grew up in an NHS family", with a dad who was GP, and a mum who was a pharmacist.
But when pressed again, Mr Sunak did not answer the question and instead said, in general, "we should be making use of the independent sector" so patients could choose where they have treatment.
A newspaper report in November last year suggested Mr Sunak was registered with a private GP practice that offers on-the-day appointments and charges £250 for a half-hour consultation.
The latest NHS figures show that, in November last year, 58% of NHS patients were not seen on the day they made an appointment.
At the same time, a record high of more than seven million people are waiting for hospital treatment, as the NHS faces one of the worst winters in its history.
Ms Cullen, the general secretary of the Royal College of Nursing, said public servants "ought to be clear with the public whether or not you are using private health cover".
"That's about being open, it's about being transparent and it's about honesty," she said.
Mr Streeting said the PM's answer to the question about his healthcare showed him to be someone who did not understand the biggest crisis in NHS history.
He said private healthcare created a two-tier system, but patients were free to make their own choices about treatment.
Mr Sunak has said he has a policy of not commenting on his family's healthcare arrangements, when asked previously.
Laura Kuenssberg said there was likely to be a political row over Mr Sunak's personal healthcare choices.
One former minister told the BBC presenter: "[Mr Sunak's] lack of transparency shows he thinks going private is a problem. It is - he's taking decisions on public spend that affect a version of 'the public' that he's not willing to be part of."
Some of Mr Sunak's predecessors have made a point of drawing attention to their use of the NHS when they were prime minister.
David Cameron often spoke about how the NHS cared for his disabled son, while Boris Johnson said the health service saved his life after he fell seriously ill with Covid.
But when Mrs Thatcher was prime minister she was candid about her use of private health insurance, which she said was vital for her to "go into hospital on the day I want, at the time I want, and with a doctor I want".
Mr Sunak was interviewed as senior doctors warn of a NHS on a knife edge, with health workers striking over pay and some hospitals in crisis.
A sharp rise in Covid-19 and flu admissions in recent weeks has put pressure on hospitals, which are also dealing with a backlog of treatment that built up during the pandemic.
A&E waits and ambulance delays are at their worst levels on record.
In Sunday's interview, Mr Sunak acknowledged the NHS was "undeniably under enormous pressure".
When asked if the NHS was "in crisis", he said while recovering from the pandemic "was going to be tough", he was optimistic "we can get to grips with this problem".
In his new year speech this week, Mr Sunak said bringing down NHS waiting lists was one of his five top priorities and has since held talks with health leaders to alleviate the crisis.
Watch: Rishi Sunak refuses to to say if he is registered with private GP
Watch: PM gives impression that he doesn't use NHS - Wes Streeting