The prime minister said Sir Gavin had been right to resign, adding his behaviour had been "unacceptable".
Mr Sunak said he had not known of "specific concerns" about his conduct in two previous ministerial roles.
But Labour's Sir Keir Starmer accused him of failing to "stand up to a run-of-the-mill bully".
Sir Gavin was appointed minister without portfolio after Mr Sunak, a political ally, won the Tory leadership and became prime minister two weeks ago.
Mr Sunak has faced criticism over the decision in recent days, following claims of abusive behaviour by Sir Gavin towards MPs and the emergence of a complaint by a civil servant.
Sir Gavin, who has denied any wrongdoing, quit on Tuesday night, vowing he wanted to "clear my name" during probes into his past conduct.
Speaking at Prime Minister's Questions, Mr Sunak said it was "absolutely right" Sir Gavin had gone, and that his behaviour was now being investigated.
Asked by Sir Keir whether he regretted appointing Sir Gavin, Mr Sunak replied: "I obviously regret appointing someone who has had to resign in these circumstances."
Complaints against Sir Gavin emerged after expletive-laden texts he sent to then-chief whip Wendy Morton last month were published in the Sunday Times.
A civil servant then told the Guardian Sir Gavin had told them to "slit your throat" and "jump out of the window" on separate occasions during his time as defence secretary.
Sir Gavin's former deputy, Anne Milton, has also claimed he had behaved in a "threatening" and "intimidating" way towards MPs while he was chief whip.
Watch: Starmer labels Gavin Williamson a 'pathetic bully'