Speaking to the BBC, he said "insults and misogyny is absolutely wrong whether it's a man against a woman or a woman against a man".
Misogyny refers to a hatred or prejudice towards women.
Opposition parties seized on his apparent confusion after he was corrected by the interviewer.
Labour's shadow justice secretary David Lammy said: "No wonder the Conservatives are hopeless at tackling violence against women and girls."
Liberal Democrat equalities spokesperson Wera Hobhouse said: "It's little wonder the Conservatives are failing to tackle misogyny when their justice secretary doesn't even seem to know what it is.
"These comments are an insult to the millions of women and girls impacted by misogyny and show just how out of touch the Conservatives are on this issue.
"Women and girls deserve better than these callous remarks."
During the interview, Mr Raab reiterated the government's opposition to making misogyny a hate crime, arguing it would lead to "criminalising insults".
Calls for misogyny to be made a hate crime have increased, following the conviction of Wayne Couzens for the murder of Sarah Everard, and such a move is backed by Labour and the Lib Dems.
Asked on BBC Breakfast if the government would make misogyny a hate crime, Mr Raab - who was made justice secretary in September - said it was a "legitimate" debate to have but argued that it wouldn't "solve the problem we've got".
When a crime is carried out against someone - such as assault, harassment or criminal damage - if it is proven that it was because of their race, religion, sexual orientation, disability or transgender identity, it is considered a hate crime.
There is no specific hate crime offence in England and Wales, but when a crime falls into one of the above categories, judges have enhanced sentencing powers and can increase the punishment as a result.
Campaigners say sex and gender should be added to this list, arguing misogyny is one of the "root causes" of violence against women.
Mr Raab added that "insults, and misogyny is or course absolutely wrong, whether it's a man against a woman or a woman against a man".
BBC Breakfast Presenter Sally Nugent told Mr Raab that the dictionary definition of misogyny is in fact hatred towards women.
Asked to clarify his comments later in the interview, Mr Raab said: "What I meant was, if we are talking about things below the level of public order offences of harassment, intimidation, which are rightly criminalised - if we are talking about, effectively, insults with a sexist basis, I don't think that criminalising those sorts of things will deal with the problem that we have got at the heart of the Sarah Everard case.
"Just criminalising insulting language even if it is misogynistic doesn't deal with the intimidation, the violence and the much higher level of offence and damage and harm that we really ought to be laser-like focused in on," he added.
One of those supporting making misogyny a hate crime is Sue Fish the ex-chief constable of Nottinghamshire.
She told the BBC it would be "one vital step" towards ensuring women are not subject to violence.