Conservative whips have said they are trying to find out the identity of the Tory MP responsible for misogynistic attacks on Angela Rayner, with a view to taking disciplinary action after Boris Johnson threatened to unleash “the terrors of the earth” against the culprit.

The prime minister hit out at the “sexist, misogynistic tripe” in the Mail on Sunday, which ran allegations from an anonymous MP that Rayner, Labour’s deputy leader, deliberately tried to distract Johnson by crossing and uncrossing her legs in the House of Commons.

Following outrage across Westminster, the Speaker, Sir Lindsay Hoyle, told MPs on Monday that he had asked for a meeting with the Mail on Sunday editor, David Dillon, as well as the chair of the press gallery in the Commons.

It comes after Caroline Nokes, the Conservative chair of the women and equalities committee, wrote to Hoyle asking him to look into revoking the parliamentary pass of the article’s author, Glen Owen.

Amid pressure to unmask the MP behind the claims, a source close to the whips said: “Questions are being asked around the palace and if the anonymous source is identified, action will be taken.” However, their action appears to stop short of a formal investigation.

The Independent Press Standards Organisation (Ipso) received more than 5,000 complaints about the story, which Rayner described as “gutter journalism” while accusing the sources of “spreading desperate, perverted smears in their doomed attempts to save [the prime minister’s] skin”.

The newspaper had reported that a Tory MP claimed Rayner was adopting a “Basic Instinct” style ploy towards Johnson, in a reference to the Sharon Stone 1992 film in which she flashes a policeman during an interview.

Following the outcry, a number of female MPs spoke of their own experiences of sexism in Westminster. Nokes told the Guardian’s First Edition newsletter that she had once been the subject of an article along with former Tory MP Claire Perry “that actually compared our boobs”. Labour MP Kim Leadbeater recounted how she had been in a meeting with a male MP who “definitely spoke more comfortably … to my male staff member than he did to me”.

The shadow chancellor, Rachel Reeves, told BBC Radio 4 that a Tory MP had questioned how, with a newborn baby, she would have been able to be a cabinet minister if Labour had won the election. She said the slur against Rayner was “the sort of thing that happens day in, day out in parliament”.

The incident “shines a spotlight” on other female MPs’ experiences of sexism and misogyny, she said. “I am sick and tired of the way that female MPs and women are treated in parliament, and if this story, this outrageous slur on Angela, gets change, that would be a good thing,” she said.

Johnson suggested his party would investigate to find out who was responsible for the claims. Asked whether there was a cultural problem in parliament, Johnson said: “It’s hard to say on the basis of that particular story. But I have to say I thought it was the most appalling load of sexist, misogynist tripe. I immediately got in touch with Angela and we had a very friendly exchange.”

In a reference to King Lear, he threatened to unleash “the terrors of the earth” on the source behind the comments if they were identified. “If we ever find who is responsible for it, I don’t know what we will do, but they will be the terrors of the earth,” he said. “It’s totally intolerable, that kind of thing.”

The Speaker has not commented on the calls for Owen to lose his Commons pass but is expected to give Dillon a dressing down when he meets the newspaper’s editor. There is also a precedent for a non-MP being hauled before the House of Commons itself to apologise for an article – in 1957. On that occasion, the editor of the Sunday Express, John Junor, was brought by the serjeant at arms to the bar and admonished by the Speaker for publishing an article that cast doubt on the integrity of MPs over their constituency petrol allowances.

In the Commons on Monday, Hoyle said: “I share the views expressed by a wide range of members … that yesterday’s article was reporting unsubstantial claims [that were] misogynistic and offensive. Those are what we believe. In being demeaning, offensive to women in parliament, it can only deter women who might considering standing for election, to the detriment of us all.”

Earlier on Monday, Chris Philp, a junior minister, said he expected government whips would investigate and that if the source’s identity emerged then he would “imagine they would be subject to discipline”.

Asked why Johnson and the culture secretary, Nadine Dorries, had posted identical tweets denouncing the claims, Philp said this was “nothing surprising” because they had “reached the same view and they have used the same words”.