It comes after The Times found debt agents for British Gas had broken into vulnerable people's homes to fit meters.
Ofgem has asked all suppliers to review the use of court warrants to enter the homes of customers in arrears.
It said firms must get their "house in order".
Jonathan Brearley, the regulator's boss, said he had ordered the review into pre-payment meters to "uncover poor practice" and that he would not hesitate to take the "strongest action in our powers" where needed.
Prepayment meters require customers to pay for their energy in advance, either through accounts or by adding credit to a card.
They are more expensive than paying by direct debit, but are sometimes the only option for people who have struggled to pay and are in debt to a supplier.
The undercover investigation by The Times revealed how agents working for Arvato Financial Solutions on behalf of British Gas had forced their way into the home of a single father-of-three to install a prepayment meter.
One debt agent is reported to have said: "This is the exciting bit. I love this bit."
Chris O'Shea, the boss of Centrica which owns British Gas, told the BBC: "There is nothing that I can say that can express the horror I had when I heard this, when I read this. It is completely unacceptable.
"The contractor that we've employed, Arvanto, has let us down but I am accountable for this."
But Mr Brearley said: "It is astonishing for any supplier not to know about their own contractors' behaviour, especially where they are interacting with the most vulnerable in our society."
British Gas has said it will suspend forcefully installing prepayment meters until at least after the winter. Arvato Financial Solutions has not commented.
A spokesman for British Gas said it had about 1.5 million customers on prepayment meters and last year had executed around 20,000 prepayment installs with a warrant. It is the country's largest supplier with 7.26 million customers.
EDF, Britain's second largest supplier, has also confirmed it is suspending the forced installation of prepayment meters and reviewing its practices.
Ovo Energy suspended its warrant activities in November and Octopus Energy said it was "not installing any at the moment" and rarely had done.
Ofgem said energy suppliers had been asked to examine their relationships with third-party contractors and to look at "incentives that could give rise to poor and unacceptable behaviours".
Mr Brearley warned no energy chief executive could "shirk their legal and moral responsibilities to protect their own customers, especially the most vulnerable".
In the case of British Gas, Mr Brearley: "We are opening a comprehensive investigation into British Gas on this issue and we will not hesitate to take the strongest action needed," he added.
Graham Stuart, the minister for energy and climate, said British Gas should "hold their heads in shame".
He told the BBC he had met all energy suppliers last week to talk about how to improve looking after vulnerable people "because there are clear rules and they have obviously not been followed".
The discovery of how some British Gas customers had been treated was "just appalling", Mr Stuart said.
"I am angry, everyone should be angry. It's completely in the face of all the promises that I've been made by suppliers," he added.
Jane, who did not want us to use her surname, came home to her recently-bought house in Poole, Dorset, in 2014 to find that someone had been in her home, installed a prepayment meter and left a letter in the kitchen.
Jane said the "horrid" experience had happened just after she had lost her unborn child at 17 weeks. She said she was mentally "broken".
The now 46-year-old said she had been sent letters addressed to a previous occupant and had posted them marked 'return to sender'. Jane added she was signed up to a direct debit plan with her energy provider, and had not missed a payment.
She called the firm and said it transpired that it had been the previous tenant who had been in arrears. She said a neighbour with a spare key was persuaded to let the installers in.
Jane said her energy provider had apologised on the phone for the mistake, removed the prepayment meter and credited her account with £45.
"It feels violating to have someone come into your house like that. It was so scary," she said. "You could sense someone had been in. The house was freezing cold."
Strict rules apply that prevent energy firms putting an at-risk customer on a prepayment meter, but in certain circumstances customers in arrears can be moved either remotely on a smart meter, or physically after the firm has been given a warrant to do so.
Suppliers are required to have exhausted all other options before installing a prepayment meter, and should not do so for those in the most vulnerable situations.
Dame Clare Moriarty, chief executive of Citizen Advice which has been calling for a ban on forced prepayment installations, welcomed Ofgem's announcement.
"The rotten core of debt collection practice in the energy sector has now been exposed for all to see. But this isn't a case of one bad apple," Ms Moriarty said.
Centrica boss Chris O'Shea: "Every one of our customers deserves to be treated with respect"