Some industries have warned firms could be forced to shut down operations.
Mr Kwarteng said he was working closely with the chancellor over possible support for energy intensive sectors - but a Treasury source denied this.
The business secretary said domestic customers would not see a change to the energy price cap this winter.
Asked on BBC One's Andrew Marr programme whether there would be additional government help for energy-intensive companies, Mr Kwarteng described the situation as "critical" and said he was "looking to find a solution".
When Andrew Marr suggested this sounded like a "yes" the business secretary said: "No, it doesn't sound like yes at all.
"We already have existing support and we're looking to see whether that's sufficient to get us through this situation."
Speaking to Times Radio Mr Kwarteng, who met leaders from heavy industry on Friday, said he was not going to commit to "any firm figure or subsidy" for companies.
Asked about whether the government would ensure factories would not have to close if they could not pay for gas he said it was a commercial decision and "up to them".
He added: "We are not in the business of bail-outs. What we are in the business of is ensuring security of supply and that is what I am focused on."
Wholesale gas prices have risen 250% since January.
CEO of British Glass Dave Dalton, who was at Friday's meeting with Mr Kwarteng, said some of the confederation's "significant" members were "teetering on the edge".
"I think some companies are staring down the ability to survive, absolutely - ultimately that obviously cascades on to jobs and impacts on the consumer," he told the BBC.
Gareth Stace, director general of UK Steel, said he was frustrated by the lack of action to support businesses.
He told the BBC that without help in the next week or so, there would be "significant and permanent damage to the UK steel sector".
Unite leader Sharon Graham said the country was "contemplating factory shutdowns across viable manufacturing and businesses" and that workers were "worried sick".
Businesses have been shouting louder and louder for support through this period of soaring energy prices.
This morning, the business secretary told the BBC he was listening to their concerns - but would not commit to any extra support.
Those industries that use a lot of energy for manufacturing say that the time for working out a way forward has long gone.
The director general of UK Steel, Gareth Stace, expressed his frustration, saying pauses in steel production will only increase.
The government says the current situation emphasises the need for a revolution in how we generate energy, moving towards home-grown renewables.
But that's little comfort for those businesses dependent on energy from fossil fuels now, competing with intense demand in a global market.
On the Andrew Marr show, Mr Kwarteng denied asking for "billions" from the Treasury to subsidise energy-intensive businesses and said supply itself was "not an issue".
A Treasury source said the business secretary had been "mistaken" to say that he had been working on possible support measures with the Chancellor Rishi Sunak.
Bridget Phillipson, Labour's shadow chief secretary to the Treasury, said the government "needs to get a grip" and called for "urgent answers on who exactly is running the show".
"The two key government departments responsible for the current cost of living crisis have spent this morning infighting about whether they were in talks with each other. What a farce," she said.
She also accused the government of having "put its out of office on", referring to reports that the prime minister is on holiday in Spain.
A number of Conservative MPs have called for the government to take action to support heavy industry.
The SNP's Westminster leader Ian Blackford called on the UK government to "nurse" businesses through the crisis, describing it as a "perfect storm".
The domestic consumer energy price cap, which is reviewed every six months, sets the maximum level a supplier can charge a consumer on a standard tariff in England, Wales and Scotland.
Mr Kwarteng told Marr that protecting consumers was his "first and foremost objective" and as such the price cap would stay at its current level until its next update which is due to in April.
Some suppliers say the cap is just delaying an inevitable increase in consumer prices and should be reviewed more regularly.
Energy regulator Ofgem has warned households will see further "significant rises" in the spring, when the cap is reviewed.
Asked by Marr if he was sure the lights would stay on this winter, Mr Kwarteng said "yes, I am".
Due to high gas prices household energy suppliers have been forced to sell gas for less than they can buy it due to the price cap, leading some to fail.
Last month, nine domestic energy supply companies went out of business, forcing 1.7 million customers to move to new suppliers and on to higher rates.
Paul Richards, chief executive of Together Energy, which he said is currently making losses, said while he supported a price cap to protect customers, the current mechanism "is not fit for industry, nor is it fit for customers".
He said it protected customers in the short term but somewhere between £1bn and £3bn in costs would be spread back across business and households as a result of suppliers going bust.
The founder of OVO Energy Stephen Fitzpatrick told Marr that it has been "too easy" for companies to enter the energy market and that there will be more companies in difficulty.
He said the market was a complicated one, and he thought some people had not understood the risks.