The border is seen as an entry point to the EU, and there have been tensions in recent weeks with Belarus being accused of pushing migrants towards it.
Ben Wallace said a small reconnaissance team of engineers had already been to the area to assess the situation.
They would not be combat troops but "guys with diggers", he added.
Belarus's long-time authoritarian leader, Alexander Lukashenko, denies facilitating the arrival of migrants and sending them towards the border with Poland in revenge for EU sanctions imposed in June.
The crisis deepened on Tuesday when Polish forces used tear gas and water cannon to repel those trying to cross into the country.
Some attempted to breach a heavily fortified crossing and threw missiles at the Polish forces, leaving several with injuries.
Mr Wallace said the engineers would be dispatched "within days or weeks" but said he would first inform parliament about the planned deployment.
Speaking to the BBC during a visit to Poland Mr Wallace said: "What we will do is send some Royal Engineers - that is part of the Army designed for building or making fences or roads or putting in infrastructure.
"We are going to be using that part of our forces to help the Poles and potentially other Baltic states to secure their border."
He stressed the soldiers would not be combat troops but people with "the ability to build roads and checkpoints and observation posts", adding: "This is not combat this is support to the Poles."
On the humanitarian situation, the defence secretary said: "I'm particularly worried for the women and children and the vulnerable people who are being trafficked by the Belarusians into this game they seem to be playing.
"It's a horrendous thing to do to force migrants to be a tool in a game to try to detribalise their neighbours."
During his visit Mr Wallace visited British troops already stationed in Poland as part of Nato's Enhanced Forward Presence. He also signed an agreement in principle to sell Poland air defence missiles.
Migrants have been living in camps in freezing conditions and with little food or water just inside Belarus, stranded between Polish guards on one side and Belarusian guards on the other.
On Wednesday, after more than a week of stalemate, Belarus said it had provided temporary shelter for about 1,000 people.
Poland, with EU backing, is determined not to let the migrants into the bloc and has warned that the situation may go on for months.
It has also been accused of pushing people back across the border into Belarus, contrary to international rules of asylum.
The European Commission has accused Belarus of promising easy entry to the EU.
But Belarus has said it is trying to repatriate stranded migrants along the border, but that many have refused to return to their home countries.