Cardiff University found that having to prove you are struggling to get a voucher for food creates more stigma.
Dr Andrew Williams said those in close-knit communities were more worried about what people thought, and likely to wait until "absolutely desperate".
It comes as communities say they are turning inwards to support each other.
In the year to March 2022, the Trussell Trust gave out 131,232 emergency food parcels in Wales, part of 2.1 million across the UK.
"I've spoken to people who were afraid to ask for a voucher from social services," said Dr Williams, who argued that community food hubs would have less stigma.
Dr Williams has been studying food banks since 2014, and has interviewed more than 100 volunteers and users.
He said they were "designed as a short-term gap before the benefits system would kick in" but said the system has "an unwritten, unspoken sense of shame".
"I've spoken to people who have walked 12 miles to go to a different town so they would not get a food parcel from their children's teacher who works at the food bank," he said.
He said that in close communities "everyone knows each others' business and there is a reluctance to out yourself as struggling".
Research published in the Journal of Poverty and Social Justice also found some people who had used food banks used words like "awkward", "embarrassed", "worthless", and "uncomfortable".
Tracy Murphy has lived in Aberfan for most of her life and when her husband could no longer work because of his ill health, her family had to turn to a food bank for help.
"My husband went to the church [to get a voucher] and he did feel embarrassed," she said.
"It's the stigma attached to it that we got that low that we had to rely on free food.
"It's just proudness, I suppose, wanting to stand on your own two feet.
"We'd never had to rely on anything like that in the past, it was a new experience and just felt a bit cap in hand, I suppose".
She is now a full-time carer for her husband, and has become involved with a local group, H Factor, which is setting up a food pantry where anyone living locally can buy food at low prices.
It will take over the delivery of half-price fresh fruit and vegetables from Merthyr Tydfil Housing Association and start growing food in the community garden, Afon Gwreiddau.
H Factory organiser Lynne Colston, who works alongside a staff of 20, has already taken over the running of the local hotel for the community and the new food pantry will be based in its former garage, for discreet visits.
"We are trying to get rid of the stigma of going to a food bank and to make everything that we grow available to people on a reduced cost," she said.
She added that people needing support were not necessarily who you would think.
"I don't think we know who the poorest are anymore. We have assumed the poor are people on benefits and we know that the poor are now people who are working," she said.
At home, Ms Murphy has tried her best to support herself and others where she can.
She had a small garden to grow vegetables and has 40 hens providing free eggs for the community.
She said: "I love it, I love it, we should all do a bit really, it's good for the environment and good for the bees."
She is now setting up egg production for the community with 40 hens.
At the community garden in Aberfan, Afon Gwreiddau, they have already started selling the fresh produce they grow to local people, with the plan to expand into two more polytunnels.
Team leader Lynne Farnham said it "will be a mass abundance of food for the community in the form of food boxes".
"They shouldn't feel shame, but there is that stigma going into a food bank.
"So coming here, where everyone is free to come whatever background you come from, whatever your income, is based on is a really calm way of collecting your food," she said.