Interest rates were hiked to 4.5% from 4.25% - the highest in almost 15 years - in the battle to slow inflation.
"It's taking longer for food price [falls] to come through," Bank boss Andrew Bailey told the BBC.
But Mr Bailey was more optimistic on how quickly the UK economy would grow, saying it would now avoid recession.
The Bank has been rapidly raising rates to try to slow the sharp rise in the cost of living.
UK inflation remains close to its highest level for 40 years, and is not dropping as quickly as predicted as prices in UK supermarkets remain high.
Some have questioned why a drop in the cost of wholesale food prices globally has not led to falls in the prices charged by UK supermarkets.
However, Mr Bailey said he did not think supermarkets and other grocers were charging customers more than they should.
"It actually doesn't look like that's going on," he told the BBC, adding that higher energy prices and the war in Ukraine had made it harder to import some foods and led to higher costs for retailers.
"Energy is quite a big element in the cost of food production and that's certainly had an effect in this crisis. Often producers have bought forward (supplies) at high prices because they were concerned about whether they were going to get the things they needed.
"But, as we said before, we are in very unusual times."
He said grocers had told him that they expected food inflation to "come down quite rapidly" throughout the rest of the year.
UK supermarkets have said there is typically a three to nine-month lag to see price falls reflected in shops.
On Thursday, the Treasury met with representatives from all major UK supermarkets to discuss food prices. The British Retail Consortium (BRC), which represents grocers, said the discussion was "very constructive".
"Retailers were able to explain the main drivers of food inflation, including high labour costs, energy prices, and manufacturing costs," it said.
It added that it had asked the government what it could do to mitigate the high cost of policy initiatives around packaging, recycling and the new Windsor Framework and UK border checks.
The Bank now expects overall inflation - the rate at which prices rise - to drop to 5% by the end of this year, above the 4% previously predicted.
The increase in interest rates will mean higher mortgage, credit card and loan payments for some people, but the rise in rates could benefit savers.
By raising rates, the Bank expects people to have less money to spend and buy fewer things, which should help stop prices rising as quickly.
However, it also makes it harder for firms to borrow money and expand.
Around 85% of all mortgages are fixed-rate, according to the Bank, and about 1.3 million households are expected to reach the end of their deals this year and face a hike of up to £200 per month, based on current rates.
Compared with pre-December 2021 before interest rates began to rise, a typical tracker mortgage customer will be paying about £417 more a month, and variable rate mortgage holders about £266 more.
Mr Bailey said that while all inflation is difficult, rising food prices hit those on lower incomes harder because they spend a higher proportion of their money on food. "We are very, very conscious that all inflation is difficult and particularly for those least well off."
The Bank's chief economist Huw Pill recently sparked a backlash when he said people in the UK needed to accept that they would be worse off.
Mr Bailey said: "I don't think Huw's choice of words was the right one... to be honest and I think he would agree with me."
Separately, the Bank of England was more positive on the outlook for the economy over the next few months - a stark contrast from its forecast six months ago when it said the UK would enter the longest recession on record.
"Modest but positive growth," is now expected, according to the Bank.
When the effects of strikes and the extra Bank Holiday for King Charles' Coronation are stripped out, the economy will have grown by 0.2% both in the first three months of the year and between April and June, the Bank predicts.
Falling energy prices as well as measures to help businesses and households announced in the Budget last month have led the Bank to change its forecasts.
The government has pledged to halve the rate of inflation by the end of the year.
Chancellor Jeremy Hunt said while it was "good news" that a recession was not longer forecast, the interest rate rise was "obviously be very disappointing for families with mortgages".
"But unless we tackle rising prices, the cost of living crisis will only carry on," he added.
Labour's shadow chancellor Rachel Reeves said the rise would leave people "wracked with anxiety".
Helen Parry, who works for DC Fruit and Veg in Stoke-on-Trent, said she had seen changes in her customers' choices and how they were avoiding treating themselves later in the month.
"They come, they do their shopping, they get what they need, and as the month goes on towards when their next payday is, it slows down a little bit," she said.