With his party coming off heavy losses in the local elections, he may have been hoping for some good news as he took his seat at St Mary's Stadium.
However, sitting in the stands in jeans and a grey hoodie, he saw Fulham score twice in the second half to end his club's 11-year Premier League stay.
Rivals Fulham fans inevitably sang the chant "Rishi, Rishi, what's the score".
Southampton's loss on Saturday puts them eight points adrift of safety with two matches to play, and not even a parliamentary intervention would be able to change their fate now.
Born and raised in Southampton, Sunak has long spoken of his support of the team.
A biography of Mr Sunak has previously said as a youngster his father Yashvir was a season ticket holder, and one of the prime minister's "most prized possessions" was a card he received for his 18th birthday signed by the entire squad.
However, during his bid to become prime minister in August last year he came under the intense glare of the football community after saying Southampton's could improve their fortunes by beating Manchester United that coming weekend.
The team were playing Leicester City, but did have United the following week.
Supporting a football team is often seen as a way for political leaders to broaden their appeal, but can prove a tricky business.
David Cameron claimed to be a Aston Villa fan, but a blunder in a speech in 2015 saw him claim the top flight's other claret and blue team West Ham - something he later put down to "brain fade".
A Newcastle United fan, Sir Tony Blair received ridicule for a number of years due an apparent claim to have seen club legend Jackie Milburn play at St James Park in the 1950s - even though the player had retired when he was five.
In 2008, the regional newspaper which first reported it clarified it had come from something misheard on the radio.
One former prime minister was never likely to make this kind of slip-up. Gordon Brown was a staunch fan of Scottish lower league side Raith Rovers and could happily recite to interviewers the full line-up of the first match he saw as a seven-year-old.
And possibly to add to Sunak's pain, Southampton's rivals Fulham have their own connection with Downing Street.
After the match, Fulham's Twitter account showed a picture of Sunak grimacing in the crowd, along with their own caption "Sorry, the only Prime Minister we recognise is Hugh Grant" - a nod to the dancing PM portrayed in 2003 romantic comedy Love Actually.