On a filthy night three days before Christmas in 1978, I was sitting on a rather ancient coach travelling across the Pennines towards Lancashire, along with about 50 other football supporters. The Bradford branch of the Manchester United supporters club catered largely for a collection of brickies and other manual workers – and that evening we were all on our way to watch a dismal 0-3 defeat at Bolton. As torrential rain poured down on the M62, the bloke sitting immediately in front suddenly turned and, with a hint of menace, said to my brother and me: “You’re not really the same as us are you?”

It may have been the drink talking after some seasonal revelry during the day, but his analysis was on the money. The sons of an academic and a teacher, Paul and I read different papers, watched different stuff on TV and spoke in a different way. But as aspiring young lefties in the late 1970s, we imagined, or hoped, that this divergence in terms of social class would be redeemed and erased by politics: after all, it was only 10 years after 1968, when radical students and workers attempted to dream a revolutionary alliance into being. So it was mortifying to my teenage self to realise that, even in the context of supporting the same football team, there might be an underlying suspicion towards the middle-class interlopers on the bus.

This uncomfortable moment was a minor lesson in the tricky social dynamics of class and status. Almost half a century later, the future of progressive politics in Britain may depend on a similar kind of learning process writ large. The usual caveats (low turnout, protest voting, local factors) apply to any analysis of last week’s council elections. But in England, the broad picture appears to confirm a changing political landscape that, while it potentially poses deep problems for the Conservative party, also confronts Labour with challenging truths. To quote the Oxford University election analysts Michael Thrasher and Colin Rallings: “The urban south is becoming more Labour as the north hangs on to its post-Brexit attachment to the Tories … but there is evidence too of a new demographic cleavage. Areas where more than a third of the population are university graduates swung sharply to Labour, those where graduates are thinner on the ground moved almost as much the other way.”

Two demographies, two economies and, increasingly, two sensibilities. On one side, liberal-minded, Labour-voting urban professionals and young graduates clustered disproportionately in the cities; on the other, elements of the post-industrial working class (some of it retired) who mourn the loss of something that has disappeared in towns that are steadily getting older.

If it cannot do much better among this second group, Labour will not win a majority in the next election. Even the success of a progressive alliance with the Lib Dems and the Greens depends on Labour doing its job in the “red wall”. But despite notable successes, such as its victories in Cumberland and Kirklees, the hoped-for revival in the north and Midlands stuttered and stalled last week to an extent that allowed Boris Johnson to brazen out an otherwise terrible night.

Viewed through a purely economic lens, some of the results might appear inexplicable. Polls indicate that a majority of the public views the government’s response to the cost of living crisis as woefully inadequate. But in one of the most deprived wards in Walsall – where one in five households are fuel poor – there was a 35% swing to the Conservatives. While red wall type areas will suffer disproportionately in the hard times to come, it would therefore seem unwise for Labour to rely on attacking the government to solve the problem of its soured relations with the traditional working class. Instead, perhaps the left should widen the horizon of its analysis to address the kind of question that my fellow United fan put to me on the coach to Bolton. Why do substantial numbers of former Labour voters sense a cultural gulf between themselves and what they think the party now represents? Why do they feel Labour is “not the same” as them any more?

Last year, the UK in a Changing Europe thinktank published an important paper co-written by the sociologist and social mobility expert John Goldthorpe. Entitled Meritocracy and Populism, a section of it summarises two main findings from red wall focus groups convened by Deborah Mattinson (now Labour’s director of strategy). The first was that these (predominantly leave) voters felt that good jobs and opportunities for younger people were no longer available in their communities. A sense of grievance at this was compounded by the perception that, as old industries had faded away, the world now belonged to new generations of degree-holders who, bluntly, looked down on them. Politically, write Goldthorpe and his co-author, Erzsébet Bukodi, such views “translated into a deep disillusionment with the Labour party. This was seen as now dominated by graduate, metropolitan elites – whether Blairite or Corbynite – obsessed with political correctness and more concerned with telling the people they were supposed to represent that they were ‘wrong’ than with trying to understand the conditions under which they were living.” Depending on how things play out, Keir Starmer’s current woes over ”Beergate” – feeding a narrative of elite hypocrisy – could prove particularly damaging in this regard.

This alienated perspective, which is almost certainly shared by large numbers of lost Labour voters, may be an unfair caricature. But if Labour is to bridge generational and educational divides in an era of culture wars, it should admit that there is a kernel of truth here. The mass expansion of higher education has helped Britain become a far better place when it comes to addressing, for example, race and gender inequality. But the widespread characterisation of Brexit as a purely xenophobic, reactionary project demonstrated that highly educated liberals are also capable of myopic intolerance. To reconstitute a relationship with leave-voting constituencies, Labour needs to do more than “move on” from 2016 and its aftermath, as Starmer has understandably but mistakenly sought to do. It needs to re-engage with why so much of its working-class support voted the way it did.

A starting point for that exercise might be the seminal essay Culture is Ordinary, written by Raymond Williams in 1958. In it, Williams describes the postwar blue collar environment in which he grew up as defined by commitment to “neighbourhood, mutual obligation and common betterment”. Mattinson’s leave voters were evidently preoccupied by the perceived loss of this sense of solid community, and clearly ill at ease in an age of more freewheeling individualism. These are not in themselves reactionary sentiments; in fact they belong to a venerable Labour tradition that includes RH Tawney and William Morris. But in the context of Brexit, they were far too easily dismissed and misrepresented, and the scars from that are still there. If they are to be healed in the places where Labour so badly needs to reconnect, the modern left needs to travel outside its cultural comfort zone with an open mind, listen properly to the messages it receives, and admit that it can learn from the red wall as well as lecture it.