London Daily

Focus on the big picture.

Algorithms can drive inequality. Just look at Britain's school exam chaos

Algorithms can drive inequality. Just look at Britain's school exam chaos

Philip blames an algorithm for potentially losing his place to study law at university.

The 18-year-old, whose full name CNN is not disclosing because he feared repercussions from universities, was among more than 300,000 pupils in England, Wales and Northern Ireland who woke on August 13 to critically important A-level exam results, which are broadly equivalent to the US high school diploma.

These exams were canceled this summer due to the pandemic. Student marks were instead determined by an algorithm, the Direct Centre Performance Model, which was chosen by the government's exam regulator. The model drew on a collection of data to produce the grades. A subsequent outcry over alleged algorithmic bias against pupils from more disadvantaged backgrounds has now left teenagers and experts alike calling for greater scrutiny of such technology.

The teachers at Philip's west London school predicted he would gain 2 A grades and a B in his exams, which would have comfortably secured his spot to study law at Exeter University.


Students demonstrate outside the Department for Education in central London on August 14.


On August 13, the student sat at home trying to access the website that would confirm whether or not he had a university place.

"I was upstairs trying to get [the website] to load and my Mum was downstairs doing the same thing," he told CNN. "She got it open and shouted out. And they'd declined me.

"I didn't feel too good," Philip added. "Yeah, I was pretty cross about it. But everyone I was with was in a similar situation."

The model awarded Philip a B grade and 2 Cs. The teenager was not alone; close to 40% of grades in England were downgraded from teacher-predicted marks, with pupils at state-funded schools hit harder by the system than their private school peers. Many subsequently lost their place at university.

Uproar followed, with some teenagers protesting outside the UK department of education. Videos from the student protests were widely shared online, including those in which teenagers chanted: "F**k the algorithm!"

Following several days of negative headlines, Education Secretary Gavin Williamson announced that students would be awarded teacher-predicted grades, instead of marks allocated by the model.

The chosen algorithm was meant to guarantee fairness, by ensuring grade distribution for the 2020 cohort followed the pattern of previous years, with a similar number of high and low marks. It drew on teacher-predicted grades and teacher rankings of students to determine grades. But crucially it also took into account the historical performance of schools, which benefited students from more affluent backgrounds.

Private schools in England, which charge parents fees, typically have smaller classes, with grades that could not easily be standardized by the model. The algorithm thus gave more weight to the teacher-predicted grades for these cohorts, which are often wealthier and whiter than their downgraded peers at state schools.

"One of the complexities that we have is that there are lots of ways an algorithm can be fair," said Helena Webb, senior researcher at Oxford University's Department of Computer Science.

"You can see an argument where [the government] said [it] wanted to get results that look similar to last year's. And at a country-wide level, that could be argued as [being] fair. But it completely misses what was fair for individuals.

"Obviously this algorithm is reflecting and mirroring what has happened in previous years," she added. "So it doesn't [reflect] the fact that schools might [improve.] And of course that's going to have worse effects on state schools than on very well known private schools which have consistently higher grades."

"What's made me angry is the way [they] treated state schools," said Josh Wicks, 18, a pupil from Chippenham in Wiltshire, western England. His marks were downgraded from 2 A* and an A to 3 As.

"The algorithm thought that if the school hadn't achieved [high grades] before, [pupils] couldn't get them now," he told CNN. "I just think it's patronizing."

The political storm has left ministers in Boris Johnson's government scrambling for explanations, following heavy criticism of its handling of the coronavirus pandemic. Covid-19 has killed more than 41,000 people in the UK, making it the worst-hit country in Europe.

Why are some algorithms accused of bias?


Algorithms are used across every part of society today, from social media and visa application systems, to facial recognition technology and exam grading.
The technology can be liberating for cash-strapped governments and for corporations chasing innovation. But experts have long warned of the existence of algorithmic bias and as automated processes become more widespread, so do accusations of discrimination.

"The A-levels thing is the tip of the iceberg," said Cori Crider, co-founder of Foxglove, an organization that challenges the alleged abuse of digital technology. Crider told CNN that the algorithms replicated the biases found in the raw data used.


Students hold placards as they protest outside the Department for Education in central London on August 14.


But Crider warned against the impulse to simply blame policy issues on the technology.

"Anybody who tells you it's a tech problem is [lying]," she said.

"What happened [with the exams] is that a political choice was made to minimize grade inflation. That's a political choice, not a tech one."

Foxglove and the Joint Council for the Welfare of Immigrants recently challenged the British Home Office over its use of an algorithm designed to stream visa applications. The activist groups alleged that the algorithm was biased against applicants from certain countries, making it automatically more likely that such applicants would be denied a visa.

Foxglove alleged that the screening system suffered from a feedback loop,"where past bias and discrimination, fed into a computer program, reinforce future bias and discrimination."

"We have been reviewing how the visa application streaming tool operates and will be redesigning our processes to make them even more streamlined and secure," a UK Home Office spokesperson told CNN.

"But we do not accept the allegations Joint Council for the Welfare of Immigrants made in their Judicial Review claim and whilst litigation is still ongoing it would not be appropriate for the department to comment any further."

Crider said the problems Foxglove found with past data leading to biased algorithms were evident elsewhere, pointing to the debate over predictive policing programs in the United States.

In June, the Californian city of Santa Cruz banned predictive policing over concerns that the analytic software program officers used in their work was discriminating against people of color.

"We have technology that could target people of color in our community -- it's technology that we don't need," Mayor Justin Cummings told Reuters news agency in June.

"Part of the problem is the data being fed in," Crider said.

"Historical data is being fed in [to algorithms] and they are replicating the [existing] bias."

Webb agrees. "A lot of [the issue] is about the data that the algorithm learns from," she said. "For example, a lot of facial recognition technology has come out ... the problem is, a lot of [those] systems were trained on a lot of white, male faces.

"So when the software comes to be used it's very good at recognizing white men, but not so good at recognizing women and people of color. And that comes from the data and the way the data was put into the algorithm."

Webb added that she believed the problems could partly be mitigated through "a greater attention to inclusivity in datasets" and a push to add a greater "multiplicity of voices" around the development of algorithms.

Increased regulation?


Activists and experts told CNN they hoped recent debates around algorithms would lead to greater oversight of the technology.

"There's a lack of regulatory oversight over how these systems are used," Webb said, adding that companies could also choose to self-regulate.

Some companies are becoming notably more vocal on the issue.

"Some technologies risk repeating the patterns developed by our biased societies," Instagram CEO Adam Mosseri wrote in a statement in June on the company's diversity efforts. "While we do a lot of work to help prevent subconscious bias in our products, we need to take a harder look at the underlying systems we've built, and where we need to do more to keep bias out of these decisions."

Facebook, which owns Instagram, subsequently created new teams to review bias in company systems.

"I would like to see democratic pushback on [the use of algorithms]," Crider said. "Are there areas in public life where it's not acceptable to have these systems at all?"

While the debate continues in boardrooms and academia, these automated systems continue to determine people's lives in numerous and subtle ways.

For Philip, the UK government's scrapping of the exams algorithm has left him in limbo.

"We emailed Exeter [University] and phoned and they're in a kind of mess," he said, adding that he was hopeful he could win his place back. "I think I'll just defer now anyway."

He said he was grateful to be given his predicted grades but said the experience had gone "pretty badly."
"[The government] had months to sort this out," he said. "I get that there's a lot of things going on with the health stuff but [...] it's a pretty poor showing."

Newsletter

Related Articles

London Daily
0:00
0:00
Close
Unelected PM of the UK holds an emergency meeting because a candidate got voted in… which he says is a threat to democracy…
You Are So Beautiful
Rob Schneider explains California reparations legislation.
Postmodern Jukebox European Tour Version
Who knew badminton could get so intense?
An old French tune (by Georges Brassens) Pomplamoose ft. John Schroeder
Farmers break through police barriers in Brussels.
Sattahip Motor Show 20
London's Iconic British Telecom Tower Sold To Become Hotel
SONATE AU CLAIR DE LUNE - Moonlight sonata
Ukraine Arrests Father-Son Duo In Lockbit Cybercrime Bust
A kiss to build a dream on
US Offers $15 Million For Info On Leaders Of Cybercrime Group Lockbit
Wonderful Tonight - Eric Clapton (Boyce Avenue acoustic cover)
Russia Claims UK Cultural Agency Spied for Ukraine
Mean Blues
Apple warns against drying iPhones with rice
La Chansonnette
Alexei Navalny: UK sanctions Russian prison chiefs after activist's death
Pattaya Addicts
German economy is in 'troubled waters' - ministry
Franz Liszt - Liebestraum - Love Dream
In a recent High Court hearing, the U.S. argued that Julian Assange endangered lives by releasing classified information.
Dream a little dream of me
New video
Unchained Melody sung like you've NEVER heard!
Tucker Carlson says Boris Johnson wants "a million dollars, in Bitcoin or cash, from Tucker Carlson to talk about Ukraine.
Dave Brubeck - Take Five
Russia is rebuilding capacity to destabilize European countries, new UK report warns
Édith Piaf - Non, Je Ne Regrette Rien (Sofie)
EU Commission wants anti-drone defenses at Brussels HQ
Rondo Alla Turca
Von der Leyen’s 2nd-term pitch: More military might, less climate talk
Kiss of fire
Global Law Enforcement Dismantles Lockbit Ransomware Operation
Tom Jones - I´ll Never Fall In Love Again 1967, 1989, 2001
Prince William Urges End to Gaza Conflict
Israel Cachao López - Guajira Clásica
UK court to hear Assange's final appeal against extradition to the US, where he faces charges related to his journalistic work—the publication of a classified video in 2010 that exposed US war crimes against humanity.
Edward Maya - Stereo Love (feat. Vika Jigulina) (Extended Mix)
About 50-60% kids either chose to be YouTuber or influencer
Strauss - Radetzky March - Karajan
A viral video of Nationals MP Barnaby Joyce lying on a Canberra footpath is celebrated by his media mates.
La vie en rose
European Countries React to Navalny's Death by Summoning Russian Diplomats
The Temptations - My Girl (Smokey Robinson Tribute) 2006 Kennedy Cent
Israel has gone ‘beyond self-defence’ in Gaza, says Labour’s Streeting
Orlando Cachaito Lopez Redencion
English farmers to be offered ‘largest ever’ grant scheme amid food security concerns
Edith Piaf - NON, JE NE REGRETTE RIEN
Cameron government knew Post Office ditched Horizon IT investigation
RADETZKY MARCH-2008-Wien, New Year Concert
EU Calls for Immediate Ceasefire in Gaza Conflict
Only you (And you alone)
EU Vows To Hold Putin "Accountable" After Meeting Alexei Navalny's Wife
Strangers In The Night
EU Launches Probe Into TikTok Over Child Protection Under Digital Content Law
Charles Aznavour - La Boheme
The EU Initiates Naval Mission to Defend Red Sea Trade Routes
Summer time
EU and UK Announce Joint Effort on Migration
Sting and Stevie Wonder - Fragile (from Sting's 60th birthday concert)
Brazil's Lula Likens Gaza Operation to Holocaust, Israel Says "Red Line" Crossed
Aux Champs Elysees
Ministers Confirm Proposal to Prohibit Mobile Phone Usage in English Schools
Stand By Me - Ben E. King (Boyce Avenue acoustic cover)
Microsoft-backed OpenAI valued at $80bn after company completes deal
La Mer (Beyond the Sea) – Avalon Jazz Band
‘Alexei would want to tell Russia not to give up fighting’
She
Rwandan Footballer's Dismissal Sparks Concerns Over UK Asylum Plan
Nathalie Song by Enzo Petrachi Stjepan Hauser Cello
Whisky Challenges China's Baijiu Market During New Year Celebrations
Shape of My Heart - Sting (Boyce Avenue acoustic cover)
Avdiivka - Symbol Of Ukrainian Resistance Now In Control Of Russian Troops
Radiohead - Creep
Putin Critic Alexei Navalny's "Killers" Refusing To Hand Over Body, Say Allies
Quizás,Quizás,Quizás - Andrea Bocelli - Jennifer Lopez
"Historic Step": Zelensky Signs Security Pact With Germany
Perhaps, Perhaps, Perhaps - Multi-Couples
"Historic Step": Zelensky Signs Security Pact With Germany
Pentatonix Havana
20 Tech Giants Sign Effort To Fight AI Election Interference Across Globe
Paula Cole - Autumn Leaves
Joe Biden Accuses Putin of Causing Navalny's Death
Oscar Benton Bensonhurst Blues
Russian opposition leader Alexey Navalny has died at the Arctic prison colony
OH NANANA vs ABUSADAMENTE
Tucker Carlson grocery shopping in Russia. This is so interesting.
Nina Simone - ”I Put A Spell On You”. Vezi aici cum cântă Jeremy Ragsd
Julian Assange's Wife Warns of His Death if Extradited to US
NIGHTWISH - The Phantom Of The Opera
‘A lot higher than we expected’: Russian arms production worries Europe’s war planners
Motorshow 2016 Tanjay Negros Oriental
Greece Legalizes Same-Sex Marriage and Adoption Rights
Monica Bellucci - Ti Amo
Hungarian Foreign Minister: Europeans will lose Europe, the Union's policy must change drastically
Michael Jackson - Billie Jean Milena The Voice France 2018
In Britain Homeowners are receiving CPO’s (Compulsory Purchase Orders) so their homes can be redistributed to migrants
Michael Buble (Help Me Make It Through The Night) feat Loren Allred
Memories Canon In D - Maroon 5 (Boyce Avenue piano acoustic cover)
Matteo Simoni - Marina
Maroon 5 - One More Night
Maroon 5 - Memories
Mark Knopfler - Brothers In Arms (Berlin 2007 Live)
Mark Knopfler & Emmylou Harris - Romeo And Juliet (Real Live Roadrunni
Marina, Marina - The LUCKY DUCKIES intimist live concert at Guimarães
Major Lazer & DJ Snake – Lean On Mauranne The Voice France 2016
Love Theme from Romeo and Juliet - Joslin - Henri Mancini, Nino Rota
LoLa & Hauser - Love Story
Linkin Park Jay-Z - Numb Encore (Live 8 2005)
Hallelujah Mennel Ibtissem, The Voice France Leonard Cohen
Leonard Cohen - Dance Me to the End of Love
Leonard Cohen & Natasha Rostova - Dance me to the end of love
La casa de papel - Bella Ciao
La Camisa Negra
L'italiano (Toto Cutugno) - The Gypsy Queens
Juanes - La Camisa Negra
Jonathan and Charlotte - Britain's Got Talent 2012 Live Semi Final - U
John Powell - Assassin's Tango
Joe Cocker - You Can Leave Your Hat On (LIVE in Dortmund)
Joe Cocker - Unchain My Heart 2002 Live
Joe Cocker - A Whiter Shade Of Pale
Jay Z & Alicia Keys - Empire State of Mind LIVE
Jason Mraz - Im Yours (live)
Jarrod Radnich - Bohemian Rhapsody - Virtuosic Piano Solo
James Blunt - You're Beautiful
James Blunt - You're Beautiful & Bonfire Heart (Live at The Nobel Peac)
If You Go Away - Helen Merrill & Stan Getz (Tribute to Virna Lisi)
I'LL BE MISSING YOU
I Say a Little Prayer
Hotel California ( Eagles ) 1994 Live
Historia de un amor - Luz Casal. Vezi interpretarea Biancăi Sumanariu
Here Comes The Sun - The Beatles (Boyce Avenue acoustic cover) on Spot
Heart - Stairway to Heaven Led Zeppelin - Kennedy Center Honors
HAVANA by Camila Cabello Zumba Pre Cooldown TML Crew Kramer Pastra
HAUSER and Señorita - I Will Always Love You
HAUSER - Waka Waka
HAUSER - Sway
HAUSER - Lambada
HAUSER - Historia de un Amor
HAUSER - Despacito
Great Pretender
Georgia May Foote & Giovanni Pernice Samba to 'Volare' - Strictly Come
Gary Moore - Still Got The Blues
GIPSY KINGS VOLARE Penelope Cruz
Fugees - Killing Me Softly With His Song
French Latino - Historia de un Amor
For A Few Dollars More The Danish National Symphony Orchestra (Live)
Flashdance • What a Feeling • Irene Cara
Filip Rudan - “Someone You Loved” Audicija 4 The Voice Hrvatska Sez
Eric Clapton - Wonderful Tonight
Enya - Only Time
Enrique Iglesias - Bailando (English Version) ft. Sean Paul
Enrique Iglesias - Bailamos
Elena Yerevan Historia de un amor
Ed Sheeran - Shape of You (Official Music Video)
Ed Sheeran - Perfect Symphony [with Andrea Bocelli]
Ed Sheeran - Perfect (Official Music Video)
Easy On Me - Adele (Boyce Avenue 90’s style piano acoustic cover) on S
ERA - Ameno
ELENA YEREVAN- Cancion Del Mariachi-IN STUDIO-2017 DPR
Dust In The Wind - Kansas (Boyce Avenue acoustic cover)
Don't Let Me Be Misunderstood
Despacito x Shape Of You - Pentatonix
Deep Purple - Child In Time - Live (1970)
David Foster When A Man Loves A WomanIt's A Mans World (SealMichael Bo
Dance me to the end of Love ( Pi-Air Design )
Coolio - Gangsta's Paradise (feat. L.V.) [Official Music Video]
Conquest Of Paradise (Vangelis), played on Böhm Emporio organ
Cielito Lindo
Chico & The Gypsies - Bamboleo
Canción Del Mariachi - Antonio Banderas, Los Lobos • Desperado
Camila Cabello - Havana (Audio) ft. Young Thug
Camila Cabello - Havana ( cover by J.Fla )
California Dreamin' - The Mamas & The Papas José Feliciano (Boyce Ave
Buster Benton - Money Is The Name of The Game
Hallelujah Pentatonix
Bobby McFerrin - Don't Worry Be Happy (Official Music Video)
Bob Dylan - Knockin' On Heaven's Door Emilia The Voice Kids France
Besame Mucho - Cesaria Evora
Ben E. King - Stand by Me Sax Cover Alexandra Ilieva Thomann
Bella Ciao
Bella Ciao - INSTRUMENTAL
Beautiful in White x Canon in D (Piano Cover by Riyandi Kusuma)
Bad Romance - Vintage 1920's Gatsby Style Lady Gaga Cover ft. Ariana Savalas & Sarah Reich(1)
BELLA CIAO 2020 - KARAOKE ITALIANO
BAMBOLEO - Gipsy Kings • Antonio Banderas, Katya Virshilas
BAILANDO (original)
Awesome Ukrainian yodeler - SOFIA SHKIDCHENKO (with English subtitles)
Avicii - The Nights
Atom - The Great Gig in the Sky
Aretha Franklin - (You Make Me Feel Like) A Natural Woman (Official Ly
Antonio Banderas - Cancion del Mariachi (Desperado)
André Rieu - Zorba's Dance (Sirtaki)
André Rieu - Can't Help Falling In Love
André Rieu & Mirusia - Ave Maria
Andrew Reyes Elton John - Don't Let The Sun Go Down The Voice 2020 (
Andreas Kümmert Whiter Shade Of Pale The Voice of Germany 2013 Showd
And I Love You So
All About That Bass - Postmodern Jukebox European Tour Version
Alan Walker - Faded (Piano Cover)
Ain't No Sunshine -- Bill Withers (cover by Canen 12 y.o.)
African music
Adriana Vidović - “Creep” Audicija 4 The Voice Hrvatska Sezona 3
Adriana Vidović - “Believer” Nokaut 3 The Voice Hrvatska Sezona 3
A Fistful of Dollars - The Danish National Symphony Orchestra and Tuva
4 Beautiful Soundtracks Relaxing Piano [10min]
2CELLOS - Whole Lotta Love vs. Beethoven 5th Symphony [OFFICIAL VIDEO]
2CELLOS - Smooth Criminal (Live at Suntory Hall, Tokyo)
2CELLOS - Smells Like Teen Spirit [Live at Sydney Opera House]
2CELLOS - Despacito [OFFICIAL VIDEO]
13 Year Old Girl Playing Il Silenzio (The Silence) - André Rieu
094.All About That Bass
00 - SADNESS PART 1
(Ghost) Riders In the Sky (American Outlaws Live at Nassau Coliseum, 1
(Everything I Do) I Do It For You - Bryan Adams (Boyce Avenue ft. Conn
What a wonderful world
Moon river
×