London Daily

Focus on the big picture.

Why we place too much trust in machines

Why we place too much trust in machines

While many people might claim to be sceptical of autonomous technology, we may have a deep ingrained trust of machines that traces back to our evolutionary past.

As Air France Flight 447 hurtled belly-first towards the Atlantic Ocean at nearly 300km per hour (186mph), pilot Pierre-Cédric Bonin wrestled with the controls. He and his crew had taken over after the autopilot suddenly switched itself off, apparently due to a build-up of ice on the aircraft. It was a situation that demanded manual intervention.

The pilots, unfamiliar with this scenario, struggled to steady the plane. Confusing messages and alarms from the aircraft's computer bombarded them and suggested the craft was not stalling when in fact it was. Bonin's last known words, captured on the flight recorder, were, "We're going to crash – this can't be true. But what's happening?"

All 228 passengers and crew on board perished that day, 1 June 2009. When accidents like this occur, involving humans and machines, there are usually multiple factors or causes at work. But analysts have blamed the tragedy of Flight 447 partly on an excessive reliance or trust in machines. They pointed to the flight crew's expectation that the autopilot would stay switched on, and that the plane's information systems would provide accurate information. It is far from the only incident in which an over-reliance on technology has contributed to fatalities.

It is a well-studied phenomenon known as automation bias, which sometimes also leads to automation complacency, where people are less able to spot malfunctions when a computer is running the show. But what is perhaps surprising is that our tendency to "overtrust" machinery is perhaps directly influenced by millions of years of evolution.

"Technology overtrust is an error of staggering proportion," writes Patricia Hardré of the University of Oklahoma in a book chapter on why we sometimes put too much faith in machines. She argues that people generally lack the ability to judge how reliable a specific technology is. This can actually go both ways. We might dismiss the help of a computer in situations where it would benefit us – or blindly trust such a device, only for it to end up harming us or our livelihoods.

The behaviour of one of our closest relatives, the chimpanzee, may contain a clue about why we are so bad at assessing the trustworthiness of machines. It could be because we are primed to evaluate other members of our species instead.

A zoo in Japan tried to teach its chimpanzees to use a vending machine by watching children – such interactions with technology are entirely built on trust

In a recent experiment, researchers set up an apparatus in which chimpanzees at a sanctuary in Kenya could pull on a rope to retrieve a food reward. One rope offered a basic food reward – one piece of banana. But they were also presented with a second option – a bigger reward of two pieces of banana and a slice of apple that could be dished out either by a machine or a fellow chimpanzee.


Sometimes it was the machine on the other end of the rope, other times it was another chimp, but never both. However, sometimes the machine would not deliver the reward, and sometimes the other chimp would choose not to share. So while there was potentially a bigger reward, it was a less certain choice to pull the second rope.

The chimpanzee participant was thus faced with either a social or a non-social condition. They'd need to trust either the machine or the other chimp in order to have the chance of a bigger food reward.

The study found that, when a fellow chimpanzee presided over the uncertain option, the primates were less likely to go for it. They refused to engage with the social trials 12% of the time, but they only displayed this aversion 4% of the time in non-social trials where a machine presided over the reward. In other words, they had greater trust in the machine.

"They were much more hesitant… when the partner was another chimpanzee," says Lou Haux at the Max Planck Institute for Human Development, who designed and ran the experiment with her colleagues. It's one of just a few studies revealing that social risk plays a big part in how chimpanzees and humans navigate the world.

Imagine how you'd feel if a bartender took your cash and then proceeded to drink your cola right in front of you. You'd probably be livid


It's called "betrayal aversion" says Haux: "The fear of being duped by another human [or chimpanzee], which is thought to cause stronger emotions." She likens this to putting money into a vending machine only for it to fail to dispense the soft drink you requested. That could prompt irritation, no doubt, but imagine how you'd feel if a bartender took your cash and then proceeded to drink your cola right in front of you. You'd probably be livid. Of course, the vending machine didn't make a decision to dupe you, it just failed to deliver, while the bartender decided to drink your order despite knowing how that might make you feel.

The research team didn't stop there, however. They went on to perform another experiment involving chimpanzees who had already gained an understanding of how likely they were to get a better food reward when selecting the uncertain option, thanks to having participated in the first experiment. The uncertain option was, in truth, no longer completely uncertain – the chimps now possessed a sense of what kind of risk they were taking.

And that's when a surprise emerged. The chimpanzees stopped discriminating between the social and non-social options – they no longer seemed to trust the machine more than the fellow chimpanzee.

"That's why we think it's an exciting finding, that they distinguish between the social world and the non-social world in cases where there's still a lot of uncertainty around," says Haux.

It makes sense when you think about how important it is for primates to negotiate their social environment, says Darby Proctor, a psychologist at the Florida Institute of Technology.

"With a machine, there's no future implications," she explains. "You don't have that additional potential social cost." After all, chimpanzees in these experiments often have to go and spend time with their fellow participants once the experiment is over – any displeasure caused by taking part could have onward consequences for their relationships.

Most of us are aware that blindly following a satellite navigation system can lead to sticky situations


Proctor and colleagues previously carried out similar tests and also found that chimpanzees were more likely to trust objects in the search for food rewards versus other chimpanzees. Proctor mentions that when one of the primates was seemingly let down by another chimpanzee who failed to give out a hearty food reward, the dejected chimp made their feelings known by spitting water at the partner. "A common display of unhappiness," says Proctor.

Proctor questions whether the chimpanzees in these experiments really trusted the machines more. It could also be described as individuals simply having a more muted reaction to a bad deal when a social partner isn't involved.

“It’s not that we have confidence that the machine will give us a good payout, it might be that we don’t see it as emotionally salient, so we might just be more inclined to gamble or take risks with this inanimate object,” she hypothesises.

But either way, evolution seems to have influenced primates' willingness to engage with uncertainty, based on whether we feel we're taking a social risk or not.

Evolution hasn't really prepared us for the fact that it can be quite costly to be betrayed by a machine, argues Francesca de Petrillo at the Institute for Advanced Study in Toulouse, who studies primates. For millions of years, there was no need to develop an ability to assess machines as carefully as we assess members of the same species. But today, when technology can have a huge impact on people's lives, there arguably is.

Many surveys have suggested that people are often uncomfortable with the idea of self-driving cars or handing over work responsibilities to machines


There are other factors involved here. Evolution aside, our willingness to trust technology is also influenced by personal knowledge about a machine or device, and cultural expectations. A 2019 study found that people were, on average, 29% more likely to give away their credit card details during a text-based chat if they thought they were speaking to a computer versus another human being. The researchers found that this effect was even more pronounced among those who had a pre-existing expectation that machines were more secure or trustworthy than humans.

Then again, sometimes people report a strong aversion to trusting technology. Many surveys have suggested that people are often uncomfortable with the idea of self-driving cars or handing over work responsibilities to machines. There are lots of reasons why suspicions about new technology can take hold. People might fear losing a part of their identity if an automaton takes over. Or they might simply feel sceptical that a computer will approach certain tasks with the required caution and dexterity. When you've seen a hundred videos of robots falling over, or experienced an obstinate computer refusing to function correctly, that's not necessarily surprising.

Among those who have studied what can influence an individual's willingness to trust a particular technological system is Philipp Kulms, a social psychologist at Bielefeld University in Germany. He and a colleague came up with a Tetris-style puzzle game in which participants collaborated with a computer partner that also had control over some of the pieces. When the computer played the game well (demonstrating competence) and allowed the human player access to high-value pieces that netted them extra points (demonstrating warmth), the participants were more likely to report that they trusted the computer player and also more likely to exchange puzzle pieces with it in a collaborative effort, which shows that they also expressed that confidence in-game. It was trust formed by mechanics, if you like.

"To our surprise, this very limited set of variables that we could manipulate was apparently enough," says Kulms.

If we accept that people are generally ill-equipped to assess the trustworthiness of machines because, from an evolutionary standpoint, we are primed to judge trustworthiness based on social cues, then this makes perfect sense. It also chimes with other research that suggests gamblers will gamble more when playing on slot machines that have been designed to display human-like properties.

Most interactions with automated systems are benign, but should we really trust them unreservedly to dispense medical prescriptions?


In other words, we aren't just bad at weighing up the trustworthiness of a machine, we're also easily seduced by mechanical objects when they start behaving a bit like a social partner who has our best interests at heart.

So, in contrast to the person who has been stung by unresponsive computers and therefore distrusts them all, the individual who has learned to put their faith in certain systems – such as aircraft autopilots – may struggle to understand how they could be wrong, even when they are.

De Petrillo notes that she's experienced a feeling of confidence in computers when interacting with voice-activated assistants like Apple's Siri or Amazon's Alexa.

"I assume that they are acting in my best interests, so I don't need to question them," she says. So long as they appear competent and reasonably warm, Kulms' study suggests, that will continue to be the case, for de Petrillo and many others. Kulm points out that this is why it's so important for designers of technology to make sure their systems are ethical and their functionality transparent.

The great irony of all this is that behind a seemingly trustworthy machine may lie a malicious human with nefarious intentions. Undue trust in a faulty machine is dangerous enough, never mind one designed to deceive.

"If we were taking the same type of risk with a human, we would be much more attuned to what the potential negative outcomes could be," says Proctor. She and Haux agree that more work is needed to tease out how much the chimpanzees in their studies genuinely trust machines, and to what extent that reveals truths about human behaviour.

But there's a hint here that our occasional, sometimes disastrous, overtrust in technology has been influenced by a simple fact: we evolved to be social animals in a world where machines did not exist. Now they do and we put our money, our personal data and even our lives under their supervision all the time. That's not necessarily the wrong thing to do – it's just that we're often pretty bad at judging when it's right.

Newsletter

Related Articles

London Daily
0:00
0:00
Close
Unelected PM of the UK holds an emergency meeting because a candidate got voted in… which he says is a threat to democracy…
You Are So Beautiful
Rob Schneider explains California reparations legislation.
Postmodern Jukebox European Tour Version
Who knew badminton could get so intense?
An old French tune (by Georges Brassens) Pomplamoose ft. John Schroeder
Farmers break through police barriers in Brussels.
Sattahip Motor Show 20
London's Iconic British Telecom Tower Sold To Become Hotel
SONATE AU CLAIR DE LUNE - Moonlight sonata
Ukraine Arrests Father-Son Duo In Lockbit Cybercrime Bust
A kiss to build a dream on
US Offers $15 Million For Info On Leaders Of Cybercrime Group Lockbit
Wonderful Tonight - Eric Clapton (Boyce Avenue acoustic cover)
Russia Claims UK Cultural Agency Spied for Ukraine
Mean Blues
Apple warns against drying iPhones with rice
La Chansonnette
Alexei Navalny: UK sanctions Russian prison chiefs after activist's death
Pattaya Addicts
German economy is in 'troubled waters' - ministry
Franz Liszt - Liebestraum - Love Dream
In a recent High Court hearing, the U.S. argued that Julian Assange endangered lives by releasing classified information.
Dream a little dream of me
New video
Unchained Melody sung like you've NEVER heard!
Tucker Carlson says Boris Johnson wants "a million dollars, in Bitcoin or cash, from Tucker Carlson to talk about Ukraine.
Dave Brubeck - Take Five
Russia is rebuilding capacity to destabilize European countries, new UK report warns
Édith Piaf - Non, Je Ne Regrette Rien (Sofie)
EU Commission wants anti-drone defenses at Brussels HQ
Rondo Alla Turca
Von der Leyen’s 2nd-term pitch: More military might, less climate talk
Kiss of fire
Global Law Enforcement Dismantles Lockbit Ransomware Operation
Tom Jones - I´ll Never Fall In Love Again 1967, 1989, 2001
Prince William Urges End to Gaza Conflict
Israel Cachao López - Guajira Clásica
UK court to hear Assange's final appeal against extradition to the US, where he faces charges related to his journalistic work—the publication of a classified video in 2010 that exposed US war crimes against humanity.
Edward Maya - Stereo Love (feat. Vika Jigulina) (Extended Mix)
About 50-60% kids either chose to be YouTuber or influencer
Strauss - Radetzky March - Karajan
A viral video of Nationals MP Barnaby Joyce lying on a Canberra footpath is celebrated by his media mates.
La vie en rose
European Countries React to Navalny's Death by Summoning Russian Diplomats
The Temptations - My Girl (Smokey Robinson Tribute) 2006 Kennedy Cent
Israel has gone ‘beyond self-defence’ in Gaza, says Labour’s Streeting
Orlando Cachaito Lopez Redencion
English farmers to be offered ‘largest ever’ grant scheme amid food security concerns
Edith Piaf - NON, JE NE REGRETTE RIEN
Cameron government knew Post Office ditched Horizon IT investigation
RADETZKY MARCH-2008-Wien, New Year Concert
EU Calls for Immediate Ceasefire in Gaza Conflict
Only you (And you alone)
EU Vows To Hold Putin "Accountable" After Meeting Alexei Navalny's Wife
Strangers In The Night
EU Launches Probe Into TikTok Over Child Protection Under Digital Content Law
Charles Aznavour - La Boheme
The EU Initiates Naval Mission to Defend Red Sea Trade Routes
Summer time
EU and UK Announce Joint Effort on Migration
Sting and Stevie Wonder - Fragile (from Sting's 60th birthday concert)
Brazil's Lula Likens Gaza Operation to Holocaust, Israel Says "Red Line" Crossed
Aux Champs Elysees
Ministers Confirm Proposal to Prohibit Mobile Phone Usage in English Schools
Stand By Me - Ben E. King (Boyce Avenue acoustic cover)
Microsoft-backed OpenAI valued at $80bn after company completes deal
La Mer (Beyond the Sea) – Avalon Jazz Band
‘Alexei would want to tell Russia not to give up fighting’
She
Rwandan Footballer's Dismissal Sparks Concerns Over UK Asylum Plan
Nathalie Song by Enzo Petrachi Stjepan Hauser Cello
Whisky Challenges China's Baijiu Market During New Year Celebrations
Shape of My Heart - Sting (Boyce Avenue acoustic cover)
Avdiivka - Symbol Of Ukrainian Resistance Now In Control Of Russian Troops
Radiohead - Creep
Putin Critic Alexei Navalny's "Killers" Refusing To Hand Over Body, Say Allies
Quizás,Quizás,Quizás - Andrea Bocelli - Jennifer Lopez
"Historic Step": Zelensky Signs Security Pact With Germany
Perhaps, Perhaps, Perhaps - Multi-Couples
"Historic Step": Zelensky Signs Security Pact With Germany
Pentatonix Havana
20 Tech Giants Sign Effort To Fight AI Election Interference Across Globe
Paula Cole - Autumn Leaves
Joe Biden Accuses Putin of Causing Navalny's Death
Oscar Benton Bensonhurst Blues
Russian opposition leader Alexey Navalny has died at the Arctic prison colony
OH NANANA vs ABUSADAMENTE
Tucker Carlson grocery shopping in Russia. This is so interesting.
Nina Simone - ”I Put A Spell On You”. Vezi aici cum cântă Jeremy Ragsd
Julian Assange's Wife Warns of His Death if Extradited to US
NIGHTWISH - The Phantom Of The Opera
‘A lot higher than we expected’: Russian arms production worries Europe’s war planners
Motorshow 2016 Tanjay Negros Oriental
Greece Legalizes Same-Sex Marriage and Adoption Rights
Monica Bellucci - Ti Amo
Hungarian Foreign Minister: Europeans will lose Europe, the Union's policy must change drastically
Michael Jackson - Billie Jean Milena The Voice France 2018
In Britain Homeowners are receiving CPO’s (Compulsory Purchase Orders) so their homes can be redistributed to migrants
Michael Buble (Help Me Make It Through The Night) feat Loren Allred
Memories Canon In D - Maroon 5 (Boyce Avenue piano acoustic cover)
Matteo Simoni - Marina
Maroon 5 - One More Night
Maroon 5 - Memories
Mark Knopfler - Brothers In Arms (Berlin 2007 Live)
Mark Knopfler & Emmylou Harris - Romeo And Juliet (Real Live Roadrunni
Marina, Marina - The LUCKY DUCKIES intimist live concert at Guimarães
Major Lazer & DJ Snake – Lean On Mauranne The Voice France 2016
Love Theme from Romeo and Juliet - Joslin - Henri Mancini, Nino Rota
LoLa & Hauser - Love Story
Linkin Park Jay-Z - Numb Encore (Live 8 2005)
Hallelujah Mennel Ibtissem, The Voice France Leonard Cohen
Leonard Cohen - Dance Me to the End of Love
Leonard Cohen & Natasha Rostova - Dance me to the end of love
La casa de papel - Bella Ciao
La Camisa Negra
L'italiano (Toto Cutugno) - The Gypsy Queens
Juanes - La Camisa Negra
Jonathan and Charlotte - Britain's Got Talent 2012 Live Semi Final - U
John Powell - Assassin's Tango
Joe Cocker - You Can Leave Your Hat On (LIVE in Dortmund)
Joe Cocker - Unchain My Heart 2002 Live
Joe Cocker - A Whiter Shade Of Pale
Jay Z & Alicia Keys - Empire State of Mind LIVE
Jason Mraz - Im Yours (live)
Jarrod Radnich - Bohemian Rhapsody - Virtuosic Piano Solo
James Blunt - You're Beautiful
James Blunt - You're Beautiful & Bonfire Heart (Live at The Nobel Peac)
If You Go Away - Helen Merrill & Stan Getz (Tribute to Virna Lisi)
I'LL BE MISSING YOU
I Say a Little Prayer
Hotel California ( Eagles ) 1994 Live
Historia de un amor - Luz Casal. Vezi interpretarea Biancăi Sumanariu
Here Comes The Sun - The Beatles (Boyce Avenue acoustic cover) on Spot
Heart - Stairway to Heaven Led Zeppelin - Kennedy Center Honors
HAVANA by Camila Cabello Zumba Pre Cooldown TML Crew Kramer Pastra
HAUSER and Señorita - I Will Always Love You
HAUSER - Waka Waka
HAUSER - Sway
HAUSER - Lambada
HAUSER - Historia de un Amor
HAUSER - Despacito
Great Pretender
Georgia May Foote & Giovanni Pernice Samba to 'Volare' - Strictly Come
Gary Moore - Still Got The Blues
GIPSY KINGS VOLARE Penelope Cruz
Fugees - Killing Me Softly With His Song
French Latino - Historia de un Amor
For A Few Dollars More The Danish National Symphony Orchestra (Live)
Flashdance • What a Feeling • Irene Cara
Filip Rudan - “Someone You Loved” Audicija 4 The Voice Hrvatska Sez
Eric Clapton - Wonderful Tonight
Enya - Only Time
Enrique Iglesias - Bailando (English Version) ft. Sean Paul
Enrique Iglesias - Bailamos
Elena Yerevan Historia de un amor
Ed Sheeran - Shape of You (Official Music Video)
Ed Sheeran - Perfect Symphony [with Andrea Bocelli]
Ed Sheeran - Perfect (Official Music Video)
Easy On Me - Adele (Boyce Avenue 90’s style piano acoustic cover) on S
ERA - Ameno
ELENA YEREVAN- Cancion Del Mariachi-IN STUDIO-2017 DPR
Dust In The Wind - Kansas (Boyce Avenue acoustic cover)
Don't Let Me Be Misunderstood
Despacito x Shape Of You - Pentatonix
Deep Purple - Child In Time - Live (1970)
David Foster When A Man Loves A WomanIt's A Mans World (SealMichael Bo
Dance me to the end of Love ( Pi-Air Design )
Coolio - Gangsta's Paradise (feat. L.V.) [Official Music Video]
Conquest Of Paradise (Vangelis), played on Böhm Emporio organ
Cielito Lindo
Chico & The Gypsies - Bamboleo
Canción Del Mariachi - Antonio Banderas, Los Lobos • Desperado
Camila Cabello - Havana (Audio) ft. Young Thug
Camila Cabello - Havana ( cover by J.Fla )
California Dreamin' - The Mamas & The Papas José Feliciano (Boyce Ave
Buster Benton - Money Is The Name of The Game
Hallelujah Pentatonix
Bobby McFerrin - Don't Worry Be Happy (Official Music Video)
Bob Dylan - Knockin' On Heaven's Door Emilia The Voice Kids France
Besame Mucho - Cesaria Evora
Ben E. King - Stand by Me Sax Cover Alexandra Ilieva Thomann
Bella Ciao
Bella Ciao - INSTRUMENTAL
Beautiful in White x Canon in D (Piano Cover by Riyandi Kusuma)
Bad Romance - Vintage 1920's Gatsby Style Lady Gaga Cover ft. Ariana Savalas & Sarah Reich(1)
BELLA CIAO 2020 - KARAOKE ITALIANO
BAMBOLEO - Gipsy Kings • Antonio Banderas, Katya Virshilas
BAILANDO (original)
Awesome Ukrainian yodeler - SOFIA SHKIDCHENKO (with English subtitles)
Avicii - The Nights
Atom - The Great Gig in the Sky
Aretha Franklin - (You Make Me Feel Like) A Natural Woman (Official Ly
Antonio Banderas - Cancion del Mariachi (Desperado)
André Rieu - Zorba's Dance (Sirtaki)
André Rieu - Can't Help Falling In Love
André Rieu & Mirusia - Ave Maria
Andrew Reyes Elton John - Don't Let The Sun Go Down The Voice 2020 (
Andreas Kümmert Whiter Shade Of Pale The Voice of Germany 2013 Showd
And I Love You So
All About That Bass - Postmodern Jukebox European Tour Version
Alan Walker - Faded (Piano Cover)
Ain't No Sunshine -- Bill Withers (cover by Canen 12 y.o.)
African music
Adriana Vidović - “Creep” Audicija 4 The Voice Hrvatska Sezona 3
Adriana Vidović - “Believer” Nokaut 3 The Voice Hrvatska Sezona 3
A Fistful of Dollars - The Danish National Symphony Orchestra and Tuva
4 Beautiful Soundtracks Relaxing Piano [10min]
2CELLOS - Whole Lotta Love vs. Beethoven 5th Symphony [OFFICIAL VIDEO]
2CELLOS - Smooth Criminal (Live at Suntory Hall, Tokyo)
2CELLOS - Smells Like Teen Spirit [Live at Sydney Opera House]
2CELLOS - Despacito [OFFICIAL VIDEO]
13 Year Old Girl Playing Il Silenzio (The Silence) - André Rieu
094.All About That Bass
00 - SADNESS PART 1
(Ghost) Riders In the Sky (American Outlaws Live at Nassau Coliseum, 1
(Everything I Do) I Do It For You - Bryan Adams (Boyce Avenue ft. Conn
What a wonderful world
Moon river
×