London Daily

Focus on the big picture.
Saturday, May 10, 2025

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."
Researchers found students to have fared better at accounting exams than ChatGPT, OpenAI's chatbot product.

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."

The researchers from Brigham Young University (BYU), US, and 186 other universities wanted to know how OpenAI's technology would fare on accounting exams. They have published their findings in the journal Issues in Accounting Education.

In the researchers' accounting exam, students scored an overall average of 76.7 per cent, compared to ChatGPT's score of 47.4 per cent.

While in 11.3 per cent of the questions, ChatGPT was found to score higher than the student average, doing particularly well on accounting information systems (AIS) and auditing, the AI bot was found to perform worse on tax, financial, and managerial assessments. Researchers think this could possibly be because ChatGPT struggled with the mathematical processes required for the latter type.

The AI bot, which uses machine learning to generate natural language text, was further found to do better on true/false questions (68.7 per cent correct) and multiple-choice questions (59.5 per cent), but struggled with short-answer questions (between 28.7 and 39.1 per cent).

In general, the researchers said that higher-order questions were harder for ChatGPT to answer. In fact, sometimes ChatGPT was found to provide authoritative written descriptions for incorrect answers, or answer the same question different ways.

They also found that ChatGPT often provided explanations for its answers, even if they were incorrect. Other times, it went on to select the wrong multiple-choice answer, despite providing accurate descriptions.

Researchers importantly noted that ChatGPT sometimes made up facts. For example, when providing a reference, it generated a real-looking reference that was completely fabricated. The work and sometimes the authors did not even exist.

The bot was seen to also make nonsensical mathematical errors such as adding two numbers in a subtraction problem, or dividing numbers incorrectly.

Wanting to add to the intense ongoing debate about how how models like ChatGPT should factor into education, lead study author David Wood, a BYU professor of accounting, decided to recruit as many professors as possible to see how the AI fared against actual university accounting students.

His co-author recruiting pitch on social media exploded: 327 co-authors from 186 educational institutions in 14 countries participated in the research, contributing 25,181 classroom accounting exam questions.

They also recruited undergraduate BYU students to feed another 2,268 textbook test bank questions to ChatGPT. The questions covered AIS, auditing, financial accounting, managerial accounting and tax, and varied in difficulty and type (true/false, multiple choice, short answer).
Newsletter

Related Articles

0:00
0:00
Close
Cardinal Robert Prevost Elected as Pope Leo XIV, Marking a Historic Papacy
Newark Mayor Ras Baraka Arrested at ICE Facility Amid Congressional Visit
India-Pakistan conflict may be first test for Chinese military tech
Bill Gates Announces Plan to Wind Down Philanthropic Foundation and Disperse Wealth
Historic Papal Conclave Set to Commence in Rome
Huge Copper, Gold, and Silver Discovery in Argentina and Chile — But the Profits Go Abroad
Prince Harry is pleading for reconciliation — but the royals are just as sick of his victimhood as everyone else
The Road to Freedom: She Protested Putin, Escaped House Arrest, and Survived a 2,800-Kilometer Journey
OpenAI's Flip-Flop: No Longer Going Commercial, Back to Nonprofit, After Musk Lawsuit and Backlash
“Trump Supporter” Aims to Bring a MAGA-Style Shift to Romania
First From China: Zhao Xintong Wins the Snooker World Championship
Nvidia Faces Billion-Dollar Losses – Warns: China Is on Its Way to Becoming an AI Superpower
Trump Rules Out Third Term, Names JD Vance and Marco Rubio as Potential Successors
Mexico Says ‘No’ to U.S. Troops: President Sheinbaum Rejects Trump’s Offer to Fight Cartels
Nigel Farage’s Reform UK Storms the Map, Wrecking the Two-Party Monopoly
DOGE: Reimagining Government Operations with AI
Common Sense Returns to Britain's Legal System: UK Supreme Court Declares a Woman Is… a Woman
Beijing Says U.S. Is ‘Reaching Out’ for Tariff Talks Amid Soaring Trade Tensions
U.K. Court Rejects Prince Harry’s Final Appeal Over Police Security
Prince Harry’s Heartfelt Outburst Rocks the Royal Family
Trump Shares AI-Generated Image of Himself as… Pope, Prompting Outrage Reaction
Transgender Swimmer Secures Five Gold Medals at U.S. Masters Championship
Prince Harry: “I Want Reconciliation with My Family”
Germany's Alternative für Deutschland (AfD) party has now been officially labeled “right-wing extremist” by the federal office for the so-called “protection of the constitution.”
Amazon Launches Satellite Internet Service Amidst Competition with SpaceX
Transformative Changes in Women's Wrestling: The Rise of WWE Superstars
The Rush to the White Gold: Global Investment Surge in Natural Hydrogen Exploration
This is a day in Spain without electricity and internet
Reform UK Surprises in British Elections, Challenging Traditional Two-Party System
180-Year-Old Christian University in South Carolina Announces Closure Due to Unmet $6 Million Fundraising Goal
Brazilian Woman Jailed for Fourteen Years for Writing “You Lost, Idiot” on Statue During Protest
Trump Administration Removes National Security Adviser Mike Waltz Amid Signal Chat Controversy
Dutch Politician Eva Vlaardingerbroek Receives Spyware Threat Alert from Apple
Paramount Board Considers Settlement in Trump’s $20 Billion Lawsuit Over "60 Minutes" Interview
U.S. Economy Shrink in Trump’s First Quarter as Tariff Policy Raises Questions
Deadline Looms for RTS Meter Replacement: Hundreds of Thousands at Risk of Heating Disruption
Sweden Grapples with Deadly Gun Violence: Suspect Arrested After Three Young Men Killed in Uppsala Hair Salon
Walz Reveals Why Harris Chose Him as Her Running Mate and Reflects on Democratic Losses
Spain Restores Power After Unprecedented Nationwide Blackout
Carney Secures Liberal Mandate in Canada’s Federal Election
Death Penalty Sought as Luigi Manion Pleads Not Guilty in CEO Murder Case
President Trump contacts Jeff Bezos after reports of Amazon considering listing tariff surcharges; company clarifies no such plan for main platform
Spain and Portugal Recover from Massive Blackout
Liverpool Clinches Record-Equalling 20th English League Title Under Arne Slot
Singapore Politicians Warn Against Foreign Interference in Election
Driver Ploughs into Vancouver Festival Crowd, Killing Nine
Depression, Fear of Defamation, and a Tragic End: New Details on Virginia Giuffre’s Suicide
“Sharia for UK, Allah Akbar!”
Massive Explosion at Iran's Bandar Abbas Port Linked to Suspicious Chemical Shipments
Incident Reflection: A Harsh Reality Check
×