London Daily

Focus on the big picture.
Saturday, May 31, 2025

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."
Researchers found students to have fared better at accounting exams than ChatGPT, OpenAI's chatbot product.

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."

The researchers from Brigham Young University (BYU), US, and 186 other universities wanted to know how OpenAI's technology would fare on accounting exams. They have published their findings in the journal Issues in Accounting Education.

In the researchers' accounting exam, students scored an overall average of 76.7 per cent, compared to ChatGPT's score of 47.4 per cent.

While in 11.3 per cent of the questions, ChatGPT was found to score higher than the student average, doing particularly well on accounting information systems (AIS) and auditing, the AI bot was found to perform worse on tax, financial, and managerial assessments. Researchers think this could possibly be because ChatGPT struggled with the mathematical processes required for the latter type.

The AI bot, which uses machine learning to generate natural language text, was further found to do better on true/false questions (68.7 per cent correct) and multiple-choice questions (59.5 per cent), but struggled with short-answer questions (between 28.7 and 39.1 per cent).

In general, the researchers said that higher-order questions were harder for ChatGPT to answer. In fact, sometimes ChatGPT was found to provide authoritative written descriptions for incorrect answers, or answer the same question different ways.

They also found that ChatGPT often provided explanations for its answers, even if they were incorrect. Other times, it went on to select the wrong multiple-choice answer, despite providing accurate descriptions.

Researchers importantly noted that ChatGPT sometimes made up facts. For example, when providing a reference, it generated a real-looking reference that was completely fabricated. The work and sometimes the authors did not even exist.

The bot was seen to also make nonsensical mathematical errors such as adding two numbers in a subtraction problem, or dividing numbers incorrectly.

Wanting to add to the intense ongoing debate about how how models like ChatGPT should factor into education, lead study author David Wood, a BYU professor of accounting, decided to recruit as many professors as possible to see how the AI fared against actual university accounting students.

His co-author recruiting pitch on social media exploded: 327 co-authors from 186 educational institutions in 14 countries participated in the research, contributing 25,181 classroom accounting exam questions.

They also recruited undergraduate BYU students to feed another 2,268 textbook test bank questions to ChatGPT. The questions covered AIS, auditing, financial accounting, managerial accounting and tax, and varied in difficulty and type (true/false, multiple choice, short answer).
Newsletter

Related Articles

0:00
0:00
Close
Satirical Sketch Sparks Political Spouse Feud in South Korea
Indonesia Quarry Collapse Leaves Multiple Dead and Missing
South Korean Election Video Pulled Amid Misogyny Outcry
Asian Economies Shift Away from US Dollar Amid Trade Tensions
Netflix Investigates Allegations of On-Set Mistreatment in K-Drama Production
US Defence Chief Reaffirms Strong Ties with Singapore Amid Regional Tensions
Vietnam Faces Strategic Dilemma Over China's Mekong River Projects
Malaysia's First AI Preacher Sparks Debate on Islamic Principles
White House Press Secretary Criticizes Harvard Funding, Advocates for Vocational Training
France to Implement Nationwide Smoking Ban in Outdoor Spaces Frequented by Children
Meta and Anduril Collaborate on AI-Driven Military Augmented Reality Systems
Russia's Fossil Fuel Revenues Approach €900 Billion Since Ukraine Invasion
U.S. Justice Department Reduces American Bar Association's Role in Judicial Nominations
U.S. Department of Energy Unveils 'Doudna' Supercomputer to Advance AI Research
U.S. SEC Dismisses Lawsuit Against Binance Amid Regulatory Shift
Alcohol Industry Faces Increased Scrutiny Amid Health Concerns
Italy Faces Population Decline Amid Youth Emigration
U.S. Goods Imports Plunge Nearly 20% Amid Tariff Disruptions
OpenAI Faces Competition from Cheaper AI Rivals
Foreign Tax Provision in U.S. Budget Bill Alarms Investors
Trump Accuses China of Violating Trade Agreement
Gerry Adams Wins Libel Case Against BBC
Russia Accuses Serbia of Supplying Arms to Ukraine
EU Central Bank Pushes to Replace US Dollar with Euro as World’s Main Currency
Chinese Woman Dies After Being Forced to Visit Bank Despite Critical Illness
President Trump Grants Full Pardons to Reality TV Stars Todd and Julie Chrisley
Texas Enacts App Store Accountability Act Mandating Age Verification
U.S. Health Secretary Ends Select COVID-19 Vaccine Recommendations
Vatican Calls for Sustainable Tourism in 2025 Message
Trump Warns Putin Is 'Playing with Fire' Amid Escalating Ukraine Conflict
India and Pakistan Engage Trump-Linked Lobbyists to Influence U.S. Policy
U.S. Halts New Student Visa Interviews Amid Enhanced Security Measures
Trump Administration Cancels $100 Million in Federal Contracts with Harvard
SpaceX Starship Test Flight Ends in Failure, Mars Mission Timeline Uncertain
King Charles Affirms Canadian Sovereignty Amid U.S. Statehood Pressure
Trump Threatens 25% Tariff on iPhones Amid Dispute with Apple CEO
Putin's Helicopter Reportedly Targeted by Ukrainian Drones
Liverpool Car Ramming Incident Leaves Multiple Injured
Australia Faces Immigration Debate Following Labor Party Victory
Iranian Revolutionary Guard Founder Warns Against Trusting Regime in Nuclear Talks
Macron Dismisses Viral Video of Wife's Gesture as Playful Banter
Cleveland Clinic Study Questions Effectiveness of Recent Flu Vaccine
Netanyahu Accuses Starmer of Siding with Hamas
Junior Doctors Threaten Strike Over 4% Pay Offer
Labour MPs Urge Chancellor to Tax Wealthy Over Cutting Welfare
Publication of UK Child Poverty Strategy Delayed Until Autumn
France Detains UK Fishing Vessel Amid Post-Brexit Tensions
Calls Grow to Resume Syrian Asylum Claims in UK
Nigel Farage Pledges to Reinstate Winter Fuel Payments
Boris and Carrie Johnson Welcome Daughter Poppy
×