London Daily

Focus on the big picture.

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."
Researchers found students to have fared better at accounting exams than ChatGPT, OpenAI's chatbot product.

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."

The researchers from Brigham Young University (BYU), US, and 186 other universities wanted to know how OpenAI's technology would fare on accounting exams. They have published their findings in the journal Issues in Accounting Education.

In the researchers' accounting exam, students scored an overall average of 76.7 per cent, compared to ChatGPT's score of 47.4 per cent.

While in 11.3 per cent of the questions, ChatGPT was found to score higher than the student average, doing particularly well on accounting information systems (AIS) and auditing, the AI bot was found to perform worse on tax, financial, and managerial assessments. Researchers think this could possibly be because ChatGPT struggled with the mathematical processes required for the latter type.

The AI bot, which uses machine learning to generate natural language text, was further found to do better on true/false questions (68.7 per cent correct) and multiple-choice questions (59.5 per cent), but struggled with short-answer questions (between 28.7 and 39.1 per cent).

In general, the researchers said that higher-order questions were harder for ChatGPT to answer. In fact, sometimes ChatGPT was found to provide authoritative written descriptions for incorrect answers, or answer the same question different ways.

They also found that ChatGPT often provided explanations for its answers, even if they were incorrect. Other times, it went on to select the wrong multiple-choice answer, despite providing accurate descriptions.

Researchers importantly noted that ChatGPT sometimes made up facts. For example, when providing a reference, it generated a real-looking reference that was completely fabricated. The work and sometimes the authors did not even exist.

The bot was seen to also make nonsensical mathematical errors such as adding two numbers in a subtraction problem, or dividing numbers incorrectly.

Wanting to add to the intense ongoing debate about how how models like ChatGPT should factor into education, lead study author David Wood, a BYU professor of accounting, decided to recruit as many professors as possible to see how the AI fared against actual university accounting students.

His co-author recruiting pitch on social media exploded: 327 co-authors from 186 educational institutions in 14 countries participated in the research, contributing 25,181 classroom accounting exam questions.

They also recruited undergraduate BYU students to feed another 2,268 textbook test bank questions to ChatGPT. The questions covered AIS, auditing, financial accounting, managerial accounting and tax, and varied in difficulty and type (true/false, multiple choice, short answer).
Newsletter

Related Articles

London Daily
0:00
0:00
Close
News roundup
Good day, everyone! We've got some gripping stories for you today, spanning from the Middle East to Europe, and even a touch of Hollywood.
Britain’s Refugee Visa Rules Stranding Children in War Zones
UK Elections Predict ‘Electoral Extinction’ for PM Sunak’s Conservative Party
Italian Activist Ilaria Salis Returns Home After Election to European Parliament
Good morning!
England Faces Serbia in Euro Opener with Defensive Concerns
Dermatologist Warns Against Sunbed Usage
Fake Pro-Reform UK Social Accounts and Their Influence on Elections
UK Man Jailed for Non-Consensual Condom Removal
Reform UK Surpasses Conservatives in Historic Poll
US, Britain, Canada Accuse Russia of Interference in Moldova’s Election
Taylor Swift Fans Create Seismic Activity in Edinburgh
Sunak Aide Under Investigation for Election Bet
Labour Leader Starmer Focuses on Wealth Creation for Upcoming UK Elections
G7 to Use Frozen Russian Assets for $50 Billion Ukraine Aid
Anti-Israel Irish MEP Clare Daly LOST her seat in the EU Election
Johnson & Johnson Settles Talc Safety Claims for $700 Million
EU Urged to Welcome Skilled Russians to Weaken Putin
EU Elections Overview: Far-Right Gains and Major Political Shifts
Israel Rescues Four Hostages from Gaza
Emmanuel Macron Calls for Snap Election
Jordan Bardella: Young Far-Right Leader Poised for Future Political Influence in France
World's Oldest Privately Owned Book Auctioned for $3.8 Million
Animal Rights Activists Deface King Charles' Portrait in Protest
Dutch Military Intel Uncovers Extensive Chinese Cyber Espionage
Turkish Student Arrested for Using AI to Cheat in University Exam
Rise in Dengue and Other Mosquito-Borne Diseases in Europe Due to Climate Change
EU Elections Overview: Far-Right Gains and Major Political Shifts
Far-Right National Rally Dominates France's EU Vote
Macron Calls Snap Legislative Elections After Far-Right Victory
Far-Right Gains Significantly in EU Election
UK Job Market Shows Signs of Recovery
Orban’s Fidesz Party Wins Majority in Hungary’s EU Elections as New Challenger Emerges
Meloni's Far-Right Party Wins European Elections in Italy
Key Insights from the European Union Elections
European Union Elections and Rise of Far-Right Parties
England Loses Over 260,000 Social Rent Homes in a Decade
Campaigners Urge Government to Block Shein's FTSE Listing
First NHS AI-Run Physiotherapy Clinic Launches This Year
British TV Presenter Michael Mosley Found Dead on Greek Island
Ukrainian Forces Claims First Strike on Russia's Su-57 Fighter Jet
Macron Dissolves Parliament and Calls Snap Elections
Russia Adds Yulia Tymoshenko to Wanted List
UK Foreign Secretary David Cameron Tricked by Hoax Caller Posing as Former Ukrainian President
Kate Middleton's Absence from Colonel's Review Due to Chemotherapy
UK Foreign Secretary Deceived by Prank Video Call
Sunak Criticised Over D-Day Exit in BBC Debate
Rishi Sunak Apologizes for Leaving D-Day Commemoration Early
UK Woman Sentenced After Causing Fatal Crash While Sending Selfies
×