ChatGPT tried to pass a college exam and it didn’t go well

Studying to pass your college exams can be a stressful experience, but what if ChatGPT could just write your papers for you? That’s the question posed by researchers at the U.K.’s University of Bath, and their findings might surprise you.

According to a report from the BBC, artificial intelligence (AI) tools like ChatGPT excel at certain tasks. James Fern, a lecturer at the University of Bath, noted that “Multiple choice questions, for example, [ChatGPT] will handle those very well. We definitely were not expecting it to do as well as it did … it was getting close to 100% correct.”

Yet there were tell-tale signs that the paper it produced was not crafted by a human brain. ChatGPT struggled with more complex questions that required a degree of creativity rather than simply regurgitating facts or picking from multiple options.

When answering a question on why it is important to understand the timing of exercise in relation to nutrition status, for instance, ChatGPT slipped up several times. It repeated parts of the question word for word on two occasions, created vague and unattributed statements, and wrote in oversimplified language without being specific.

The chatbot even invented fake academic papers and then cited them in its answer. While the names of the authors and journals were correct, the cited articles simply did not exist, which would be easy for examiners to spot with just a cursory check.

So, while ChatGPT can produce convincing answers in many different contexts, writing your college thesis for you seems to be a step too far for now.

An uncertain future

Hatice Baran / Unsplash

ChatGPT’s academic future isn’t entirely in tatters, however. Gillian Keegan, the U.K.’s Education Secretary, believes that it is “making a difference in schools and universities already,” and that it could provide assistance to teachers when it comes to lesson planning and marking.

It can also help students get started on tasks and generate ideas that they can then expand upon. While it might not be able to properly answer exam papers, it can at least provide a starting point for further research.

In any case, ChatGPT isn’t going away, and universities and colleges around the world are going to have to adapt to it. It has a lot of potential to improve productivity in the future, and colleges are already teaching students how to use it to improve their output.

But while ChatGPT is unlikely to be awarded a degree any time soon, its potential for misuse is still very real. Apple founder Steve Wozniak has just warned that chatbots could make scams even more realistic, and he was part of a group of prominent tech leaders who signed an open letter in March calling for a pause on further chatbot development. ChatGPT’s future is very much uncertain, but it is no doubt being studied at colleges and universities up and down the land.

Source