By Associate Professor Guy Curtis, School of Psychological Science

When ChatGPT launched in November 2022 universities around the world scrambled to understand what this technology meant for educational assessment. The very rapid conclusion that many people came to was that this new technology could be used for cheating.

It’s important, however, to consider what we really mean by “cheating” and what the objective of educational assessment is anyway. If we take a concrete example of the kind of things people were worried about, it might look something like this: A student is set an assignment where the topic is to “Write an essay on the theme of death in Shakespeare’s play Hamlet”. ChatGPT would allow a student to enter the essay topic and produce a substantial amount of text that the student could put their name to and submit as if it was their own work.

Stepping back from this example, we can ask the very reasonable question: What is the purpose of this essay as an educational assessment? The purpose of the assessment would be to understand whether a student had met learning outcomes that they were supposed to meet in their studies. These learning outcomes could include research skills, critical thinking, clear academic writing and communication, and knowledge of the subject matter.

A critical point that educators must keep in mind is that for any educational assessment to meet its intended purpose it has to be “valid”. By valid, what we mean is that an assessment should measure what it is supposed to measure. Validity of assessment is a concept that is regularly discussed in psychology (does an IQ test really measure intelligence?) But, validity is not so often talked about in the context of educational assessment.

Image: Associate Professor Guy Curtis, School of Psychological Science.

If a student uses ChatGPT to answer their Hamlet essay question, ultimately what we’re really worried about is that the assignment is no longer validly testing whether the student has met intended learning outcomes of their course. Cheating, essentially, is not doing an assessment in the way that it was supposed to be done, and taking a shortcut of some kind that undermines the validity of the assessment.

Artificial intelligence (AI) has been seen as a new problem in higher education because it appears as though students can easily use it to cheat on many traditional forms of assessment. However, most of the problems that have been associated with AI and academic assessment have been known or foreseen for several years. For example, researchers have known for over a decade about the challenge of students using computer-powered paraphrasing to attempt to evade plagiarism detection via text-matching software. And, authors of the book Cheating Academic Integrity, which was published eight months before the release of ChatGPT, speculated about the impending threat to assessment validity posed by AI.

Unlike copying from existing sources and colluding with other students, material produced by ChatGPT may be impossible to detect with text-matching software. But, this too is not really a new challenge to the validity of educational assessment. For many years we have known that students can submit freshly written work produced by other people, even family and friends, where detecting such outsourcing is difficult. And yet, we have ways to do this.

In good news, UWA was quick to respond to the challenge of ChatGPT use in educational assessment; updating our Academic Integrity policy with clear guidance for students as to its acceptable use at the start of 2023. Nonetheless, educators will need to continue to think carefully about whether the assessments they set their students continue to be valid in a world where AI exists. The Tertiary Education Quality and Standards Agency has released some excellent guidelines to help educators continue to validly assess students’ learning in a post-ChatGPT world. Still, significant time, effort, and thought will be needed to ensure that educational assessment remains valid in the future.

Guy Curtis is an Associate Professor in Applied Psychology at the School of Psychological Science. He researches applied psychology with a particular focus on individual differences in academic integrity and leadership.

Read the full issue of the Winter 2024 edition of Uniview [Accessible PDF 12MB]