Top marks for ChatGPT in the Leaving Certificate Computer Science Examination
Blog
- International Women’s Day: Celebrating Ad Astra Scholar Ava Canning
- Chinese New Year 2024 Celebration
- MSc Advanced Software Engineering Alumni
- Generative AI in Computing Education: Wrecking Ball or Holy Grail?
- My internship in digital strategy at Limerick City & County Council
- Clown Computing 101
- International Men's Day 2023
- Top marks for ChatGPT in the Leaving Certificate Computer Science Examination
- Alumnus Interview
- Student profile: Pasika Ranaweera, PhD student
- Staff profile: Associate Professor Neil Hurley, Head of School
- UCD CS PhD candidate award from IEEE Consumer Electronics Magazine
- UCD-Insight Collaboration Wins Prestigious Publication Award
- W@CS Alumni Roundtable
- Staff profile: Dr. Fatemeh Golpayegani
- Interning at a Smaller Tech Company
- Powering through the pandemic: My Remote Research Internship Experience
- Exploring Sense of Belonging in Computer Science Students
- Student Inter-Society Tech & Enterprise Meetup (SISTEM) held in UCD
- Computer Science Research and the COVID -19 Pandemic
- Zoom fatigue: how to make video calls less tiring
- SIGCSEire Launched at UCD CS
- Best Paper at the International Conference on Case-Based Reasoning (2019)
- ‘Spare tire genes’ explain why some genes can be lost by cancer cells
- Bi-annual CS Graduate Research Symposium
- UCD CS Postdoctoral fellow Claudia Mazo selected as a member of the ACM Future of Computing Academy
- Security, Privacy and Digital Forensics in the Cloud
- Chidubem Iddianozie: PhD student and GitHub Ambassador in UCD
- UCD CS PhD student selected to attend the Heidelberg Laureate Forum
- New Project: Evidence-Based Decision Support for Real-Estate Investment
- UCD projects celebrate Europe Day
- Research Award at the 2019 ACM SIGCSE Technical Symposium on Computer Science Education
- Top Tips for Student Scholarships
- I am a Computer Scientist and a Cancer Biologist
- Critical thinking and data ethics in UCD CS
- Teaching at BDIC Beijing
- Reading 35,000 Books
- Secret to a Great Internship
- 12 Tips for PhD Researchers
- Buddy Coders - a new initiative to support women in Computer Science
Top marks for ChatGPT in the Leaving Certificate Computer Science Examination
ChatGPT Plus has demonstrated the ability to successfully pass the Higher Level (opens in a new window)Leaving Certificate Computer Science (LCCS) examination by answering nearly all questions correctly.
The LCCS examination consists of an end-of-course examination worth 70% of the grade and coursework assessment worth 30%. We conducted an experiment that utilized the 2021 LCCS Higher-Level examination, which included 16 questions with a total of 62 sub-questions. The sub-questions were presented to ChatGPT in separate chat sessions without any additional information. The answers provided by ChatGPT were then compared to the official Leaving Certificate marking scheme, resulting in a majority of correct answers (55 out of 62).
There is an element of choice in the LCCS examination, so considering only ChatGPT's best answers would likely yield close to or possibly 100%, depending on the marker's interpretation of the marking criteria. This would correspond to the highest grade possible (H1) in the 2021 Higher Level exam. With the examination comprising 70% of the overall grade, achieving a grade of H3 would be possible even before considering the coursework component worth 30%. For comparison, only (opens in a new window)14% of students achieved a H1 in 2021.
The Leaving Certificate is an invigilated exam, and was invigilated long before Generative AI, and for good reason. So although ChatGPT can excel in the LCCS, it does not compromise the exam's integrity. Furthermore, the LCCS includes a (opens in a new window)coursework assessment component, and this aspect is supervised by the students' teachers. While ChatGPT could be used for this, the nature of this project work is much more involved than the questions found on a paper exam. You can read more about this work at (opens in a new window)brettbecker.com or at (opens in a new window)independent.ie.
Joyce Mahon is a Computer Science PhD student at UCD and part of the Science Foundation Ireland Centre for Research Training in Machine Learning ((opens in a new window)https://www.ml-labs.ie/ ) supervised by (opens in a new window)Dr. Brett Becker and (opens in a new window)Dr. Brian Mac Namee.
Published 04.07.23