Assessment Integrity in the Era of Large Language Models: Threats and Opportunities within the UCD College of Engineering & Architecture

This project responded to the epochal launch of ChatGPT in late 2022. Could we prevent these tools from subverting the integrity of the coursework assessments we mandate in our engineering degrees?

Our project first diagnosed the scale of the threat. We systematically audited over 850 live modules across the UCD College of Engineering & Architecture. We inspected granular assessment strategies to gauge how readily a student could enlist ChatGPT as a cheating aid to subvert the academic integrity of particular modules. We used a “traffic light” system to flag the threat level: alas, there was a lot of red ink in these audit report.

Internally, we sounded the alarm about this threat as loudly as we could. We published papers about this for the wider community. We engaged with students themselves to brainstorm remedies; we rolled out remediations to assessment strategies in two case study modules.

SATLE priority area: Academic Integrity

Context and key activities

ChatGPT’s release in December 2022 broke assessment integrity in higher education overnight. Large language models could suddenly write prose, debug code, solve technical problems, design circuits: exactly what we test in engineering. Our project launched mid-2023 in direct response. Traditional assessments (homework, coding exercises, essays) faced a destabilising threat from tools any student could access through a browser.

Our audit of 850 modules across UCD’s College of Engineering & Architecture wasn’t busywork. We wanted to show leadership the real extent of our exposure. We used a traffic-light framework: red for modules most vulnerable to AI plagiarism. As engineers, we needed numbers based on objective, college-wide data. The early discourse about AI in education was often too anecdotal, too evasive. Our analysis used actual assessment strategies from live module descriptors and aimed at real action by leadership and module coordinators.

Goals and Key Activities

Two primary aims, three work packages:

First: diagnose the most urgent threats through a comprehensive audit, creating a traffic-light classification for vulnerability to AI-enabled cheating.

Second: scope creative solutions for “LLM-aware” assessments that maintain quality while acknowledging AI exists: including teaching students to use these tools appropriately.

Work Package 1:

This “Threat Analysis” audited 850+ modules across six Schools. We classified assessments by vulnerability: low, medium, high. The findings went into an internal report (September 2023) shared with School leadership to inform protective measures. We published that framework at IEEE RTSI in Milan. We tested it against specific disciplines: one IEEE ITHET paper showed how ChatGPT handled power system optimization coursework, revealing which problems stayed secure and which became trivial. We spread these findings through an Engineers Ireland webinar (March 2024) and a UCD Teaching & Learning session (September 2023) where we asked coordinators bluntly: “How should you evolve your assessments in the ChatGPT era?”

Work Package 2:

This “Opportunity Analysis” centered on the AI for Learning Hackathon we ran in partnership with UCD ElecSoc in October 2023. This competition engaged students directly in solution development, offering substantial prizes to incentivize serious participation. Twenty-three students across eleven teams spent four days brainstorming interventions: from the kick-off event through to final pitches before an expert panel.

One team, Wrexham FC, noted: “Dread it, run from it, AI arrives all the same” Another team, ChatGPA, proposed “a whole new lab structure”, featuring “Face to face lab defences”

A nice provocation was offered by team DxJ: “AI didn’t break education. It was already flawed. If an AI can ace the test, is the test worth acing?” They suggested emphasising “Peer interaction and collaboration”: “Academic institutions should encourage the use of AI to help students achieve their learning objectives and tackle complex problems.”

While we appreciated the energy of the student teams, this hackathon surfaced a sobering conclusion: no clever tricks can salvage unsupervised, take-home assignments. Students admitted it during discussions. If you’re not watching them, some will cheat.

We shared these blunt findings at a Teaching & Learning session the week after (October 2023). Another output: we systematically tested ChatGPT on circuit design problems from actual textbooks. The results were dismaying enough to win Best Paper at IEEE ITHET 2024 in Paris; the tools handled undergraduate exercises with ease. We entered Work Package 2 open-minded about constructive AI integration. We left clear-eyed: we found no easy wins for upholding assessment integrity in unsupervised settings.

Work Package 3:

Pilot Implementations meant actually changing how we teach and assess, not just talking about it. We redesigned Arduino programming in EEEN10020 from take-home exercises to supervised in-class sessions using browser-based TinkerCAD simulations. In MEEN30130, we scrapped the traditional essay for an interactive poster session: harder to delegate to ChatGPT when you’re standing there defending your work. Both interventions were written up as public case studies documenting what worked and what didn’t.

As we worked through the project, monitoring AI’s escalating capabilities, running the hackathon, talking with students and colleagues, a consensus position emerged. Take-home assignments have always been insecure; AI just heightens this. Furthermore, students want meaningful learning experiences on campus. There’s a win-win staring us in the face.

Consider what take-home assignments actually look like: weeks of procrastinator’s guilt, then a panicked flurry before the deadline. AI or not, is that a genuine university experience? Compare it to this: come to campus, do a good day’s work in a semi-supervised environment, submit what you’ve completed, go home and enjoy your evening. Who knows, you might even socialize with your classmates. The culture of all-nighters and endless toil was perhaps always the wrong status quo in engineering education. AI might provoke us to undo that knot forever.

We can create memorable, meaningful, authentic learning experiences through better face-to-face timetabling. The hackathon students told us they wanted this. Our pilot implementations proved it works. We have a lovely campus: what are we waiting for?

Strategic alignment with national policies and relevant SATLE priority area

This project addresses the SATLE priority area of Academic Integrity, directly aligning with QQI’s Academic Integrity Guidelines and Ireland’s 2018 legislation targeting contract cheating. QQI has identified essay mills and AI-enabled cheating as growing threats to Irish higher education’s integrity; our work operationalized these national directives at institutional scale.

We quantified the threat large language models pose to assessment integrity across an entire engineering college: 850+ modules audited, vulnerabilities mapped, leadership briefed with actionable data. We measured actual exposure and tested whether ChatGPT could solve real coursework problems from textbooks. It could. That’s the integrity crisis, diagnosed and documented through systematic traffic-light classification published at IEEE RTSI. Our IEEE ITHET paper on power system optimization showed which problems remained secure and which became trivial. Another on circuit design won Best Paper at IEEE ITHET 2024: the tools handled undergraduate exercises with ease.

National discussions about AI in education can sometimes be frustratingly vague: preparing students for an “AI-augmented future” without confronting students using AI to cheat right now. QQI’s Academic Integrity Guidelines (2021) acknowledge that institutional support gaps and staff apathy contribute to misconduct. We tackled both: School leadership got data-driven threat assessments, faculty got practical interventions they could implement immediately.

The integrity challenge extends beyond undergraduates. Our IEEE paper on doctoral students using ChatGPT asked whether they actually acquire the advanced research skills their PhDs supposedly certify, addressing concerns raised in Ireland’s National Policy Statement on Ensuring Research Integrity (2019).

Integrity is also about perception and morale, concerns central to the HEA’s statutory responsibility to promote student engagement and student success. The National Forum report “Understanding and Enabling Student Success in Irish Higher Education” (2019) emphasizes that success extends beyond academic attainment to include student wellbeing, motivation, and meaningful engagement. We cannot tolerate the idea that anyone can coast to a UCD degree with a ChatGPT subscription. Even if most students are honest, suspicion that everyone else is cheating poisons effort and engagement. When assignments feel like farce, motivation collapses. When staff suspect widespread cheating, teaching becomes theatre, undermining the “high-quality educational experience” that the 2022 HEA Act mandates institutions provide.

Our project directly supports SATLE’s role in helping institutions take strategic, long-term approaches to teaching and learning enhancement, particularly in promoting academic integrity as a national priority.

Innovation in specified SATLE priority area

This project demonstrated meaningful innovation in Academic Integrity through four interconnected approaches:

Novel Methodological Innovation: The Traffic-Light Framework

The project’s most significant methodological innovation was the development of a systematic, data-driven traffic-light vulnerability assessment framework. This quantitative approach to evaluating assessment integrity risk was unprecedented in Irish higher education. By analyzing 850+ modules and classifying each assessment component as “In Person,” “At Home,” or “Hybrid,” the framework generated numerical exposure estimates on a 0-5 scale, providing module coordinators with clear, granular actionable intelligence about vulnerability levels of each assessed component they use.

This framework represented a paradigm shift from reactive, anecdotal responses toward evidence-based strategic planning across an entire portfolio of modules. The visual traffic-light system (green-amber-red) made complex data immediately comprehensible to diverse stakeholders, facilitating informed decision-making. The framework’s replicability means other institutions can adapt this methodology, making it a sector-wide innovation rather than a local solution.

Genuine Student Partnership

The AI for Learning Hackathon, run in conjunction with UCD ElecSoc, exemplified authentic student partnership innovation. Rather than imposing top-down solutions, the project positioned students as expert co-creators whose insights would shape institutional responses. Our use of attractive monetary incentives signalled our genuine valuation of student contributions, distinguishing this from tokenistic consultation exercises.

This approach challenged traditional power dynamics in curriculum design, acknowledging that students (as eager early-adopter AI tool users) possessed unique perspectives on both threats and opportunities. The hackathon format created a competitive yet collaborative environment where eleven teams developed concrete proposals addressing specific vulnerabilities identified in the audit. This moved beyond abstract discussion toward tangible solutions grounded in student experience.

Innovative Pedagogical Redesign

The project piloted two groundbreaking assessment innovations that balanced integrity protection with enhanced learning:

The Arduino programming redesign in EEEN10020 transformed a vulnerable take-home assignment into supervised in-class exercises using browser-based TinkerCAD simulations. This innovation addressed multiple objectives simultaneously: preventing AI misuse, reducing student procrastination and anxiety, fostering peer learning through social presence, and improving grade distribution authenticity. The approach demonstrated that integrity-preserving redesign could actually enhance educational quality and student wellbeing.

The essay-to-poster transformation in MEEN30130 replaced a traditional written essay with an interactive poster presentation featuring oral examination. This innovation selectively embraced AI tools (permitting image generation via Midjourney/DALL-E) while prohibiting text generation, teaching students nuanced AI literacy. The 70% weighting on oral presentation ensured authentic knowledge demonstration while acknowledging AI’s legitimate role in professional practice.

Institutional and Cultural Change

The project catalyzed cultural shifts in how UCD approaches assessment integrity. By framing AI as requiring adaptation rather than prohibition, it fostered institutional maturity about technological change. The emphasis on data-driven decision-making, scholarly dissemination through IEEE publications, and knowledge sharing through Teaching & Learning Communities of Practice embedded evidence-based practice into institutional culture.

Integration of Research and Practice

Finally, the project innovatively integrated pedagogical research with immediate practical application, producing four IEEE conference papers while simultaneously implementing solutions. This research-practice integration ensured innovations were both theoretically grounded and pragmatically viable, setting a model for scholarship in teaching and learning.

Outcomes

Impact

Building the Evidence Base for Institutional Action

The project’s primary impact was transforming a sectoral crisis into strategic, evidence-based response. When ChatGPT launched in December 2022, higher education discourse defaulted to anxious anecdotes. Our 850+ module audit generated quantitative data: a traffic-light vulnerability classification across UCD’s entire College of Engineering & Architecture. The internal report (September 2023) armed School leadership with actionable intelligence about their exposure. This wasn’t theoretical: our IEEE papers systematically tested whether ChatGPT could solve actual textbook problems in power systems optimization and circuit design. It could. The circuit design paper won Best Paper at IEEE ITHET 2024, validating our methodology while delivering uncomfortable truths about assessment security.

This evidence base catalyzed institutional change. Six Schools received vulnerability profiles for their modules. Assessment strategies shifted from wishful thinking (“maybe students won’t cheat”) to pragmatic redesign. Two exemplar modules, EEEN10020 and MEEN30130, moved from compromised take-home formats to supervised on-campus sessions. Published case studies (January 2025) documented what worked: reduced procrastination stress, improved student morale, higher engagement through face-to-face interaction.

Leading Sectoral Response at National Level

The project positioned UCD as a leader in Ireland’s response to AI-enabled academic dishonesty. We published our traffic-light framework at IEEE RTSI: making the methodology available for other institutions facing identical threats. The framework is discipline-agnostic; it works beyond engineering, as demonstrated when the project lead served as juror for a landscape architecture studio experimenting with AI integration.

National dissemination occurred through multiple channels. An Engineers Ireland webinar (March 2024) reached professional engineering educators across the country. Internal UCD Teaching & Learning Community of Practice sessions (September 2023, October 2023, January 2024) spread findings across disciplines. The Bridges & Bytes student podcast (November 2024) captured rare honest dialogue about cheating: students admitted what faculty didn’t want to hear about AI use. A lightning talk at the UCD Teaching & Learning Symposium (May 2025) sustained the conversation beyond the project end date.

Embedding Change and Enabling Replication

The project’s impact extends beyond its formal timeline. Assessment redesigns in EEEN10020 and MEEN30130 are now permanent module features. The audit framework remains available for ongoing module reviews as AI capabilities escalate. Four IEEE papers and two published case studies provide a complete toolkit for institutions building their own responses.

The work continues. A paper now in press with IEEE Transactions on Technology and Society (October 2025) explores how AI might reshape infrastructure design education: demonstrating the project’s methodology adapting to new questions. The project didn’t find silver bullets for assessment integrity, but it generated replicable frameworks, documented honest failures alongside successes, and showed that data-driven institutional responses beat panic and denial.

Resources

• Y. Mormul, J. Przybyszewski, A. Nakoud and P. Cuffe, “Reliance on Artificial Intelligence Tools May Displace Research Skills Acquisition Within Engineering Doctoral Programmes: Examples and Implications”, presented at IEEE International Conference on IT in Higher Education and Training, Paris, France, November 2024
• A. Hickey, C. O’Faolain and P. Cuffe, “Large Language Models in Power Engineering Education: A Case Study on Solving Optimal Dispatch Coursework Problems”, presented at IEEE International Conference on IT in Higher Education and Training, Paris, France, November 2024
• Y. Mormul, J. Przybyszewski, T. Siriburanon, J. Healy and P. Cuffe, “Gauging the Capability of Artificial Intelligence Chatbot Tools to Answer Textbook Coursework Exercises in Circuit Design Education”, presented at IEEE International Conference on IT in Higher Education and Training, Paris, France, November 2024
• A. Hickey, C. O’Faolain, J. Healy, K. Nolan, E. Doheny and P. Cuffe, “A Threat Assessment Framework for Screening the Integrity of University Assessments in the Era of Large Language Models”, presented at 8th IEEE International Forum on Research and Technologies for Society and Industry Innovation, Lecco, Italy, September 2024
• J. Li, A. Mukherjee and P. Cuffe “Guarding Code Originality and Motivating Student Engagement: In-class Simulated Microcontroller Assignments in a Stage 1 Robotics Module” case study in themed collection ‘Using Generative AI, GenAI, in T&L and Assessment in Higher Education’ January 2025
• J. Li, P. Cuffe and J. O’Donnell “From Essay to Poster: Exploring a New Format for an Energy Module Assignment” case study in themed collection ‘Using Generative AI, GenAI, in T&L and Assessment in Higher Education’ January 2025

Source: UCD Teaching & Learning showcase.