Assessments and The Power of AI

Good day Colleagues,

Welcome back to our TTT Blog, a place where we provide you with the most current, contextual tips and resources that support you in teaching excellence.

This week, as we continue to navigate the mid-semester phase, we would like to explore assessments and the power of artificial intelligence (AI). The integration of AI in Higher Education has profoundly impacted the processes and outputs, that is, creating change in how things are done, created or managed. This change ranges from the way course curricula are designed, to the implementation of learning activities and creation of teaching and learning artifacts. According to Selwyn, 2021, and Luckin et al, 2016, many educators have begun to view AI in Education (AIEd) as less of a threat and more as a pedagogical instrument or resource as such tools provide real-time feedback and meet personal learning needs. But perhaps most notably, the revolutionizing impact of AIEd can be seen in the domain of educational assessments.

A – Assessments and The AI Factor

The power of AI in educational assessments lies in the benefits it consistently provides. With responsible and ethical use, AI can assist in designing various formal (graded) or informal (ungraded) assessment activities, and may even assist in decoding large amounts of assessment data. For learners, it is often used as a guide that highlights various points or bits of information for consideration in answering questions or completing tasks. In such cases, AI is used as a resource. But while AI may provide a plethora of benefits, it is important that users are guided by  ethical guidelines that ensure academic integrity, as failure to do so may result in AI becoming a threat to the aforementioned benefits it provides.

B – AI and Online Assessments

When it comes to online assessments, conventional avenues include multiple choice quizzes, short-answer questions, essay questions and the like. However, AI has vastly improved the ways in which online assessments are designed, delivered, marked/graded and more. Whether it is through automated item generation and marking, to personalized assessments and malpractice detection, AI continues to revolutionize the way we teach and asses, and the way students learn. The following are some ways in which AI can benefit your online assessments:

1. Practical Assessments with AI:

   This may serve as a formal or informal online assessment activity, where students engage in role-play scenarios with AI-powered bots. These interactive and immersive experiences are almost likened to students completing a problem-based/project-based, proficiency-based or authentic assessment, as the AI technology can easily replicate real-world situations for a more impactful learning (and assessment) experience. This can be adapted to assessments and tasks in most fields including language learning, consultancy and other scenarios that the learner may eventually encounter in their future careers.In so doing, it helps to prepare the current learner for the future!

2. Automated Marking with AI:

   You may require your learners to submit written and/or spoken (verbal) responses to various assessment items/questions e.g. essays, SAQs etc. There are AI-powered tools, equipped with Natural Language Processing (NLP) models and/or speech and linguistic criteria, that can accurately analyze and assess learners’ written or verbal responses for marking/grading purposes. This automation allows for a more efficient, consistent and more objective grading process.

3. Assessment Analysis with AI:

   AI can perform various statistical and psychometric analyses which may provide you with substantive insight to the learners’ performance. In addition to interpreting results, it can provide visualizations of the learners’ performance, and thus allow you to see any possible strengths and weakness in their learning and assessment submissions. Moreover, it will highlight areas for improvement, thus allowing you to take a more data-driven approach to subsequent lesson planning, revision activities and other courses of action to address and/or mitigate the identified learning gaps.

4. Plagiarism Detection with AI:

   There is currently a range of AI marking tools that can now detect instances of plagiarism in learners’ written submissions (e.g. SAQ and essay responses). One example of this in our UWI context is the Turn-it-in platform. Accessible through all myeLearning courses, this is an activity tool that facilitates students’ submission of text-based content/assessments. Moreover, it has the ability to detect plagiarized content and with recent upgrades, it can now detect AI-generated content that the students may have unethically used in their submissions. Although this may not be 100% fail-proof in its detection, it is still an avenue you may consider to ensure that students use AI technologies responsibly and ethically.

5. Test Construction with AI:

   AI can now modify the ways in which assessments are created. No longer do you have to create assessment items manually via a Word document. AI allows for a more efficient process in which question items are created automatically, and in the case of MCQs for example, it can also generate distractors. How does this work? Well, it is based on AI algorithms that analyze existing question banks, find patterns and create new questions that fit within certain rules or conditions. It also has the potential to ensure validity in assessments by examining the quality and reliability of questions that comprise the assessment. It is recommended, however, that human-based quality assurance be in place to further avoid any possible issues relating to assessment validity and the like – in fact, we recommend that AI should always be monitored/reviewed by human input. Nevertheless, AI allows you to create your assessments in less time, and provide you with more time to do other things in your busy schedules.

C – Additional Resources

We hope these points are useful to you. While AIEd can provide an array of benefits to teaching, learning and assessments, it still requires human input or intervention to ensure validity in your assessment items, and academic integrity in students’ assessment submissions. With continued developments in AI, it will continue to give birth to various innovations and transformations in teaching, learning and assessments. The following resources may help you in capitalizing on the power of AI in your assessments:

1. The 2024 UWI/Guardian Group Teaching Awards and Special Presentation – The AI in Education Era with Professor Mike Sharples

2. The 2024 UWI/Guardian Group Conversations – The AI in Education Era with Professor Mike Sharples

3. For Further training and resources, please see The CETL’s Dates For Your Diary (Training Schedule) AY 202425

4. Assessment for Student Success – An Upcoming CETL Webinar – Registration Link

We hope the aforementioned resources are helpful.
Should you need any further assistance, please feel free to contact us.

Regards,
The Centre for Excellence in Teaching and Learning (CETL),
The University of the West Indies (UWI),
St. Augustine.