Author: Dr. Jo Ann Smith, University of Central Florida
Editor: Dr. Denise Lowe, University of Central Florida
Dear ADDIE,
I’ve been using quizzes in my online course to assess student learning, but I’m not sure how to make the most of the data provided by the LMS quiz analytics tools. I want to ensure my quizzes are effective and truly reflective of student understanding. What should I look for in the data, and how can I use this information to improve my quizzes?
Signed,
Quizzer in Quandary
Dear Quizzer,
What a great and relevant question in today’s data-driven landscape! Learning Management Systems (LMS) like Canvas, Blackboard, and Moodle offer quiz analytics tools based provide you with useful information about student learning and engagement. By understanding the types of analytics available and how to interpret them, you can enhance the effectiveness of your quizzes and ensure assessments are providing you with meaningful information about your students’ learning (Brown & Race, 2012; Deetjen-Ruiz et al. 2023; Kumar, et al 2021; Gamage, et al, 2019). Let’s break down the key types of analytics commonly provided by LMS quiz tools and discuss how you can leverage these insights for quiz improvement.
Using Quiz Analytics to Refine Assessments
When analyzing quiz data, there are several key statistics that can guide your decision-making and provide valuable insights into student performance and question effectiveness. The goal is to use this data to enhance both the quality of your quiz items and support the overall assessment strategy. Below is a table outlining five common types of analytics provided by LMS quiz tools and how to use these insights to improve your quiz items. Each data type is paired with an example to illustrate how it can inform specific actions for you to take to refine assessments.
By analyzing these types of data, you can refine your quizzes to ensure that each question is clear, fair, and aligned with your student learning objectives. This iterative process of reviewing and revising assessments based on data helps create more effective learning experiences and ensures that quizzes serve as meaningful tools for both learning and evaluation.
You could also experiment with adaptive tests that adjusts question difficulty based on student performance. Heat map quiz analytics are also a good visual to quickly identify question items where rows represent a question, and each column represents a student. A specific question might show mostly green colors except for a few patches of red, suggesting that while most students answered quickly and correctly, a few took longer and answered incorrectly. This visual clue could prompt a review of that question to determine if it was unclear or too difficult for those students.
I also like to use data from open-ended responses to understand the depth of student understanding beyond what multiple-choice questions can provide. Additionally, you could explore the impact of quiz timing (e.g., open-book vs. closed-book quizzes) and provide immediate feedback to further enhance student learning and engagement.
We’d love to hear your thoughts on innovative uses of LMS quiz tools to enhance learning outcomes. What other ideas or plans for the use of data analytics have you applied or are exploring at your higher education institution? Please share your thoughts with our TOPkit community on LinkedIn!
References
Brown, S., & Race, P. (2012). Using Effective Assessment to Drive Student Learning. Higher Education Academy.
Deetjen-Ruiz, R., Esponda-Pérez, J. A., Haris, I., García, D. S., Osorio, J. L. Q., & Tsarev, R. (2023). Evaluating the Reliability of Tests Used in LMS Moodle for E-Learning. In Proceedings of the Computational Methods in Systems and Software (pp. 1-8). Cham: Springer Nature Switzerland.
Kumar, D., Jaipurkar, R., Shekhar, A., Sikri, G., & Srinivas, V. (2021). Item analysis of multiple choice questions: A quality assurance test for an assessment tool. Medical Journal Armed Forces India, 77, S85-S89.
Gamage, S.H.P.W., Ayres, J.R., Behrend, M.B. et al. (2019). Optimising Moodle quizzes for online assessments. IJ STEM Ed 6, 27