higher education Archives

Author: Dr. Jo Ann Smith, University of Central Florida

Editor: Dr. Denise Lowe, University of Central Florida

Dear ADDIE,

I’ve been using quizzes in my online course to assess student learning, but I’m not sure how to make the most of the data provided by the LMS quiz analytics tools. I want to ensure my quizzes are effective and truly reflective of student understanding. What should I look for in the data, and how can I use this information to improve my quizzes?

Signed,

Quizzer in Quandary

The goal of data analytics is to enhance the quality of quiz items and support an overall assessment strategy.

Dear Quizzer,

What a great and relevant question in today’s data-driven landscape! Learning Management Systems (LMS) like Canvas, Blackboard, and Moodle offer quiz analytics tools based provide you with useful information about student learning and engagement. By understanding the types of analytics available and how to interpret them, you can enhance the effectiveness of your quizzes and ensure assessments are providing you with meaningful information about your students’ learning (Brown & Race, 2012; Deetjen-Ruiz et al. 2023; Kumar, et al 2021; Gamage, et al, 2019). Let’s break down the key types of analytics commonly provided by LMS quiz tools and discuss how you can leverage these insights for quiz improvement.

Using Quiz Analytics to Refine Assessments

When analyzing quiz data, there are several key statistics that can guide your decision-making and provide valuable insights into student performance and question effectiveness. The goal is to use this data to enhance both the quality of your quiz items and support the overall assessment strategy. Below is a table outlining five common types of analytics provided by LMS quiz tools and how to use these insights to improve your quiz items. Each data type is paired with an example to illustrate how it can inform specific actions for you to take to refine assessments.

Data Type Applications table with item analysis, description, improvement applications, and examples provided.

Data Type Applications Table Download

By analyzing these types of data, you can refine your quizzes to ensure that each question is clear, fair, and aligned with your student learning objectives. This iterative process of reviewing and revising assessments based on data helps create more effective learning experiences and ensures that quizzes serve as meaningful tools for both learning and evaluation.

You could also experiment with adaptive tests that adjusts question difficulty based on student performance. Heat map quiz analytics are also a good visual to quickly identify question items where rows represent a question, and each column represents a student. A specific question might show mostly green colors except for a few patches of red, suggesting that while most students answered quickly and correctly, a few took longer and answered incorrectly. This visual clue could prompt a review of that question to determine if it was unclear or too difficult for those students.

I also like to use data from open-ended responses to understand the depth of student understanding beyond what multiple-choice questions can provide. Additionally, you could explore the impact of quiz timing (e.g., open-book vs. closed-book quizzes) and provide immediate feedback to further enhance student learning and engagement.

We’d love to hear your thoughts on innovative uses of LMS quiz tools to enhance learning outcomes. What other ideas or plans for the use of data analytics have you applied or are exploring at your higher education institution? Please share your thoughts with our TOPkit community on LinkedIn!

References

Brown, S., & Race, P. (2012). Using Effective Assessment to Drive Student Learning. Higher Education Academy.

Deetjen-Ruiz, R., Esponda-Pérez, J. A., Haris, I., García, D. S., Osorio, J. L. Q., & Tsarev, R. (2023). Evaluating the Reliability of Tests Used in LMS Moodle for E-Learning. In Proceedings of the Computational Methods in Systems and Software (pp. 1-8). Cham: Springer Nature Switzerland.

Kumar, D., Jaipurkar, R., Shekhar, A., Sikri, G., & Srinivas, V. (2021). Item analysis of multiple choice questions: A quality assurance test for an assessment tool. Medical Journal Armed Forces India, 77, S85-S89.

Gamage, S.H.P.W., Ayres, J.R., Behrend, M.B. et al. (2019). Optimising Moodle quizzes for online assessments. IJ STEM Ed 6, 27

Author: Charlotte Jones-Roberts, University of Central Florida

Editor: Dr. Denise Lowe, University of Central Florida

Dear ADDIE,

As an online faculty member in higher education, I find myself grappling with the multitude of course modalities available, from fully online to blended to hyflex. Each modality seems to come with its own set of challenges and advantages. How can I navigate these different modalities effectively to ensure the best learning experience for my students while also managing my workload as an instructor? Any guidance would be greatly appreciated.

Sincerely,

Mixed-up Modalities

Dear Mixed-up,

Navigating the ever-changing world of online learning can feel like trying to find your way through a maze, especially with the plethora of course options available. But fear not! You’re not alone in this journey.

Thanks to the unexpected shift brought on by COVID-19, online learning has become more prevalent than ever before. This has opened up a world of possibilities, but it’s also introduced its fair share of challenges. One thing that’s become clear is the need for flexibility to meet the diverse needs of today’s students – and there are many modalities to choose from.

Students prefer a mix of learning experiences for availability, convenience, and content suitability.

According to a 2023 report by Garrett et. al., face-to-face enrollment for traditional undergraduates is either stagnant or declining, with 57% of chief online officers (COOs) reporting stagnation and 24% reporting declines. In contrast, online and hybrid program enrollments are on the rise, with 36% and 20% of COOs reporting growth, respectively. To meet the growing demand for online and hybrid programs, institutions are swiftly realigning their strategic priorities, with approximately 50% of COOs confirming support for greater emphasis on online and multi-modal learning, though resource constraints remain a challenge, and 36% are currently reconsidering their strategic priorities (Garrett et. al., 2023).

Institutions are now offering a smorgasbord of options, ranging from fully online courses to traditional face-to-face instruction, and everything in between. Students increasingly prefer a mix of classroom, online, and hybrid learning experiences due to factors like availability, convenience, and suitability for the content (Garrett et. al., 2023).

This trend results in most students, both at the undergraduate and graduate levels, encountering various delivery modes throughout their academic journey, including the innovative Hyflex model championed by Beatty (2019). This approach gives students the freedom to choose whether they want to attend class in person or participate remotely, giving them the flexibility they crave in their busy lives. The beauty of the Hyflex model lies in its ability to seamlessly blend the best of both worlds. By incorporating a mix of synchronous and asynchronous elements, instructors can create a dynamic learning environment that caters to the needs of all students, no matter where they are.

For instance, in a hyflex biology course, students could choose to participate in lab experiments physically on campus or virtually via live-streamed sessions. Assignments, discussions, and assessments would be accessible and identical for both in-person and remote learners, ensuring equitable participation and learning outcomes. It’s all about giving you options and making sure everyone’s on the same page, whether you’re in the classroom or chilling at home in your PJs.

The key is keeping clear expectations and communication with students. This includes which modality has been selected, what that modality means at your institution, expectations for participation, and guidelines for accessing course materials in both face-to-face and online environments to ensure that students understand what is required of them regardless of the mode of instruction they choose.

So as you embark on this adventure, remember to keep an open mind and embrace the opportunities that come your way. With a little bit of creativity and a whole lot of flexibility, you’ll be sure to create engaging and inclusive learning experiences for your students.

Happy navigating!

ADDIE

References

Beatty, B. J. (2019). Hybrid-Flexible Course Design (1st ed.). EdTech Books. https://dx.doi.org/10.59668/33

Garrett, R., Simunich, B., Legon, R., & Fredericksen, E. E. (2023). CHLOE 8: Student Demand Moves Higher Ed Toward a Multi-Modal Future, The Changing Landscape of Online Education. Quality Matters and Encoura Eduventures Research, 15. Retrieved from: https://qualitymatters.org/qa-resources/resource-center/articles-resources/CHLOE-8-report-2023

Tag: higher education

Unlocking the Power of Quiz Analytics for Smarter Assessments (Issue 35)