Tag: online learning

Posted on by Addie

Unlocking the Power of Quiz Analytics for Smarter Assessments (Issue 35)

Computer screens displaying various data analytics graphs

Author: Dr. Jo Ann Smith, University of Central Florida 

Editor: Dr. Denise Lowe, University of Central Florida

Dear ADDIE, 

I’ve been using quizzes in my online course to assess student learning, but I’m not sure how to make the most of the data provided by the LMS quiz analytics tools. I want to ensure my quizzes are effective and truly reflective of student understanding. What should I look for in the data, and how can I use this information to improve my quizzes? 

Signed,

Quizzer in Quandary 

The goal of data analytics is to enhance the quality of quiz items and support an overall assessment strategy.

Dear Quizzer, 

What a great and relevant question in today’s data-driven landscape! Learning Management Systems (LMS) like Canvas, Blackboard, and Moodle offer quiz analytics tools based provide you with useful information about student learning and engagement. By understanding the types of analytics available and how to interpret them, you can enhance the effectiveness of your quizzes and ensure assessments are providing you with meaningful information about your students’ learning (Brown & Race, 2012; Deetjen-Ruiz et al. 2023; Kumar, et al 2021; Gamage, et al, 2019). Let’s break down the key types of analytics commonly provided by LMS quiz tools and discuss how you can leverage these insights for quiz improvement. 

Using Quiz Analytics to Refine Assessments 

When analyzing quiz data, there are several key statistics that can guide your decision-making and provide valuable insights into student performance and question effectiveness. The goal is to use this data to enhance both the quality of your quiz items and support the overall assessment strategy. Below is a table outlining five common types of analytics provided by LMS quiz tools and how to use these insights to improve your quiz items. Each data type is paired with an example to illustrate how it can inform specific actions for you to take to refine assessments. 

Data Type Applications table with item analysis, description, improvement applications, and examples provided.
Data Type Applications TableDownload

By analyzing these types of data, you can refine your quizzes to ensure that each question is clear, fair, and aligned with your student learning objectives. This iterative process of reviewing and revising assessments based on data helps create more effective learning experiences and ensures that quizzes serve as meaningful tools for both learning and evaluation. 

You could also experiment with adaptive tests that adjusts question difficulty based on student performance. Heat map quiz analytics are also a good visual to quickly identify question items where rows represent a question, and each column represents a student. A specific question might show mostly green colors except for a few patches of red, suggesting that while most students answered quickly and correctly, a few took longer and answered incorrectly. This visual clue could prompt a review of that question to determine if it was unclear or too difficult for those students. 

I also like to use data from open-ended responses to understand the depth of student understanding beyond what multiple-choice questions can provide. Additionally, you could explore the impact of quiz timing (e.g., open-book vs. closed-book quizzes) and provide immediate feedback to further enhance student learning and engagement. 

We’d love to hear your thoughts on innovative uses of LMS quiz tools to enhance learning outcomes. What other ideas or plans for the use of data analytics have you applied or are exploring at your higher education institution? Please share your thoughts with our TOPkit community on LinkedIn!

References 

Brown, S., & Race, P. (2012). Using Effective Assessment to Drive Student Learning. Higher Education Academy. 

Deetjen-Ruiz, R., Esponda-Pérez, J. A., Haris, I., García, D. S., Osorio, J. L. Q., & Tsarev, R. (2023). Evaluating the Reliability of Tests Used in LMS Moodle for E-Learning. In Proceedings of the Computational Methods in Systems and Software (pp. 1-8). Cham: Springer Nature Switzerland. 

Kumar, D., Jaipurkar, R., Shekhar, A., Sikri, G., & Srinivas, V. (2021). Item analysis of multiple choice questions: A quality assurance test for an assessment tool. Medical Journal Armed Forces India, 77, S85-S89. 

Gamage, S.H.P.W., Ayres, J.R., Behrend, M.B. et al. (2019). Optimising Moodle quizzes for online assessments. IJ STEM Ed 6, 27 

Posted on by Addie

Multi-modal Learning Offerings and Expectations (Issue 33)

"Variety" reflected by many different paintbrushes

Author: Charlotte Jones-Roberts, University of Central Florida

Editor: Dr. Denise Lowe, University of Central Florida

Dear ADDIE,

As an online faculty member in higher education, I find myself grappling with the multitude of course modalities available, from fully online to blended to hyflex. Each modality seems to come with its own set of challenges and advantages. How can I navigate these different modalities effectively to ensure the best learning experience for my students while also managing my workload as an instructor? Any guidance would be greatly appreciated.

Sincerely,

Mixed-up Modalities

Dear Mixed-up,

Navigating the ever-changing world of online learning can feel like trying to find your way through a maze, especially with the plethora of course options available. But fear not! You’re not alone in this journey.

Thanks to the unexpected shift brought on by COVID-19, online learning has become more prevalent than ever before. This has opened up a world of possibilities, but it’s also introduced its fair share of challenges. One thing that’s become clear is the need for flexibility to meet the diverse needs of today’s students – and there are many modalities to choose from.

Students prefer a mix of learning experiences for availability, convenience, and content suitability.

According to a 2023 report by Garrett et. al., face-to-face enrollment for traditional undergraduates is either stagnant or declining, with 57% of chief online officers (COOs) reporting stagnation and 24% reporting declines. In contrast, online and hybrid program enrollments are on the rise, with 36% and 20% of COOs reporting growth, respectively. To meet the growing demand for online and hybrid programs, institutions are swiftly realigning their strategic priorities, with approximately 50% of COOs confirming support for greater emphasis on online and multi-modal learning, though resource constraints remain a challenge, and 36% are currently reconsidering their strategic priorities (Garrett et. al., 2023).

Institutions are now offering a smorgasbord of options, ranging from fully online courses to traditional face-to-face instruction, and everything in between. Students increasingly prefer a mix of classroom, online, and hybrid learning experiences due to factors like availability, convenience, and suitability for the content (Garrett et. al., 2023). 

This trend results in most students, both at the undergraduate and graduate levels, encountering various delivery modes throughout their academic journey, including the innovative Hyflex model championed by Beatty (2019). This approach gives students the freedom to choose whether they want to attend class in person or participate remotely, giving them the flexibility they crave in their busy lives. The beauty of the Hyflex model lies in its ability to seamlessly blend the best of both worlds. By incorporating a mix of synchronous and asynchronous elements, instructors can create a dynamic learning environment that caters to the needs of all students, no matter where they are.

For instance, in a hyflex biology course, students could choose to participate in lab experiments physically on campus or virtually via live-streamed sessions. Assignments, discussions, and assessments would be accessible and identical for both in-person and remote learners, ensuring equitable participation and learning outcomes. It’s all about giving you options and making sure everyone’s on the same page, whether you’re in the classroom or chilling at home in your PJs.

The key is keeping clear expectations and communication with students. This includes which modality has been selected, what that modality means at your institution, expectations for participation, and guidelines for accessing course materials in both face-to-face and online environments to ensure that students understand what is required of them regardless of the mode of instruction they choose.

So as you embark on this adventure, remember to keep an open mind and embrace the opportunities that come your way. With a little bit of creativity and a whole lot of flexibility, you’ll be sure to create engaging and inclusive learning experiences for your students.

Happy navigating!

ADDIE

References

Beatty, B. J. (2019). Hybrid-Flexible Course Design (1st ed.). EdTech Books. https://dx.doi.org/10.59668/33

Garrett, R., Simunich, B., Legon, R., & Fredericksen, E. E. (2023). CHLOE 8: Student Demand Moves Higher Ed Toward a Multi-Modal Future, The Changing Landscape of Online Education. Quality Matters and Encoura Eduventures Research, 15. Retrieved from: https://qualitymatters.org/qa-resources/resource-center/articles-resources/CHLOE-8-report-2023

Posted on by Addie

Implementation of AI in Online Teaching and Learning (Issue 29)

AI-generated woman sitting with computer and robot

Author: Dr. Jann Sutton, University of North Florida

Editor: Dr. Denise Lowe, University of Central Florida

Dear ADDIE,

I am concerned! My colleagues are talking about the educational impact of Artificial Intelligence (AI) tools like ChatGPT. I see news articles pop up and am wondering how AI will influence the instructor and instructional designer roles. Some of my faculty are really freaking out! I really don’t know where to start.

Help! AI is melting my brain!

Signed,

OA-Organic Acumen

Dear OA,

The speed of technological change is enough to melt anyone’s brain, I feel for you! However, like any new technological application, we need to approach it methodically and thoughtfully.

Take a deep breath dear reader, as artificial intelligence in the educational setting has been around for decades. It is embedded in adaptive learning, analytics, grading systems, plagiarism checkers, and in our chat messaging systems. AI is typically considered a computer system that has been developed (trained) over time to aid our cognitive capacity. AI can be a valid time-management aid!

Take a deep breath...artificial intelligence in the educational setting has been around for decades.

Recently, some of the more popular AI tools like ChatGPT (text generator),  Dall-E2 (graphics generator), Pictory (video/narrator generator, editor), Grammarly (writing assistant), and Otter.ai (audio/video transcription) have become hot topics for our community. Educators are concerned about how AI will be used to derail students’ learning and influence their ability to write and create. Of course, this is a real concern, but let’s consider whether we can approach the proliferation of AI options as a teaching moment.

Let’s explore how AI, specifically ChatGPT, can be harnessed as a teaching and learning tool. Review the following suggestions which provide entry points to investigate text generators.

  1. Experiment with ChatGPT, make an account, input (ask) questions, and define your parameters. Start by asking simple questions and evaluate the responses – are they accurate, incomplete, or completely inaccurate? During one session, continue to input more complex scenarios and questions. See how some instructors are using ChatGPT to create discussion prompts and lesson plans!
  2. Create assignments utilizing ChatGPT. For example, in a multi-part assignment/project, students can use the tool to draft a short paper, solve an equation/problem, or write a discussion post about a course-related topic of interest. Part of the assignment should be to compare the tool’s responses to their course texts. Consider asking them to keep a reflective journal of their experience and their AI prompts. In a follow-up discussion, students can collectively discuss their experiences using an AI tool: Is it ethical to use, did it help them gain a broader understanding of the topic, and was it accurate? Of course, the instructor needs to be THE guiding voice through this process and carefully review the outputs to help students discern the AI results.
  3. Harness strategies to develop authentic assessments which will make it difficult for a text generator to respond if you or your instructors are not ready to embed the tool in pedagogical activities. Focus on recent local events specific to your course topic which require critical thinking and analysis that cannot be easily replicated by a text generator.
  4. Reflect on your current AI policy and help instructors draft a statement for their syllabi and/or assignment instructions. Can students use it when preparing an outline, but not a final draft? Define your expectations and make them explicit.

Further exploration might include “talking” directly to ChatGPT, asking it how you can incorporate AI into your specific course, and experimenting with how to evaluate its use. Get to know the AI tools, their strengths, and their limitations.

AI-generated conversation with instructor

To give you an idea of how this could work for a research or discussion-related assignment, this image is a screen shot of a discussion between myself and ChatGPT.

As you can see, the ChatGPT AI system provides ideas based on the specificity of my queries. The more specific you are, the better responses are generated by the software. The AI-generated results might also provide you with new ideas to think about, enhancing the depth and breadth of your creativity and research. This is how it works with the graphic image AI software as well – specificity can bring awesome results!

As with all technological advances, there are potential concerns regarding AI, such as privacy issues, equity considerations, and resource allocations. While these concerns should be taken seriously, this article focuses primarily on the positive applications of AI in education.

Personally, I see ChatGPT as a type of Google search engine on steroids. It is powerful, but only as powerful as my own discerning capabilities. A search engine provides resources that are not created equal; similarly, I need to review the AI output and then decide on my own how I will use the results.

What other ideas or plans for the use of AI have you applied or are exploring at your higher education institution? Please share your thoughts with our TOPkit community on LinkedIn!

References

Guan, C., Mou, J., & Jiang, Z. (2020). Artificial intelligence innovation in education: A twenty-year data-driven historical analysis. International Journal of Innovation Studies, 4(4), 134-147.

EDUCAUSE Learning Initiative. (2017). Seven things you should know about artificial intelligence in teaching and learning. EDUCAUSE. Retrieved April 20, 2023, from https://library.educause.edu/-/media/files/library/2017/4/eli7143.pdf.

McMurtrie, B. (2023). What you can learn from students about CHATGPT. The Chronicle of Higher Education.

Rose, R. (2023). ChatGPT in Higher Education: Artificial Intelligence and Higher Education. University of North Florida Digital Pressbooks.

University Center for Teaching and Learning. (nd). ChatGPT resources for faculty. University of Pittsburgh. Retrieved April 24, 2023, from https://teaching.pitt.edu/resources/chatgpt-resources-for-faculty/