by Sue Bauer,
Overview
This section covers checklists, rubrics, and guidelines for evaluating online programs and course quality. The Selection/Application/Implementation Guide and Resources/Examples may be used to create a customized program and course evaluation process for an institution’s specific needs. These artifacts may be used in a variety of contexts such as self-assessment, instructional designer review, peer review, institutional online program effectiveness, and accreditation standards.
- Identify fundamental, critical, and essential elements for your institution’s online courses, programs, and institutional initiatives.
- Use samples and resources to identify factors and elements related to the importance of your evaluation for high quality online course development and delivery.
- List, categorize, and prioritize developed guidelines into personalized checklists, rubrics, or guidelines for your institutional needs, requirements, and implementation.
Note: If you are interested in more information on program evaluation and effectiveness, visit the Evaluation page where you will find more detailed information, tips and resources.
Selection/Application/Implementation Guide
This section presents recommendations that may be utilized when analyzing, selecting, and applying criteria to the institution’s assessment method for effective online programs, course requirements, and sometimes to meet accreditation standards. How will “quality” and “success” in online learning be defined at your organization? The first recommendation is to start at a higher level of prioritizing needs. What national or institutional needs must be met for your institution? This could include matters such as copyright, Family Educational Rights and Privacy Act (FERPA), and online accessibility protocols. Along with institutional criteria, departments and programs may also have specific criteria for online course content. Program criteria is a much more specialized area to develop and although some patterns may emerge, particular criteria may be highlighted and applied to specific courses and programs.
The overall process of analyzing and selecting criteria for online course/course program development can be narrowed down to the determination of specific course and content requirements. All courses will generally include a syllabus and content for delivery.
Elements deemed critical for these areas should also be listed and implanted in the evaluation piece. Once the above levels of criterion have been considered and listed, it is recommended that the criteria be categorized and prioritized for your application.
Considerations for Implementing Rubrics, Checklists, and Guidelines
Although all of the listed criterion are important and should be highlighted, prioritizing can assist faculty in their process for online content development. Once there is a comprehensive list of institution criteria, it is time to develop the rubrics, checklists, or guidelines. Consider how and when the criteria will be used. It is very probable that more than one of these rubrics, checklists, or guidelines will be created to meet your development needs. Ask yourself the following for context and usage:
- Will these rubrics, checklists, and guidelines be used to assess courses during the delivery of your professional development course?
- Will they be used for self-assessment, an instructional designer review, a peer review activity, or all of the above?
- Will you develop the rubrics, checklists, and guidelines as formal assessments that will prohibit or enhance an instructor’s ability to teach online?
- Will they be used more informally as self-assessment or peer review activities?
- Will elements be included to assess the course’s effectiveness after the delivery of the course online?
Although this section may seem an overwhelming list of considerations, all are important in the development of your criterion for online course effectiveness. It is critical/imperative to decide what is important at your institution and how it will be clarified to instructors in your professional development course.
Resources/Examples
This section provides high quality examples of rubrics, checklists, and guidelines used to assess online programs and course evaluation/delivery standards. These resources were developed and utilized by other institutions. It is suggested that you reference these items, as appropriate, while you develop your own list of standards. They may also be considered “as-is” for adoption at your institution.
| Program Evaluation Methods | |
|---|---|
| Planning Guide for Online and Blended Learning | This document is designed to serve educational leaders in Michigan who are tasked with integrating online and blended learning planning into the larger school improvement planning effort. Written as a practical guide, this document is organized into narrative sections that present key focus areas for district and building leaders. |
| OLC Quality Scorecard | Criteria for Excellence in the Administration of Online Programs |
| The School Technology Needs Assessment | An assessment intended to help school-level decision makers—administrators, technology facilitators, media coordinators, or technology committee members—collect data to plan and improve uses of technology in teaching and learning activities. |
| Distance Education Programs Guidelines | Guidelines developed to assist institutions in planning distance education and to provide an assessment framework for institutions already involved in distance education. |
| National Standards for Quality Online Learning | This document, the International Association for K-12 Online Learning’s (iNACOL) National Standards for Quality Online Programs, is the third of iNACOL’s online education standards, following the National Standards of Quality for Online Courses and National Standards for Quality Online Teaching. |
| Online Learning at Duke: A Planning Guide | An overview of planning for an online program delivered at Duke University. |
| Virtual Learning Program Rubric | A rubric created by the Center on Innovations in Learning (CIL). A rubric for evaluating virtual programs to ensure high quality and rigor in virtual programs within a school’s curriculum. |
| Course Evaluation Methods | |
|---|---|
| Florida Online Course Design Quality Designations | With the aim of increasing the number of quality online courses in Florida, the Board of Governors Quality Workgroup provided guidance for an online course design quality review processes flexible enough to span contexts of all State University System and Florida College System higher institutions. Online courses vetted through an online course design quality review will be designated as "Quality" in the FloridaShines course catalog. Reviews can result in a "Quality" or "High-Quality" designation based on the Quality Matters (QM) Higher Ed with a process unique to Florida. The Florida initiative uses QM as a default but also, through an “Institutional Equivalency Process,” allows individual institutions to have their own institutional standards and processes approved for use instead. The Florida State University System, through its 2025 Strategic Plan for Online Education, has called for a state-wide initiative for recognizing “quality” and “high quality” online courses. |
| Open SUNY Course Quality Review | An open rubric from SUNY for assessing quality in an online course. |
| Online Course Standards Rubric | The Online Course Standards Rubric provides a framework for online course design based on University of New Mexico's expectations for quality online courses. |
| Course Design Rubric Standards | A national non-profit organization dedicated to quality assurance in online education. This site offers resources such as QM rubrics for fully online or hybrid and blended courses, course design based on QM standards, peer review, workshops, and conferences. |
| 5 Pillars Quality Review | Developed by the University of Missouri, The 5 pillars quality review page can help identify key components in a course that are working well, may be missing, or may need improvement. |
| University of Central Florida/Educause Blended Course Implementation Checklist | This checklist seeks to assure quality in a blended course. |
| University of Florida Standards and Markers of Excellence Document | UF has developed a set of standards that should be met by all online courses. The ones marked with an “e” are the Markers of Excellence, which are above and beyond the required standard. |
| SUNY Online Course Quality Review Rubric | To help campuses ensure that their online courses are learner centered and well designed, a team of SUNY staff and campus stakeholders has designed the OSCQR rubric, a customizable and flexible tool for online course quality review |
| University of Central Florida Course Rubric Checklist | Informal rubric IDL6543 participants utilize to self-assess themselves and other participants. |
| UMass Darmouth Quality Blended Course Rubric | UMass Darmouth's goal of designing and/or reviewing blended courses with this rubric is to ensure an effective learning experience for all blended learners. This rubric is based on the Open SUNY Course Quality Review (OSCQR) rubric. |
| OLC Quality Course Teaching and Instructional Practice (QCTIP) | Designed to evaluate the overall classroom experience, this tool can be used to evaluate teaching effectiveness in several areas, including: Course Fundamentals, Learning Foundations, Faculty Engagement and Student Engagement. Note: Access to this rubric may require OLC Membership. |
| UCF Quality Course Review UCF High Quality Course Review | In support of Florida’s state-wide quality initiative, UCF’s Center for Distributed Learning developed a tiered system for reviewing courses with 2 designations: Quality and High Quality. The standards align with Quality Matters (QM) and additional standards have been added to address UCF’s specific needs. |
Sample Course
The TOPkit sample courses have additional information regarding the rubrics used to assess courses and modules during a peer-review activity included in the courses.
Recommended Articles
Beyond the above list of resources and examples, listed below are addition readings that may assist in the development of your campus resources.
- Brown, V., Lewis, D., & Toussaint, M. (2018).
- Students’ Perceptions of Quality Across Four Course Development Models.
- Online Learning, 22(2).
- Thompson, K. (2016).
- Lee, C., Dickerson, J., Winslow, J. (Spring 2012).
- An Analysis of Organizational Approaches to Online Course Structures.
- Online Journal of Distance Learning Administration, Volume 15, n1.
- McGahan, S., Jackson, C., Premer, K. (December 2015).
- Online Course Quality Assurance: Development of a Quality Checklist.
- InSight: A Journal of Scholarly Teaching, Volume 1, p126-140.
- Office of Educational Technology. (November 2014).
- Shelton, K. (2010).
- A Quality Scorecard for the Administration of Online Education Programs: A Delphi Study.
- Journal of Asynchronous Learning Networks, Volume 14: Issue 4, p 36-62.
- The Institute of Higher Education Policy. (April 2000).
References
Bauer, S. & Sugar, A. (2012 April). Professional Development for Preparing Online Faculty. Conference Presenter at USDLA 2012 NATIONAL CONFERENCE, Hilton St. Louis, St. Louis, MO. Retrieved from https://sites.google.com/site/idl6543redesign/
Michigan Virtual University (2012). Planning Guide for Online and Blended Learning. Retrieved from https://micourses.org/resources/pdf/toolkit/MVU_RPT_PlanningGuide.pdf
Sue is an instructional designer with the Center for Distributed Learning at the University of Central Florida. Sue collaborates with faculty to deliver professional development opportunities, apply best practices in course development, and offer consultations and instructional design guidance. Sue’s special interest areas are project management for instructional designers in higher education, mobile teaching/learning, and MOOCs.