This section covers checklists, rubrics, and guidelines for evaluating online programs and course quality. The Selection/Application/Implementation Guide and Resources/Examples may be used to create a customized program and course evaluation process for an institution’s specific needs. These artifacts may be used in a variety of contexts such as self-assessment, instructional designer review, peer review, institutional online program effectiveness, and accreditation standards.
Note: If you are interested in more information on program evaluation and effectiveness, visit the Evaluation page where you will find more detailed information, tips and resources.
- Identify fundamental, critical, and essential elements for your institution’s online courses, programs, and institutional initiatives.
- Use samples and resources to identify factors and elements related to the importance of your evaluation for high quality online course development and delivery.
- List, categorize, and prioritize developed guidelines into personalized checklists, rubrics, or guidelines for your institutional needs, requirements, and implementation.
This section presents recommendations that may be utilized when analyzing, selecting, and applying criteria to the institution’s assessment method for effective online programs, course requirements, and sometimes to meet accreditation standards. How will “quality” and “success” in online learning be defined at your organization?
The first recommendation is to start at a higher level of prioritizing needs. What national or institutional needs must be met for your institution? This could include matters such as copyright, Family Educational Rights and Privacy Act (FERPA), and online accessibility protocols.
Along with institutional criteria, departments and programs may also have specific criteria for online course content. Program criteria is a much more specialized area to develop and although some patterns may emerge, particular criteria may be highlighted and applied to specific courses and programs.
The overall process of analyzing and selecting criteria for online course/course program development can be narrowed down to the determination of specific course and content requirements. All courses will generally include a syllabus and content for delivery. Elements deemed critical for these areas should also be listed and implanted in the evaluation piece.
Once the above levels of criterion have been considered and listed, it is recommended that the criteria be categorized and prioritized for your application. Although all of the listed criterion are important and should be highlighted, prioritizing can assist faculty in their process for online content development.
Once there is a comprehensive list of institution criteria, it is time to develop the rubrics, checklists, or guidelines. Consider how and when the criteria will be used. It is very probable that more than one of these rubrics, checklists, or guidelines will be created to meet your development needs.
Ask yourself the following for context and usage:
- Will these rubrics, checklists, and guidelines be used to assess courses during the delivery of your professional development course?
- Will they be used for self-assessment, an instructional designer review, a peer review activity, or all of the above?
- Will you develop the rubrics, checklists, and guidelines as formal assessments that will prohibit or enhance an instructor’s ability to teach online?
- Will they be used more informally as self-assessment or peer review activities?
- Will elements be included to assess the course’s effectiveness after the delivery of the course online?
Although this section may seem an overwhelming list of considerations, all are important in the development of your criterion for online course effectiveness. It is critical/imperative to decide what is important at your institution and how it will be clarified to instructors in your professional development course.
This section provides high quality examples of rubrics, checklists, and guidelines used to assess online programs and course evaluation/delivery standards. These resources were developed and utilized by other institutions. It is suggested that you reference these items, as appropriate, while you develop your own list of standards. They may also be considered “as-is” for adoption at your institution.
Program Evaluation Methods
- Planning Guide for Online and Blended Learning – This document is designed to serve educational leaders in Michigan who are tasked with integrating online and blended learning planning into the larger school improvement planning effort. Written as a practical guide, this document is organized into narrative sections that present key focus areas for district and building leaders. (https://micourses.org/resources/pdf/toolkit/MVU_RPT_PlanningGuide.pdf)
- OLC Quality Scorecard: Criteria for Excellence in the Administration of Online Programs.(http://onlinelearningconsortium.org/consult/quality-scorecard/)
- The School Technology Needs Assessment (STNA) is intended to help school-level decision makers—administrators, technology facilitators, media coordinators, or technology committee members—collect data to plan and improve uses of technology in teaching and learning activities. (http://www.serve.org/uploads/docs/STNA3.0.0.pdf)
- Distance Education Programs – Interregional Guidelines for the Evaluation of Distance Education (Online Learning). (https://www.msche.org/publications/Guidelines-for-the-Evaluation-of-Distance-Education-Programs.pdf)
- National Standards for Quality Online Learning – This document, the International Association for K-12 Online Learning’s (iNACOL) National Standards for Quality Online Programs, is the third of iNACOL’s online education standards, following the National Standards of Quality for Online Courses and National Standards for Quality Online Teaching. (http://www.inacol.org/wp-content/uploads/2015/02/national-standards-for-quality-online-programs.pdf)
- Overview of planning process for an online program delivered at Duke University. (https://cit.duke.edu/pdf/onlinelearningguidecombined.pdf)
- Virtual Learning Program Rubric created by the Center on Innovations in Learning (CIL). A rubric for evaluating virtual programs to ensure high quality and rigor in virtual programs within a school’s curriculum.(http://www.centeril.org/resources/2014_10.28_VLPRubric.pdf)
Course Evaluation Methods
- The Florida State University System, through its 2025 Strategic Plan for Online Education, has called for a state-wide initiative for recognizing “quality” and “high quality” online courses. Please note, this is currently a work in progress and resources will be updated as soon as they are available.
- Open SUNY Course Quality Review (OSCQR) Rubric from the Open SUNY Center for Online Teaching Excellence (COTE) – https://docs.google.com/spreadsheets/d/11gZstRZ-uyg9PE8R-FbRwlCIMd7sym0rKYVtPj6crmw/edit#gid=1921399422
- Chico, Rubric for Online Instruction. Developed by California State University, Chico, the use of this rubric represents a developmental process for online course design and delivery, and provides a means for an instructor to self-assess course(s) based on California State University expectations. (https://www.csuchico.edu/tlp/resources/rubric/rubric.pdf)
- Blackboard Exemplary Course Program Rubric – The Exemplary Course Program recognizes instructors and course designers whose courses demonstrate best practices in four major areas: Course Design, Interaction & Collaboration, Assessment, and Learner Support. (http://www.blackboard.com/resources/catalyst-awards/bbexemplarycourserubric_rebrand_july2015.pdf)
- Quality Matters – A national non-profit organization dedicated to quality assurance in online education. This site offers resources such as QM rubrics for fully online or hybrid and blended courses, course design based on QM standards, peer review, workshops, and conferences. (https://www.qualitymatters.org/qa-resources/rubric-standards/higher-ed-rubric)
- The California State University Quality Online Learning and Teaching (QOLT) Instrument. (https://drive.google.com/file/d/0BxN4M6qCVbDPOEl0d1dKWmFXOEk/view)
- UCF/EDUCAUSE BlendKit – Assure Quality in Your Blended Learning Course
- UF Standards and Markers of Excellence Document – UF has developed a set of standards that should be met by all online courses. The ones marked with an “e” are the Markers of Excellence, which are above and beyond the required standard. (http://teach.ufl.edu/wp-content/uploads/2012/08/UF_standards_markers.pdf)
- University of Wisconsin Milwaukee’s Learning Technology Center (LTC) Evaluation Checklist for Online and Blended Courses. (http://uwmltc.org/?p=3813)
- UCF/CDL’s Course and Module Rubric – Informal rubric IDL6543 participants utilize to self-assess themselves and other participants.
- The UW-L Online Course Evaluation Guidelines – The Guidelines outline specific features of online courses and recommend ways to design, deliver and improve courses. (https://www.uwlax.edu/uploadedFiles/Academics/online/guidelines.pdf)
Discuss this Topic
Please join ongoing discussions related to this topic with colleagues in the TOPkit community of practice. This is a great opportunity to ask questions, seek feedback, or share effective practices with an active, global professional community.
Beyond the above list of resources and examples, listed below are addition readings that may assist in the development of your campus resources.
Thompson, K. (2016). EDUCAUSE BlendKit: Quality Assurance in Blended Learning, Second Edition. Accessed July 11, 2016 from https://blended.online.ucf.edu/blendkit-course-blendkit-reader-chapter-5/
Lee, C., Dickerson, J., Winslow, J. (Spring 2012). An Analysis of Organizational Approaches to Online Course Structures. Online Journal of Distance Learning Administration, Volume 15, n1. Retrieved from http://eric.ed.gov/?id=EJ973021
McGahan, S., Jackson, C., Premer, K. (December 2015). Online Course Quality Assurance: Development of a Quality Checklist. InSight: A Journal of Scholarly Teaching, Volume 1, p126-140. Retrieved from https://eric.ed.gov/?q=Online+Course+Quality+Assurance+Development+of+a+Quality+Checklist&id=EJ1074062
Office of Educational Technology. (November 2014). Online Professional Learning Quality Checklist. Retrieved from http://tech.ed.gov/wp-content/uploads/2014/11/Section-5-Online-Professional-Learning-Quality-Checklist-FINAL.pdf
Shelton, K. (2010). A Quality Scorecard for the Administration of Online Education Programs: A Delphi Study. Journal of Asynchronous Learning Networks, Volume 14: Issue 4, p 36-62. Retrieved from http://files.eric.ed.gov/fulltext/EJ909912.pdf
The Institute of Higher Education Policy. (April 2000). Quality On the Line: Benchmarks for Success in Internet-Based Distance Education. Retrieved from https://eric.ed.gov/?id=ED444407
Bauer, S. & Sugar, A. (2012 April). Professional Development for Preparing Online Faculty. Conference Presenter at USDLA 2012 NATIONAL CONFERENCE, Hilton St. Louis, St. Louis, MO. Retrieved from https://sites.google.com/site/idl6543redesign/
Michigan Virtual University (2012). Planning Guide for Online and Blended Learning. Retrieved from https://micourses.org/resources/pdf/toolkit/MVU_RPT_PlanningGuide.pdf
The TOPkit sample courses have additional information regarding the rubrics used to assess courses and modules during a peer-review activity included in the courses.