Launching a (Popular!) Peer Review Program: Fast, fun and formative

by

“Our intention was to create a formative and fully voluntary process to promote a culture of collegial scholarship.”

Overview

A university can implement a voluntary peer supportive review process with the adoption of open-source materials and simple administrative design. Faculty engagement can be high in programs which are voluntary and formative in nature and can set the stage of peer collaboration before more comprehensive peer-review programs are implemented.

Establishing a Peer-Review Framework

With the expansion of online learning and our growing faculty, MUIH required an expanded and online-friendly peer-review program. While some peer review programs can evoke a groan, our intention was to create a formative and fully voluntary process to promote a culture of collegial scholarship. The voluntary and formative nature of this peer review process assists in creating a refined and mature peer-review process prior to implementing summative or mandatory peer-review evaluation. The initial phase of the peer review process seeks to fulfill three goals:
  • Set a positive, formative process for peer review
  • Utilize a group of voluntary participants to initiate and inform the process
  • Gather data to implement a broader peer review plan
In Peer Review of Teaching: A Sourcebook (2007), Nancy Chism discusses the difference between formative and summative evaluation of teaching as follows: “Within the context of teacher evaluation, the term formative evaluation describes activities that provide teachers with information that they can use to improve their teaching. The information is intended for their personal use, rather than for public inspection, and thus is private and confidential. The information should be rich in detail so that teachers can obtain clear insights in the nature of their teaching strengths and weaknesses. Often, text comments or a multitude of very specific rating items tied to course goals and practices will be employed to provide this… Formative evaluation is informal, ongoing, and wide-ranging. It is the basis for the development of effective teaching throughout one’s career. In contrast, summative evaluation of teaching focuses on information needed for a personnel decision – for example, hiring, promotion, tenure, merit pay. Consequently, the information is for public inspection rather than for the individual faculty member. Since it is not intended to provide rich and detailed data for the improvement of teaching, it is often more general and comparative in nature than data for formative evaluation… The information should provide comparative information as well, enabling the evaluator to determine the quality of the teaching performance with respect to the performance of other peers… The attempt is to judge merit or worth to the institution generally. Summative evaluation, in contrast to formative evaluation, is conducted at given intervals, such as annual or promotion and tenure reviews.” [p5] With limited faculty resources and a desire for speedy implementation we created a simple, yet highly effective, peer review pilot based around open access materials generously provided by Penn State University. This process can easily and effectively be reproduced at other universities. The materials provided by Penn State are designed to be modified for your institutional application (with permission, which was simple to acquire). At our university, there are many faculty who teach courses they have not designed or created, so it was important to assess the teaching of the course and not specifically the design of the course. We modified the forms to this effect and created a modified version for our face-to-face courses.

The Peer-Review Process

Our process as the faculty development and assessment committee to launch the peer review program consisted of the following:
  1. Review, revise and customize the forms provided by Penn State University: http://facdev.e-education.psu.edu/evaluate-revise/peerreviewonline.
  2. Create an online course portal where we offered the following:
    • an overview of the program
    • a flow chart of the process (for both reviewer and instructor)
    • an instructor input form to gather information on their course (which is then provided to the reviewer)
    • extensive instructions for the reviewer and the instructor, both written and in video form e. the forms and resources necessary to complete the review
  3. A category of “peer reviewer” was added to our Student Information System to allow reviewers full and unrestricted access to the course for the purposes of review.
  4. A communication was sent to all faculty which explained the process and included a link to a survey to ask if they were interested in participating, in what role, and if they were teaching online or F2F course.
  5. Instructors were able to choose their course reviewed in this voluntary program. An instructor form was completed with course details and sent to the reviewer.
  6. Review is conducted.
  7. A report is created by the reviewer and presented to the instructor. In this fully voluntary and formative process the instructor is not required to share the results of this peer review although they may choose to reflect upon it in their annual review.
  8. A survey of all participants is created and implemented.
  9. Ongoing revision and evaluation as well as the development of a more summative, required peer review program.

Summary

Our initial two trimesters of the program included instructors from each of our departments as well as a majority of our ranked faculty body. Faculty cited the collaborative and positive peer experience as appealing, the ease of participating (including no required synchronous activities) and many faculty chose to engage without concern of the results being provided as part of their annual formal evaluation. Our intention as a Faculty Development and Assessment team was to create a culture of peer support and collaborative growth prior to creating a more summative and required format. As was reported in our evaluations, faculty were overwhelmingly positive about the experience and many participated in sequential trimesters.

References

Taylor, A. (Ed.). (2016). Faculty peer review of online teaching. Retrieved from http://facdev.e-education.psu.edu/evaluate-revise/peerreviewonline Van Note Chism, N., & Chism, G. W. (2007). Peer review of teaching: a sourcebook (2nd ed.). Bolton, MA: Anker.