Conducting a Program Review​ Part Four: Judging Performance and Interpreting Ratings​

After the team has collected materials and examined documentation, employ a process for conducting ratings in each of the component areas of the standard. This stage in the program review involves clarifying rating criteria, facilitating the rating process using evaluative evidence, and negotiating differences in group ratings.

RatingsThe Council for Advancement of Standards in Higher Education (CAS) has developed self-assessment guides for each of the functional area standards as resources to support the rating process. In addition to providing a practical workbook format for the program review team to use, the self-assessment guides also convert the standards into multiple measurable criterion statements, which are broken down into clusters by the standards’ subsections to allow raters to be more detailed and focused in evaluating their judgments.

Ultimately, the activity of judging performance should be a group process, evaluating how well the area under review meets the related CAS standard on a five-point rating scale—does not meet, partly meets, meets, exceeds, or exemplifies.

The program review team might choose to have each member complete their assessments individually, and then meet to compare ratings, discuss differences, and build consensus on a final collective evaluation. They also might choose to have the group collaborate through this process to review criteria, examine materials, and establish the ratings together during team meetings. Unless there are institutional expectations or norms established, the decision on how to best facilitate this group rating process is up to the review team, and there are advantages and disadvantages for each approach. 

If the team feels that adequate information was not available for anitem, note that they may also choose to rate it as “insufficient evidence,” and in rare circumstances, a criterion could be found not applicable to the program.  Through the group discussions in negotiating final ratings, an important tenet to consider is that conducting meaningful program reviews is not about “getting all A’s on the report card.” Self-assessment is a learning process for the organization, and therefore, the review team should be honest, open, and objective when engaging in dialogue about measuring each item. 

In the next issue, the process for creating an action plan based on the completed ratings will be described. 

ACUI members have the access to relevant functional area standards here.
Load more comments
New code