Up in the Air on Faculty Evaluation

George Clooney’s 2009 movie Up in the Air showed us one way in which21^st-century online job assessments are conducted. Employees sit ina room and receive the summative evaluation of their work (in the movie they areall terminations) delivered via Skype by someone whom the employee has nevermet.

Most moviegoers were aghast at the prospectof the evaluation process we saw in the movie, but they considered it a veryreal possibility. Technology is changing everything, from job assessment toonline teaching and learning. And it has empowered new groups of stakeholders.Our research on the online faculty teaching evaluation system at the JohnsHopkins School of Education (SOE) reveals that technology has given us thepolicy opportunity and political forces to create a 21^st-centuryfaculty assessment system. We believe that it is possible, through acollaboration of newly empowered faculty and student stakeholders, to designand implement new assessments that will improve the teaching and learning atSOE and other institutions of higher education.

Organizational change through politics and power

One approach to organizational change isto seize a moment in time when an organizational problem merges with thatorganization’s policy needs and political opportunity for its stakeholders (McLendon,Cohen-Vogel, and Wachen 2015). It is particularlyimportant to seize these strategic moments in higher education, wherebureaucracies and institutional traditions mitigate against rapid change. Theproblem under discussion, SOE’s teaching evaluation system, has emerged due tothe development of SOE’s new online master’s and doctoral programs. The faculty believe that the currentevaluation system does not accurately reflect their online instructionalresponsibilities. At the same time, SOE online students, empowered by theironline relationships with faculty, feel they should have a bigger impact on thedesign of instruction and on the use of student feedback. This merging ofpolicy problems and political uncertainty has led to an organizational moment thatcalls for a policy change for instructional evaluation and, at the same time,to empower SOE faculty and students.

Studentevaluation of teaching policy problem

The continued growth of online educationpresents new technological and pedagogical challenges, among other policyproblems, requiring new thinking about assessing effective instruction. Theincreasing pressure from the accountability movement, the emergence of rapidgrowth in online education, and the current challenges associated with commonlyused student evaluations of teaching (SETs) are forcing higher educationleaders and faculty to change assessments in order to improve university-levelteaching in online courses. Technology also makes it possible for a newstakeholder partnership of faculty and students to address this particularpolicy problem together and to advocate for 21^st-century onlineassessment systems that will have the political support to make themsustainable.

In order to analyze the current SOEinstructor evaluation process and to identify alternative assessment policies,the researchers interviewed school leadership and faculty and surveyed currentstudents in the new online education doctorate (EdD) program. Like over 90percent of North American higher education institutions, SOE relies on SETs.Our findings show that SOE faculty still debate the merits of the studentevaluation of teaching, often not using this data to inform their teachingpractices. One reason for the lack of faculty support is that online instruction,like face-to-face, remains of secondary importance to many university faculty.Our results showed that even those School of Education faculty on a clinicaltrack (instructional), as opposed to the research track, continue to undervalueteaching as a promotional tool. Further, regardless of the SET survey data theysee, SOE faculty are not very inclined to improve their teaching methods.

Schoolof Education student evaluation of teaching

The SOE currently uses a process forevaluating research and clinical facultyteaching effectiveness that includes the individual development and educationalassessment (IDEA) student survey and an EdD program-administered teacherevaluation survey. To administrative and faculty leadership, the steps of theevaluation process were clear. However, the data suggested that the use andvalue of the survey findings were not clear to faculty or students.

From those faculty interviewed, we learnedthat a key problem for the personalized student feedback is that the SET is of varyingimportance to the faculty. Many voiced frustration, saying that leadershipuses the survey findings when ratings are bad and ignores them when ratings aregood. SOE leadership indicated that it is still possible for faculty to advancethrough the promotional process, even with average or marginal teachingratings. Research faculty suggested that evaluation survey findings increase inimportance directly preceding or during the promotion year, while clinicalfaculty reported that as long as the ratings stay above the minimum requirement,no action is taken or required. Furthermore, one semester of poor adjunctteacher evaluation data at SOE typically signals the end of an adjunct’s job.Full-time faculty who receive poor ratings, defined as less than average, areplaced on a performance improvement plan.

SOE leadership also acknowledged that facultyteaching evaluations can be a source of dissatisfaction among faculty,particularly among the faculty who identify as researchers who teach. Manyof these faculty members believe that research is most important in thepromotional process, and they strive to minimize their teaching time. Those researcherswho teach that we interviewed reasoned that the small sample size ofstudent respondents to SET “reflects the small minority that are unhappy” (Borkoskiand Kobett 2015); they maintain that happy students do not submit evaluations.Despite their skepticism of potential rewards, there are faculty who considerinstruction to be an important part of the job of an online professor (on the clinical track). These teacherswho do research believe that “research matters, but not as much asquality teaching and service” (Borkoski and Kobett 2015). The recognition ofthis division of faculty between researcherswho teach and teachers who researchcontributes to a political debate that opens the door to policy alternatives.

Studentsunhappy with format

Interestingly, another political forceemerged in our analysis of the SOE faculty assessment program. Over 50 percentof the EdD students surveyed reported disagreement or a neutral response to thestatement, “My feedback is utilized to improve courses” (Borkoski and Kobett 2015).In other words, students feel unable to influence the instructional process.This suggests that new policy alternatives are needed to improve transparencywith SOE students and also to communicate the importance of their instructorevaluations. In a technological age, student unhappiness with a program iscommunicated quickly and widely. This reality also increases student power inthis political situation.

Variation in evaluation support andimplementation, by students and by the two faculty groups, could empower an SOEfaculty/student coalition to drive policy changeto implement more effective teacher evaluations and student impact on thatprocess. Furthermore, such a stakeholder coalition—empowered by intimate onlinerelationships between teachers and learners, and seeking technology-drivenpolicy alternatives—could alter the power relationships in higher education forthe foreseeable future. 

Conclusion

In the 21^st century, student needsare changing, and higher education faculty need “effective pedagogical skillsfor delivering student learning outcomes [to build] new relationships regardingaccess to teachers, and a wider range of communication and collaborativeworking through learning platforms” (Hénard andRoseveare 2012). Numerous policy options for the present SOE teacherevaluation system, offering better response to student needs, were suggested bythose interviewed in our survey. One faculty member discussed the importance ofconsistent and ongoing reflection throughout the semester. Another discussedgiving frequent and consistent feedback to online students, engaging inpersonal phone calls, and facilitating topical synchronous sessions withstudents. Still another faculty member identified the need to develop an onlinepersonality, including the ability “to lighten up the tension and concernswithin the community” (Borkoski and Kobett 2015) to ease students’ stress andanxiety. As one member of the faculty put it: Successful online instructorsmust adopt student-centered strategies including an ability to personalizefeedback, engage within a community as a member of the group, andestablish presence throughout the course (Borkoski and Kobett 2015).

Because the online EdD program is new, thepolitical environment is opportune for SOE faculty and students to join forcesand design a system that will change the constructs of instruction, for SOE andfor other higher-education institutions. This coalition can develop andimplement a new system that includes effective online teaching competencies,hiring practices that reflect them, and an evaluation system connected to theseidentified traits and feedback mechanisms. A clearly connected and integratedsystem of evaluation steps will lead to an evaluation process that not onlycollects data but uses the feedback in ways that improve teaching and learning.

Michel Foucault (1984) points out that it isour duty to break the bonds of professional constraints, reallocate power, andchange our environments. Beyond reforming the School of Education, faculty andstudents could infuse value into college instruction in research universitiesthrough the design and implementation of a new instructional evaluation system.This merger of SOE’s policy needs and politicalopportunity means that the time is now for a coalition for policy change.Getting to satisfactory policy alternatives for instructional evaluation andreform will be more complex than simply firing poorly performing faculty via Skype.However, the resulting policy alternatives will do a great deal more for highereducation teaching and learning as well as faculty and student empowerment.

References

Borkoski, Carey, and Beth M. Kobett.“Politics, Power, and Teacher Evaluation.” Johns Hopkins University, 2015.

Foucault,Michel.The Foucault Reader. Edited byPaul Rabinow. New York: Pantheon Books, 1984.

Hénard, Fabrice, and Deborah Roseveare. Fostering Quality Teaching in Higher Education: Policies and Practices: An IMHE guide for higher education institutions. OECD Institutional Management in Higher Education, 2012. https://www.oecd.org/edu/imhe/QT%20policies%20and%20practices.pdf

McLendon, Michael,Lora Cohen-Vogel, and John Wachen. “Understanding Education Policy Makingand Policy Change in the American States: Learning from Contemporary Policy Theory.”In Handbook of Education Politics and Policy, 2^nd edition, edited by Bruce S. Cooper, JamesCihulka, and Lance Fusarelli. New York: Routledge, 2015.