Do the Best Researchers and Society Win? Fairness, Validity and Bias in Promotion and Tenure Decision Making

Promotion and tenure decisions at universities are of critical significance to the integrity of the research enterprise because they determine the career progression of scholars and scientists.  Despite significant investments in pipeline interventions to diversify academia, faculty of color and women are underrepresented in tenured and tenure track positions.  Underrepresentation has important implications for the nation’s prospects of developing a diverse STEM workforce.  This project examines the validity and fairness of external review letters (ERL) provided by arm’s length reviewers as part of promotion and tenure (P & T) decision making.  Academic administrators often view ERLs as the most impartial and critical components of P & T portfolios, providing qualitative, independent evaluations of the candidates’ past accomplishments, reputation, potential, and the prospect of continued, sustainable contribution levels.    Despite their sensitive nature, given the criticality of ERLs in academic P & T decisions, the limited review on validity and bias in ERLs is stunning.  This project closes this gap by analyzing the linguistic characteristics of ERLs and examining the relationship between ERL linguistic characteristics, promotion candidate characteristics, letter writer characteristics, and P & T voting outcomes at the department, college, and university-level committees. The project examines ERLs through the theoretical work on the social psychology of language use, social comparison processes, and social role theory.  Using data from more than 2000 faculty P & T decisions at 10 research universities and more than 10,000 ERLs, we discuss validity, fairness and bias, and discuss policy implications for funding agencies and universities.

The current research was supported with funding from the National Science Foundation (NSF grant ADVANCE IT grant #1409928; NSF EHR research grant #2100034) and the Alfred P. Sloan Foundation.  Any opinions, findings, and conclusions or recommendations expressed are those of the author(s) and do not necessarily reflect the views of the NSF or the Alfred P. Sloan Foundation.