Standardized tests are simply thermometers. Yet they’ve been abandoned by many.
In his book Measuring Up, Daniel Koretz reminds readers that the reason we call something a “standardized test” is simply because it means that everyone takes the same test. The term appears to have taken lots of other (often unpleasant) meanings in people’s minds. But what has been lost is just this simple idea that a test is a thermometer that is standardized. Just as how a thermometer is more reliable and valid to assess a fever than putting your hand to your child’s forehead, using a standardized test is more reliable and valid to assess academic readiness.
When used in the context of admission to gifted and talented programs, colleges, or graduate schools, tests function as thermometers of the developed reasoning of students and their relative preparedness or academic readiness for the rigors of advanced educational programming.
And yet testing has been under attack for years, whether that means the SHSAT or any other test in gifted and talented admissions, the SAT or ACT in college admissions, or the GRE, PCAT, or numerous other graduate admissions exams. In many cases, tests have either been made optional or completely abandoned.
The New York Times now suggests that perhaps removing the thermometers was a misguided idea.
Now, David Leonhardt of the New York Times, explains that though many of these tests have been made optional or even completely abandoned, perhaps not using empirical evidence to make policy decisions was a mistake, even hurting underserved students. In “The misguided war on the SAT”:
“After the Covid pandemic made it difficult for high school students to take the SAT and ACT, dozens of selective colleges dropped their requirement that applicants do so. Colleges described the move as temporary, but nearly all have since stuck to a test-optional policy. It reflects a backlash against standardized tests that began long before the pandemic, and many people have hailed the change as a victory for equity in higher education.
Now, though, a growing number of experts and university administrators wonder whether the switch has been a mistake. Research has increasingly shown that standardized test scores contain real information, helping to predict college grades, chances of graduation and post-college success. Test scores are more reliable than high school grades, partly because of grade inflation in recent years.
Without test scores, admissions officers sometimes have a hard time distinguishing between applicants who are likely to do well at elite colleges and those who are likely to struggle. Researchers who have studied the issue say that test scores can be particularly helpful in identifying lower-income students and underrepresented minorities who will thrive. These students do not score as high on average as students from affluent communities or white and Asian students. But a solid score for a student from a less privileged background is often a sign of enormous potential.”
The enormous body of research supporting standardized tests is nothing new.
Leonhardt goes on to review some recent data from the Opportunity Insights project and from MIT, among others, which summarizes more or less what we’ve known about the predictive validity of these tests for decades (and arguably, for the last century). His arguments parallel a piece I wrote with Matt Brown and Christopher Chabris in 2019 in The Washington Post titled “No one likes the SAT. It’s still the fairest thing about admissions: Eliminating standardized testing would remove the one admissions criteria that can prevent fraud and increase social mobility.”
In that article, we summarized and linked to the enormous body of research supporting the use of the SAT and ACT to be one important thermometer used along with other indicators to help admissions officers make more objective decisions, in particular to help underserved students from low-income families who don’t have the resources to pad their application with extracurriculars or other aspects to have a fair shot. However, missing from the Leonhardt piece is data showing that testing is linked to a wide range of later life outcomes throughout the entire range of test scores. This includes data from David Lubinski, Camilla Benbow, and colleagues of Vanderbilt University showing that among academically talented youths, SAT scores before age 13 predict life outcomes decades later such as doctorates, publications, patents, higher income, and even university tenure. Additionally, Paul Westrick and colleagues have shown that tests actually help identify students most likely to persist in college.
The discussions and policy decisions surrounding standardized tests sadly seem to have little to do with the actual empirical evidence.
Perhaps most presciently, Leonhardt writes: “In today’s politically polarized country, however, the notion that standardized tests are worthless or counterproductive has become a tenet of liberalism. It has also become an example of how polarization can cause Americans to adopt positions that are not based on empirical evidence.”
In fact, in the majority of the discussions surrounding standardized testing of all kinds, empirical evidence seems to be one of the least important aspects when policy decisions are made about test use or removal. Research led by Don Zhang published in Collabra: Psychology argues that “policy preferences toward admission tests depend on the perception of statistical evidence, which is malleable and depends on how evidence is presented.”
And in an earlier article for Forbes titled “What the research says on tests and test-optional policies in college admissions” I summarized the full body of research to date from multiple disciplines around tests and test-optional policies at every level, finding that at best the evidence for tests being made optional was weak or unclear and most likely would have unintended consequences.
Additionally, in order to understand the impact of the pandemic on test optional admissions policies, the College Board Admissions Research Consortium (on which I serve as an advisor) has been conducting ongoing research with 80 different colleges and associations. Based on a recent report titled “New evidence on recent changes in college applications, admissions, and enrollments: Focus on the Fall 2021 admissions cycle,” I interviewed lead author and VP of research Jess Howell. Her responses to two relevant questions adds to the Leonhardt analysis:
How did students navigate the choice about whether or not to submit their SAT or ACT scores on their college application?
“Among those with a test score and a choice to make about whether or not to share it on their college applications, we find that the biggest driver of the decision is whether their score is high or low relative to typical scores at the college to which they’re applying. Applicants whose scores are relatively high are very likely to submit their scores, while applicants whose scores are relatively low are not very likely to submit their scores.”
Did test-optional change diversity at these colleges?
“Prior research on test-optional college admissions finds either no change in racial and socioeconomic diversity or small changes. In our new research, we find that, because the enrollment of all subgroups of students increased between fall 2020 and fall 2021, the proportional representation of student subpopulations—by race and socioeconomic status—changed very little at these institutions. Black, Hispanic, and Native students made up about 25% of college enrollees in the sample before and after the pandemic and test-optional admissions policies were in place. Similarly, students from disadvantaged schools and communities made up the same proportion of college enrollees before and after. So, overall, we don’t find a change in trend to the racial/ethnic representation of students in this first year of near universal test-optional policies.”
The misguided war on standardized tests impacts students at every level. It often hurts underserved students we are intending to help.
Perhaps due to space limitations and the need to focus on contemporary data to make one point exceptionally clearly, Leonhardt wrote about the SAT and data on a limited set of outcomes in his New York Times piece. Here, I point to summaries of the research at every level, including gifted and talented admissions, college admissions, and graduate admissions. Using universal screening can improve the representation of underserved gifted and talented kids, can narrow income gaps in college, or allow graduate students of all backgrounds, especially those who don’t have connections to get early research experience on their resumes, or those from other countries, to have at least one thermometer of academic readiness where everyone takes the same test.
The research at every level shows that standardized tests are reliable, valid, and predict a wide range of academic and life outcomes and are useful as one tool in admissions and selection. At the very least, the weight of the cumulative evidence (to date) does not support the removal of testing. Tests have their imperfections, for example a line of research suggests they miss identifying and supporting students with spatial reasoning strengths. But this does not mean we should abandon these mathematical and verbal focused tests, but rather that we should expand them to include spatial reasoning measures to find and support even more talent. Frank Worrell and I have long argued that the removal of tests, perhaps counterintuitively, has contributed to “How talented low-income kids are left behind.”
