Please note: This article is published as an archive copy from Philadelphia City Paper. My City Paper is not affiliated with Philadelphia City Paper. Philadelphia City Paper was an alternative weekly newspaper in Philadelphia, Pennsylvania. The last edition was published on October 8, 2015.
|
|
|
|
|
Also this issue: A New D-Hole Ring Up the King Made Music Man Chow, Fun Commissioner Gordon |
|
|
|
|
 |
|||||||||
August 15-21, 2002
city beat
Failing the Test of Fairness
 |
|
Ever notice how expensive restaurants go out of their way to fill the air of their bathrooms with the refreshing scents of a pine forest after a gentle rain? Hoping to cover up the smells that would otherwise predominate in such an environment, the keepers of luxury lavatories bombard their patrons with diversionary scents, presumably to make one’s overall dining experience more pleasant.
Frankly, I've always perceived such efforts as more than a little inadequate to the task at hand. Shit, after all, even on a pine cone, is still shit. Likewise, there's a good reason why the makers of incense don't market a patchouli and crap stick. As we say in the South, you can pretty up a pig by slapping a dress on it, but in the end, it's still a pig.
Such is a lesson we would do well to remember in the wake of the recent announcement that the Educational Testing Service is going to "revamp" the SAT, ostensibly to make it fairer and more relevant for a 21st-century educational system.
Despite their insistence that the new SAT will better predict student ability while reducing unfairness by eliminating culture-bound items like analogies, the announced changes actually overlook the largest problems with standardized tests.
Although eliminating analogies is an admirable first step since studies have found these to be biased against those from non-white, non-middle-class backgrounds, what with questions involving words like "regatta," the problems with the SAT were always deeper than that. In fact, whatever cultural bias the ETS has eliminated with the ban on analogies will likely be re-triggered with the addition of a "writing" section, whose graders will no doubt emphasize stylistically and grammatically Standard English, marking students down whose writing style employs idioms, phrases or merely word patterns more common to communities of color. Poetic license will have no place, one suspects, on the SAT writing test.
Though internal cultural bias is a real phenomenon, and one that has been observed in testing for many years, the bigger issue is that supporters of the SAT presuppose that administering a standardized test to profoundly unstandardized students from unstandardized schools, and then using results on that test to determine college placement, can ever be fair.
The fact is, even if such biased items are removed from the SAT, the unequal educational experience of the students taking the test -- especially in terms of class and race -- all but guarantees a persistent scoring gap between whites on the one hand, and blacks, American Indians or Latinos on the other.
Furthermore, the announcement that Algebra II will be added to the test can only cause alarm for those concerned about the racial score gaps; after all, tracking in schools is so pernicious that blacks, even when they score at the top of 8th grade achievement test distributions, are about 40 percent less likely than whites whose scores are merely average to be placed in upper-level math courses in high school. As such, they won't even get around to Algebra II by the time they take the SAT.
But indeed, even tracking isn't the biggest issue here. Oh sure, it matters. On the one hand it means that certain students of color will be underexposed to the kind of material found on a test like the SAT; and on the other hand it means that certain students -- especially whites and many Asians who are presumed to be "good at math" early on, and thus tracked accordingly -- will have an edge going in to the test. But still, tracking is not the clincher that makes the SAT inherently problematic.
The two biggest issues are of a different nature altogether and incapable of being fixed with piecemeal reforms.
The first is what Claude Steele, Chair of the Psychology department at Stanford University, has called "stereotype threat."
As Steele and his colleagues have noted in a number of ingenious experiments, black students take standardized tests under a cloud of group suspicion that hinders performance -- suspicion on the part of the larger society that they are less intelligent and capable than others.
Black students are well aware of the negative stereotypes held about them by members of the larger society. As such, when blacks who are highly motivated and value educational achievement take a standardized test and expect the results to be used to indicate cognitive ability, the fear of living down to the stereotype negatively impacts their performance. These students may rush through the test -- so as to seem more confident than they truly are -- or alternately take too much time, trying desperately not to make mistakes. The self-doubt engendered by the racist beliefs of the larger culture is added to the general anxiety that all test-takers feel, to produce, for black students, a unique disadvantage.
As proof that it is stereotype threat and not inherent ability differences that explain racial gaps on standardized admissions tests, Steele notes that when the same test questions are given to whites and blacks in experimental settings, and the students are told that the results are not indicative of ability, and will not be graded, the stereotype threat dissipates and the black students perform as well as their white counterparts.
In other words, so long as racist beliefs about black ability are common, those stigmatized by these beliefs will often underperform as a function of the anxiety generated by the stereotype itself. Certainly there is nothing that ETS can do to the structure of the test that can alleviate this problem.
And finally, racial gaps are ultimately a function of the way that tests like the SAT are developed. Indeed, the gaps are all but built-in.
As anyone who has taken the SAT or a similar test remembers, there is an experimental section on the exam -- either an extra verbal or extra math section -- which contains questions that are not counted toward a student's score. The section exists so as to "pre-test" questions for use on future versions of the test.
But as ETS concedes, questions chosen for future use must produce (in the pre-test phase) similar gaps between test-takers as existed in the overall test taken at that time. In other words, questions are rarely if ever selected for future use if students who received lower scores overall answer that particular question correctly as often or more often than those who scored higher overall.
The racial implications of such a policy should be clear. Because black, Latino and American Indian students tend to score lower on these exams than white and Asian students, any question in the pre-test phase that these students answer correctly as often as (or more often than) whites and Asians would be virtually guaranteed never to appear on an actual standardized exam.
In practice, questions answered correctly by blacks more than whites have been routinely excluded from future use on the SAT. Although questions that whites answer correctly 30 percent more often than blacks are allowed to remain on the test, questions answered correctly even seven percent more often by blacks than whites have been thrown out.
Although the rationale for this practice is not overtly racist -- the testing company, for example, does not intentionally seek to maintain lower scores for blacks -- the thinking has a racist impact.
Essentially, the company's position is that for any question to have "predictive validity" it should be answered correctly or incorrectly in rough proportion to the overall number of correct or incorrect answers given by test-takers. But since the general scores have tended to exhibit a racial gap, such logic results in the virtual guarantee of maintaining that gap, as a function of test development itself.
If test questions were made less culturally biased, so that the racial gap shrunk or disappeared in the pre-test phase, those questions would likely be thrown out, simply because, being less culturally biased, they failed to replicate the racial gaps produced by the rest of the exam.
Interestingly, as testing expert Jay Rosner has demonstrated, the makers of the SAT could reduce the racial gap between whites and blacks while still maintaining the same level of overall test difficulty by choosing questions that, although equally tough, produce less differentiation between white and black test-takers. That instead they maximize these differences by way of the questions they choose, and that reforms of this nature are not being offered by ETS, indicates how unconcerned they truly are about test fairness.
Instead of trying to pretty up this pig, persons concerned about educational equity, true opportunity and fairness should be calling for colleges and universities to either eliminate the use of the SAT in admissions decisions, or at least to heavily downplay its importance, given its irrelevance in predicting actual academic ability.
SAT gaps of as many as 300 points between two students (or groups of students) can be completely insignificant in terms of indicating actual ability differences, and gaps of 125 points between students are considered random by the test-makers themselves, and say nothing about the different abilities of the students in question.
That SAT scores have little to do with one's ability is borne out by a number of studies and even data provided by the test-makers themselves, which indicate that only 10 percent (at most) of the difference between students in terms of freshman grades can be explained by results on the SAT. Further, the correlation between SAT scores and overall four-year college grades or graduation rates has been so low as to be essentially nonexistent, explaining no more than 3 percent of the difference between any two students.
If ETS wants to promote fairness -- and indeed they insist that they are committed to changing the unequal educational system that helps produce scoring gaps -- they must first stop promoting a test battery that replicates and reinforces that inequity. If they wish to provide tests purely for the purpose of gauging how much is being taught and learned in K-12 schools, so be it.
But so long as they release test scores prior to college admission, knowing that such scores will be used to dole out opportunities that themselves result in still more opportunities upon graduation, ETS can only be seen as complicit in the maintenance of racial and economic stratification. They are not reformers, but merely gatekeepers for the status quo. And that smells the same, no matter how one tries to cover it up.
| -- Respond to this article in our Forums -- click to jump there |
