How the GRE’s have been “fuzzified” in order to “close the gap”
Lydia McGrew writes:
You have discussed with some of your readers the possible dumbing down of tests to close racial gaps. As you know, the SAT was changed some years ago. I believe one of the motives of that was to close male-female gaps. At the moment I don’t have the details of those changes in front of me, but at the time it was pretty blatant.
I can speak to the changes that are taking place in a yet higher-level test, the Graduate Record Examinations (GRE’s). Several years ago the GRE analytic section, which consisted of logic puzzles, was scrapped. It was replaced with an essay, ostensibly to test the students’ ability to write analytically. Speaking as the wife of a professional philosopher, I will simply say that this is looked on somewhat skeptically by professional logicians. The scoring of an essay is of necessity less objective than the scoring of a set of logical puzzles, and it’s unclear who grades the essays. English professors?
Next month, the test is changing still more. Most notably, the section of the verbal portion of the test that included synonyms is going to be eliminated. There will now be no testing at all on vocabulary without context. I think there used to be antonyms, but if so, they were scrapped long ago.
Another technique by which “gaps” are closed in tests (I don’t know whether this will be used on the GRE) is simply to make some questions so strange and subjective, or so truly unreasonably hard, that no one likely to take the test, even the very best students, will get them right any more often than anyone else. This randomizes results and hence helps to “close gaps.”
In essence, tests are now being designed with closing gaps as a more or less overt goal. This will sometimes take the form of direct dumbing down but may also take the form of fuzzifying or of making more sections difficult to grade objectively. Anything to avoid actually discovering academic merit in a clear-cut fashion. In essence, the new goal of assessment is to assess inaccurately and imprecisely.
LA replies:
And at bottom all these changes, all this destruction of standards and quality, all this dishonesty, all these lies—it’s about equalizing blacks.
I could be wrong, but I don’t think that closing male/female gaps was a major factor in the re-norming of the SATs in the mid ’90s. I think it was the totality of reducing all differences between higher and lower scores, with the main goal being the reduction of white/black differences. Since girls were already doing very well academically, it doesn’t make sense that any male/female gap in the SATs would have been enough motivation for renorming the test in the radical way that was done—so radical that there is a complete disjunction between the SATs before and after that time. Do we see constant stories in the media about how girls are “lagging” and about the need to “close the gap” between males and females in school testing? No. But we do see constant stories about blacks “lagging” and the need to close the gap between whites and blacks.
Lydia replies:
My recollection is that the SATs were not only re-normed in the ‘90’s, but were revised in the early ‘00’s. An online friend who taught SATs prep courses at the time told me expressly that one goal was to reduce male/female differences, and as I recall he laid out a pretty good case. However, the changes could easily have had a dual purpose.
- end of initial entry -
Jim C. writes:
There’s no question that the types of question all standardized test organizations are trying to eliminate are those tests which measure “g,” or general intelligence. And which group has difficulty with those tests?
Here’s an article on the new GRE.
James N. writes:
Since my high SATs in 1967 and later my high MCAT score in 1971 had the effect of erasing years of laziness, I have followed the actions of my friends at the Educational Testing Service with interest.
The 1990s renorming, together with elimination of analogies, and the addition of an essay, were specifically intended to erase the tremendous male-female gap in SAT scores, with the goal of increasing the matriculation of females in four-year baccalaurate programs.
This policy initiative has been wildly successful, since the percentage of bachelors degrees awarded to females has now exceeded that awarded to males, and the gap is growing.
At the time, there was no discussion of how to fix the black-white gap, and, apart from increasing graft and corruption in the public schools, there still isn’t.
Norman P. writes:
You wrote: “it’s about equalizing blacks.”
Yes, a most noble goal. Thanks for bringing this to our attention; we tend to lose sight of this objective going about our daily chores. The idea of uplifting our fellow man, our brothers, indeed the whole society is what drives us to become even better people. This needs to be emphasized from time to time so that our energy doesn’t begin to flag and our efforts dimmed by time. Once again, thanks.
Greg W. writes:
If I remember correctly, calculators were introduced to equal the playing field between men and women for the math section of the SATs . I believe they also looked into weighting the verbal more than math being that women do better than men in that area.
Dan K. writes:
It is worse than you suspect. If you want to get your results from the Educational Testing Service they only go back 20 years. Anyone who took the GRE test longer ago than that will be rebuffed as I was a few years ago. I even had an official from my undergraduate school try and he failed to get the scores released. At the time the official said that the reasoning given made no sense to him. Future data mining of GRE data is being made to be nearly impossible. Without such data mining the QUALITY of the test becomes suspect over time, especially given the continual modification of the test components.
LA replies:
That’s just amazing. Testing results, meet Memory Hole.
Ken Hechtman, VFR’s leftist reader in Canada, writes:
I used to teach test-prep in the late 1980s and early 1990s. I’m not nearly as familiar with what the SATs and related tests are like now but I can tell you quite a lot about what they were like then. The verbal section was basically a vocabulary test. If a student had grown up reading real books he’d do well. If not, test-prep could help a bit (I promised a 100 point increase) but it wasn’t a substitute for a lifetime of reading real books. That “gap” was built-in—rich kids would do better than poor; private school students would do better than public; and, have it your way, whites would do better than blacks. Those are all just different ways of saying the same thing. [LA replies: With your implication of some sort of injustice in reliance on vocabulary, you seem to be forgetting that the very purpose of the SATs and the GRE’s was not to make all groups equal, but to determine (note that I say “was” not “is”) who was able to do college and post-graduate academic work. And a large vocabulary correlates highly with that ability.]
And then there was one obvious attempt even then to narrow the gap, and I used to teach my students how to spot and use it.
I used to say, “Unless you are realistically aiming for a score of 700, skip the most difficult-looking of the Reading Comprehension sections. You will not finish the test if you try to answer it and you’ll miss easier questions you could have gotten right. One of the Reading Comps will be the “Ethnic Passage”—a passage specifically about non-whites in America. NEVER skip this one, it will be the easiest of all the Reading Comps. And when you answer the questions on it, remember that the answer that puts the particular ethnic group or personality in the best light is ALWAYS going to be the answer ETS wants.”
AM writes:
Renorming the SATs has probably diminished the role of intelligence in admissions in elite colleges. The renorming effectively added 100 points to everyone’s score, and hugely increased the number of students with perfect and near perfect scores. This is the equivalent of holding a 100 meter dash, and then setting any race time underneath say, 11 seconds, up to 11 seconds; Usain Bolt would be indistinguishable from his slower rivals. You could not distinguish the fastest from the very fast. With the SAT nowadays, a perfect score is achievable by the very bright as well as the brilliant. When one cannot distinguish between the two, and very high scores become prevalent, intelligence perforce plays a smaller role in admissions. Other, more amorphous qualities fill the void.
And, truly, high intelligence alone may not ultimately serve the modern university well; Caltech is reputed to have some of the brightest students in the US, yet, unlike its rivals, it hardly fills the halls of power. It’s tempting to blame racial and sexual preferences, but future alumni wealth and power are probably a factor as well. Selecting solely with regard to intelligence might mean less charismatic students, and thus fewer leaders in the future among alumni ranks.
D. from Seattle writes:
Lydia McGrew said:
” … the section of the verbal portion of the test that included synonyms is going to be eliminated. There will now be no testing at all on vocabulary without context. I think there used to be antonyms, but if so, they were scrapped long ago.”
As someone who took the GRE in 1990, I can confirm that back then there were indeed synonyms and antonyms sections in the verbal part of the GRE, in addition to reading comprehension and possibly another section. I remember synonyms and antonyms clearly because that was the hardest part of the whole GRE for me, a non-native English speaker. Synonyms and antonyms were the only part of the test for which I had to study hard, and I probably spent a couple of months cramming all those words in order to improve my score. Everything else was easy by comparison, and I mean everything: reading comprehension and all of math and analytic (logic) sections. I didn’t even think that anything else deserved studying, for if you don’t already know math and if you can’t reason logically and if you can’t understand and analyze written English, you’re not going to learn that in a month or two or even six. But you can memorize hundreds or thousands of unfamiliar words and improve that part of your test score.
I won’t mention my scores, but I did really well on the math part, almost as well on the analytical part, and OK on the verbal part given the circumstances. And that’s how I ended up in a grad school in the U.S. I stayed in the country afterwards and am now a naturalized citizen.
Aaron S. writes:
In the mid 90s, when I was in graduate school, I had a job one summer with a SAT tutoring service. I remember distinctly a training session for new employees in which I asked whether and how the exam had been changed from ten years earlier, when I had taken it. The instructor answered yes, and that a score of, say, 1400 at present would correlate with a score of about 1250-1300 from a decade previous. This was good, he said, because it “got everyone’s scores up a bit.” I was astounded and looked at the fellow next to me (another graduate student in philosophy) and we simultaneously burst out laughing. No one else laughed. It was hard to say whether they weren’t paying attention, or they just didn’t get it. Perhaps some of both.
As to Greg W.’s mention of calculators, that was the other thing about the job that struck me. I had several tutees unable to perform elementary arithmetic operations quickly; they fumbled with calculators for things I’d supposed people had been made to do in elementary school math class. These were white, college-bound students in very well-to-do Boston suburbs.
Posted by Lawrence Auster at July 14, 2011 09:30 AM | Send