The "New" SAT 2005: A Better Test or Just a Marketing Tool?

Provided by FairTest

In June 2002, the College Board announced a series of changes to the SAT-I that will be implemented in March 2005. The move primarily responds to threats that the University of California, the SAT's biggest customer, planned to drop the test and to the growing number of colleges who have made test scores optional for many applicants. Nonetheless, the "new" SAT-I is being billed as an exam that better relates to what students learn in high school.

Despite the slick packaging of the "new" test by the College Board, many questions and concerns still remain unanswered. None of the revisions address the SAT-I's fundamental flaws such as the test's inaccuracy, bias, and susceptibility to coaching. Nor has the College Board acted to crack down on widespread misuses of the SAT-I, such as requiring minimum scores for admissions or scholarships. Moreover, contrary to the marketing claims accompanying the promotion of the "new" SAT-I the revised exam will do little to improve the quality of K-12 education.

HOW WILL THE "NEW" SAT-I BE DIFFERENT FROM THE CURRENT TEST?

The overall format and content of the "new" SAT-I will remain largely unchanged from the current test. It will still be primarily multiple-choice and administered under strictly timed conditions. The SAT-Verbal will be renamed "Critical Reading" and will include additional short Reading Comprehension passages in place of the much-criticized verbal Analogies. The math section will contain some Algebra II questions (it currently only covers Algebra I and geometry), and the arcane Quantitative Comparison items will be removed. Responding to criticism about the SAT-I being far removed from classroom learning, the College Board will also add a so-called "Writing" section to the exam. Modeled after the current SAT II: Writing Test, the new section of the SAT-I will be about two-thirds multiple-choice copy editing questions and one-third a 20-25 minute essay. Each section will still be graded on a 200 to 800 point scale, so the addition of the third section will bump up a "perfect" SAT-I score to 2400. The total testing time will rise from 3 hours to 3 ½ hours. These changes are expected be accompanied by a cost increase of $10-$12.

College Board technical reports acknowledge that a student's high school grades and courses taken provide a better forecast of college performance than the SAT-I does. The current test does a particularly poor job of predicting how females, students of color, and older test-takers will do in college. Since the revised SAT-I is still under development, there is no research demonstrating how the test's "predictive validity" will be affected by the changes. However, several sources of information do provide clues about the changes' likely impact.

With the exception of the added writing component, the "new" SAT-I will closely resemble the current test in form and content. This means the test will likely remain a weak predictor of college grades and bachelor degree attainment. Although the exam's predictive validity may increase slightly due to the addition of a third section (predictive ability tends to rise with each additional test score), this improvement will probably not be substantial nor be equal across all demographic groups. College Board research on the Test of Standard Written English (TSWE) it administered as part of the SAT prior to 1994, and the English Composition Achievement Test (ECT), which together were the prototypes for the current SAT II: Writing Test, calls into question the predictive power of the "new" SAT-I writing section. This research showed that the TSWE and ECT did a particularly poor job of predicting the college performance of African American students and students whose strongest language was not English. In fact, the TSWE was removed from the SAT in the early 1990s because the College Board recognized that the test was not a useful tool in the college admissions process.

WILL CHANGING THE SAT-I LEVEL THE PLAYING FIELD FOR STUDENTS FROM DIVERSE BACKGROUNDS?

College Board research demonstrates that the SAT-I systematically under-estimates the academic potential of young women, students whose first language is not English, and applicants over 25. This is unlikely to change with the "new" SAT-I. The underlying causes of the score gaps - including the test's multiple-choice format, highly-speeded pace, and rewards for strategic guessing - will remain in place. The gender gap may be reduced slightly with the addition of the essay question, since females tend to score slightly higher than males on the SAT II: Writing Test, but will probably not be completely eliminated. However, the score gap for students from non-English backgrounds will likely grow larger due to the added challenge from the high pressure, timed conditions of the essay question. On the SAT II: Writing Test, African American and Latino test-takers score on average 80-100 points lower than White students. In fact, the SAT II: Writing Test has one of the largest Black-White test score gaps among the twelve most popular SAT II tests, second only to the SAT II: Literature Test. These gaps will likely carry over to the "new" SAT-I, given its similarity in form and content. Moreover, the College Board has made no indication it will act to stop test score misuses, such as minimum score cut-offs, that have a particularly harmful impact on the opportunities available to African-Americans, Latinos, low-income students, and students with special needs.

WILL THE "NEW" SAT-I BE ANY LESS SUSCEPTIBLE TO COACHING?

While there is still debate over how much test prep can boost students' scores, the College Board has backed away from its historic claim that the SAT-I is not "coachable" and now sells its own test preparation materials. Regardless of how much coaching can increase SAT scores on average, it can substantially enhance some students' scores, thus further tilting the college admissions playing field. The coaching industry is anticipating a boom in business from a revised SAT-I, since firms such as Kaplan and the Princeton Review know that any change in the admissions process feeds student anxiety. They say the "new" SAT-I will be just as "coachable" as the current test, if not more so. One major test preparation company is already touting its success in training test-takers on how to raise their scores on the SAT II: Writing Test as evidence that the writing section on the SAT-I will increase the exam's coachability. The ability of coaching to boost students' test scores skews the college application process in favor of students from higher-income families that can afford the $800 or more that an intensive course costs.

A proposed SAT-I essay section was scrapped in the early 1990's because of potential logistical problems in grading it and grave equity questions. These concerns have yet to be resolved. A June 2002 notice on the College Board's website announced that SAT II: Writing Test scores would be delayed due to the unusually high volume of tests administered. Since there are close to ten times as many SAT-I's administered as SAT II: Writing Tests, such delays call into question the test-maker's capacity to grade more than 2 million tests annually in a timely and fair manner.

Each essay will be read in less than four minutes and rated on a "holistic" 1-6 scale by two readers. If the readers' scores differ by more than 2 points, a third person will read the essay to resolve the discrepancy. Readers will evaluate writing skills using a vague set of criteria, relying on the same guide now used to rate the SAT II: Writing Test. This guide includes things such as variety in sentence structure and range of vocabulary, which means test takers can earn high marks for complicated sentences with plenty of "10-cent" words just as easily as they can for concise, interesting writing. The criteria also include measures such as "effectively and insightfully" responding to the writing prompt with "well organized and fully developed" arguments. While these goals may seem like worthwhile writing skills, they open the exam up to subjective judgments from essay readers.

There is an old adage in the measurement profession: "What is tested becomes what is taught!" No matter how the SAT-I is altered, there will be strong pressure on teachers to drill their students on the narrow subject matter and formats it covers. One College Board study estimated that at least half of the high schools in the U.S. already offer SAT prep. Such practices will surely increase given the anxiety students will feel as the revised SAT-I is phased in. These drills come at the expense of more worthwhile learning opportunities. While some have praised the "new" SAT-I as providing an incentive for schools to teach writing skills, it will actually encourage educators to focus on how to write formulaic five-paragraph essays rather than developing students' writing skills more broadly. In addition, almost every state assessment system already includes a writing test, many of which allow students to use more time than the twenty-five minutes proposed for the SAT-I essay. Therefore, no additional "incentive" is needed in order to encourage teachers to focus on the limited writing skills covered by these assessments.

WHY SHOULD ANY COLLEGE REQUIRE THE SAT-I, OLD OR "NEW"?

Nearly 400 bachelor degree-granting institutions nationwide do not consider SAT-I or ACT scores before making admissions offers to substantial portions of their entering classes (see this list at http://www.fairtest.org/univ/optional.htm). They recognize that there is ample information in applicants' files to make superior admissions decisions without the distortions cause by SAT-I scores. The "test-score optional" list already includes some of the most selective private colleges in the nation, such as Bates, Bowdoin, and Mount Holyoke, as well as large public campuses like the University of Texas at Austin. Rather than accepting the College Board's promotional claims about the "new" SAT-I, colleges and universities should look with a critical eye at both the "new" and "old" exams and follow the lead of "test-score optional" institutions.