Do SAT Preparation Courses Really Work?

For decades, colleges have relied upon the SAT Reasoning Test as a tool for determining a student's eligibility for enrollment. Not surprisingly, courses of all shapes and sizes have emerged professing the ability to markedly improve student scores. In parallel to the claims of the SAT preparation courses has been the mantra of those who suggest that, not only are SAT courses unable to significantly raise test scores, but that the SAT is a "pristine" exam immune to such efforts. In order to answer the question "Does SAT preparation really work?" it is necessary to carefully consider the positions of both camps.

One might argue that, as a seasoned SAT expert, author, and instructor, I have a particular interest in promoting the virtues of SAT prep courses in general, but this could not be further from the truth. What almost three decades of SAT preparation expertise has instilled in me is an interest in promoting effective SAT courses and steering students away from so many ineffective options. Those courses that routinely fail to produce desired results are detrimental to the test preparation industry as a whole! Any objective discussion of SAT preparation success or failure must begin with a consideration of statistics. Some "experts" have suggested a very simple study involving a few hundred students, some of whom receive coaching and some of whom do not. The results can then be statistically analyzed to determine whether or not SAT preparation really works. Sounds simple, logical, and fair, right? Wrong! In fact, such a simplistic compilation would result in grossly inaccurate, misleading data and erroneous conclusions. While I applaud the idea of an unbiased, statistical evaluation, a truly fair and accurate study would prove to be profoundly complicated and complex.

Here's why. In order to begin such an analysis, one would first have to sift through literally hundreds of SAT courses to find a suitable preparation vehicle. The sad truth is that a great number of SAT courses simply do not work, and some of them are miss the mark entirely. Approximately fifteen years ago, one of the major financial periodicals published an eye opening expose' concerning the plethora of ineffectual and, perhaps, fraudulent SAT coursework on the market at that time. I dare say that, today, those numbers seem to be on the increase. Armed with false promises and fictitious data, such organizations bilk customers out of thousands of dollars, yet are no longer limited to small, local, no-name organizations. Students who unknowingly subscribe to such programs typically assimilate little or no insight into the nature of the SAT test itself and realize little or no score improvement. Whenever I interview students who have come to me after completing any SAT course, I routinely ask a few fundamental questions such as "How many sections comprise a typical SAT?" Amazingly, more than half of those students are completely unaware of the structure of the exam! Would it be fair, then, to include data from such courses in a statistical analysis?

Let us not forget to include many high school-generated prep courses among the ineffective. Those make-shift (albeit well-intentioned) programs, which the schools once offered free of charge but can now cost hundreds of dollars, lure thousands of students each year! The problem with these courses is that, by and large, high school mathematics and English teachers (not SAT experts) present the coursework. While these teachers may be highly-skilled educators with considerable knowledge and expertise in specific subjects (algebra, geometry, writing, grammar, etc.) they are generally untrained in the explicit reasoning skills necessary to master the SAT. Practically every student whom I have interviewed who has attended such a course has felt that the preparation was at best a waste of time. Indeed, some have emerged discouraged, disgruntled, and opposed to any further preparation! Are we also to include those data in an objective analysis?

And what about the group-oriented programs that routinely hire SAT instructors of limited insight and experience? A few years ago, a woman boasting of a whopping six-months' SAT instructional experience, called me requesting employment. Although I certainly did not hire her, I would bet that a number of companies would be perfectly willing to do so in spite of her extremely limited experience (expertise is so very important that I never permitted any of my tutorial staff to teach SAT prep). Are those companies to be ranked equally with the one-to-one preparation of seasoned SAT experts? To understand this, let's consider dining as an analogy. No one would believe a fast-food chain to be comparable in quality to a five-star restaurant, but the fast food joint does have several major advantages. A large fast-food chain can serve far more customers than can a single dining establishment, and can do so at a considerably lower price. A customer willing to sacrifice quality for price and availability would be far better served at such an establishment.

Does the analogy extend to SAT preparation, however? In some cases it does. A number of SAT experts are simply too high-priced (0-0 per hour) for the average customer, and have chosen to offer their services to those willing and able and afford them. When it comes to SAT coaching, however, price is not at all a definitive indicator of quality. Some of the most highly skilled experts offer their services at relatively moderate prices (comparable to those of the "fast-food" courses), while certain neophytes are happy to charge exorbitant prices! Clearly, no rational person can justify lumping such diverse data into the same statistical bundle while, at the same time, professing to objectively analyze the general effectiveness of SAT preparation coursework.

In addition to discussing the disparity in quality among SAT programs, we must also consider that of the students themselves. Even the best of SAT courses will inevitably entertain a certain number of unmotivated clients, and compliance has always been a primary component of SAT preparation. Although a number of SAT "programs" promise success with little or no study or homework (indeed, some profess to prepare a student in a weekend or two) no legitimate SAT course can offer something for nothing. A student unwilling to complete his or her assigned work is unlikely to show a major improvement in score. There are also certain numbers of students with inherent learning difficulties. Those students may to show less improvement than students of average ability. Is it fair, therefore, to include all students in an overall analysis?

This brings me to another important factor in determining the success or failure of an SAT preparation course. Just how does one evaluate improvement? Unaware that I am an SAT instructor, a representative from another SAT prep company contacted me recently peddling his company's services on the promise that a student will improve his or her score by at least 100 points. So I asked him, "If a student came to you with a prior score of 2300, do you promise a perfect score of 2400 upon the completion of your coursework?" Not surprisingly, his answer was "Well...er...no." I then asked him whether a student can expect a 2300 if he or she started with a 2200. Although I did not set out to put this fellow on the spot, I did intend to point out the sheer ludicrousness of his company's promises. As SAT scores increase, incremental improvements are, naturally, much smaller. Indeed, a fifty point increase from 2100 to 2150 may be just as significant and may require just as much diligent study and effort as does a 200 point increase from 1700 to 1900! So where exactly does one begin to evaluate the significance of score improvement? In the absence of complicated statistical adjustments, gifted students with high initial SAT or PSAT scores would actually diminish numerical averages even though those students may have experienced satisfying increases in total score!

Unaware of such statistical complexities, well-intentioned parents routinely inquire as to a particular instructor's rate of success. Understandably, potential clients would like to know the average increase in score that they can expect after taking a particular course. I am sure that, to some of these folks, my answer must sound somewhat convoluted. On one hand, I can certainly say that the majority of my students show a marked improvement in score, but I cannot present any hard numbers without meticulously filtering the data. In the absence of my deleting unmotivated and learning disabled student performances and adjusting for high initial scores, the numbers would be misleading, and I am sure that this dilemma extends to other SAT courses as well. Therefore, when a parents make such a request (as all prospective clients should), I give them a reasonable estimate of expectation based upon students who are similar in scholastic performance, motivation, etc. to their students.

Hopefully, one can now begin to appreciate the fallacies and limitations associated with simple statistical analyses. Depending on the variables selected and controlled, a simplistic compilation and analysis could produce dramatically varying results which would, in turn, lead to erroneous conclusions. If one were somehow able, however, to eliminate from the equation all of the factors that would unfairly diminish the averages, it is my contention that one would notice a remarkable average increase in performance after completing a legitimate, high-quality SAT program taught by a seasoned SAT expert, but the experimental conditions would, by definition, remain extraordinarily complex.

That said, let us turn our attention to those who, even to this day, insist that it is virtually impossible to significantly improve an SAT score. In 1995, the same year that the College Board changed the test title from SAT to SAT 1 and altered the test format, I began to hear rumors that the new design was supposed to be preparation-proof. Those rumors turned out to be utter nonsense (the newer format actually proved somewhat easier to prep than was its predecessor), but it is interesting that someone, somewhere, had felt compelled to circulate them. Thankfully, one hears such nonsense less frequently today than a decade ago, but some folks still believe in the "pristine" nature of the SAT and continue to disseminate the rumors. Although, we may never know the source of that myth, we are well aware of its impact... every year, the misassumption that the SAT is not teachable prevents some sadly misled students from actively working to improve their scores!

In the early 1990s, rumors began to circulate suggesting that the College Board (the company that designs and administers the SAT), in an effort to promote the purity of the SAT test design, might have been one of the original proponents of this myth. These rumors may have been the result of the notion that, if the College Board were to have admitted that SAT preparation courses were valuable assets in improving scores, such an admission would be tantamount to an acknowledgement that the test was not even close to being the "pristine" evaluation tool that, ideally, many people thought that it should have been. Such an admission would, in turn, open wider the Pandora's Box of cultural and economic bias claims... "The rich can afford the prep courses that will ultimately get a student into a prestigious college or university." Nonetheless, whether or not College Board originally promoted the notion of a pristine test, it clearly has not done so for many years. As a matter of fact, the College Board currently markets its own SAT preparation coursework, an indirect suggestion, that one can, indeed, improve one's scores through study and practice!

Yet, in spite of overwhelming evidence substantiating the value of good, solid, SAT-specific preparation, there are still those who diminish or even deny it. Although relatively few in number today, those who would oppose SAT preparation coursework tend to fall into one of two categories: parents who, after subscribing to such a course, have witnessed poor results, and high school counselors and teachers who, years ago, had fallen for the rumors but who have never taken a hard look at the facts.

And those facts are becoming more and more evident each year...the SAT would seem to be a test that yields readily to proper study protocols. Pragmatically speaking, is it not reasonable to suggest that, if SAT preparation truly cannot significantly improve a score, then no one would improve as a result of such preparation? The fact that so many students do improve, however, attests to the value of SAT prep. Although statistics suggest that a score may increase by as much as forty points (about four correct answers) by simply retaking the test, those statistics do not account for the 100-400 point increases that seasoned, professional SAT experts routinely produce. Moreover, merely retaking the test without concerted study is no more than a toss of the dice; a student may or may not improve (and may do even worse). I recall a seventeen year-old high school student who had come to me after taking the test five times and getting roughly the same score at each sitting. That is not an isolated incident, either. Rarely have I seen a student improve his or her score by merely retaking the test!

Clearly, therefore, there are important factors to consider before subscribing to an SAT preparation course. An inappropriate or hasty decision is likely to result in a dismal, discouraging outcome. The following is a list of criteria that I recommend any student interested in preparing for the SAT Reasoning Test take into consideration:

1. Accept the fact that not all SAT prep courses (including some of the more popular ones) are likely to produce substantial results. Avoid the temptation to subscribe to a particular course simply because "everyone else is" or because "the company has made an elaborate presentation at your high school or because you have seen or heard advertisements promising results.

2. Seek a one-to-one or small group environment with a specific, seasoned SAT expert (not merely "someone on staff"), and be sure to inquire as to the number of years that the instructor has coached SAT.

3. Plan on several months' training (typically one or two sessions per week). Most students simply cannot assimilate SAT skills in a few short sessions or in an accelerated program of daily coaching.

4. Be committed to a concerted practice regimen... Input increases the probability of a desired outcome.

5. Beware of programs that "guarantee" results. Some courses may promise a small increase in score that may sound substantial to an unsuspecting client. Does the guarantee really mean, however, that a client will get a full refund if the student does not achieve a certain score, or does it simply mean that the student can sit through additional classes free of charge? The important point here is that there is no way that any SAT program can be certain that a student will show marked improvement.

6. Be suspicious of courses eager to accommodate a student's every whim. While such "feel-good" programs may appear to be client-oriented, they are probably not results-oriented. Remember that students will typically do as little work as they are permitted to do, so it is up to the instructor to maintain a reasonable amount of control and to expect a specific level of compliance.

Although I certainly allow for a limited amount of flexibility, as a general policy I will not work with students unwilling to accept all of the terms listed above. Experience has taught me that any student who does not commit to a strict protocol will simply waste time and money and will occupy an appointment slot that a serious student could use. For students eager and willing to put their shoulders to the wheel, however, SAT prep can be a remarkably rewarding venture. The key is to shop carefully, be selective, and work hard.