“It is difficult to get a man to understand something, when his salary depends on his not understanding it.” –Sinclair Lewis

I’m not going to debate the value of standardized tests in admissions. I’m not going to go over the research that shows differences without distinctions (a 3.2 GPA for one group vs a 3.4 for another after a year of college, for instance) and point out how absurd the premise is. I’m not even going to ask whether any company led mostly by people who have never taught, led alone worked in education, can create one test for millions of students in 38,000 American high schools in 50 states with what seems like 38,000 different missions, and expect the results to be meaningful.

Today, I’m just going to ask a question. First, if you want to read the latest shill piece several people have sent me, written by one of the biggest shills for tests, be my guest. It’s filled with the typical puffery and cherry-picking of data to ostensibly scare people into believing that if their children don’t get into one of the hot colleges, their life is over, and that the only way to get into one of those hot colleges is to improve test scores. If you believe this, it will give you good reason to continue in your paranoia. If you realize it’s typical self-serving bullshit, I hope you’re immune. Either way, don’t say I didn’t warn you.

The author is a member of the NTPA (the test preppers, not the tractor pullers). Not surprisingly, he’s on the board, where his biography touts his exploration of 17 different departments in college. Elsewhere, his website says 14, but of course one can never trust a fisherman describing his catch.

Curiously, it was in going to his website that something seemed off. I seemed to recall that at one point his site had bragged about the success of his test prep company, but it’s nowhere to be found today. So I went to the Wayback Machine (the Internet one, not the Peabody and Sherman one), and found his page from the early part of the decade, and this paragraph (now deleted) jumped out at me:

Three years ago, my company’s average student score increased by four points on the ACT. I thought better was impossible (the national average with prep is a one point increase). Today, our average increase is six points. 

And that generated my question: How can you believe that a) standardized tests are predictive because they measure the academic preparation necessary for success in college while also believing b) that some coaching on the test can raise your score by, well, by a lot? (It’s impossible to tell how much a six-point increase really means because the scores are not raw but scaled: If you have 100 students, that gain could move you up between 6 and 26 places in the distribution, assuming a reasonable starting point). Are we to believe that student will do better in college because they took a test prep course? And not just any test prep course: One that apparently teaches you more in a couple of months than you’ve learned in the 11 or 12 years of education to that point? I’m skeptical, but maybe that’s just me.

A six-point (or even a four-point) improvement on the ACT would likely trigger some deep examination by ACT (or it would have, back in the days before the VC bros took over; I don’t know if they care anymore). But that’s neither here nor there.

Test prep people I know fall into two camps: Those who hate the test because they know it and realize what a hoax has been perpetuated on the American public; and those who love it because they believe their high scores are reflective of some level of superior intelligence. Either way, I have no concerns about making a buck off the people who can afford it; you pays your money and you takes your pick, as they saying goes. The teenaged next door neighbor I had 30 years ago could not understand why I’d pay someone to change my oil when I could do it myself; we all have different utility functions.

I just don’t like people being mislead, or scared into doing things because their emotions about their kids have been activated. Love tests, hate them, use them, don’t use them: I don’t care. Just don’t piss on my leg and tell me it’s raining.