In the weeks since DePaul announced that we were going “test optional” for freshman admission, I’ve received quite a few phone calls and emails with questions. Fortunately, most of them have been very positive, and many of the questions have been thoughtful and engaging.  As is often the case with Internet discussions, the ones who jumped into the fray early were negative, aggressive, and almost always ill-informed.  It’s clear that most had not read the announcement in its entirety.

The biggest sticking point seems to be our collective societal belief that the ACT and SAT are intelligence tests.  Even the testing agencies would tell you they’re not.  So, I’m re-posting an edited version of an earlier blog post I wrote, hoping it clears things up.  This is considerably shorter than the original.

The most fundamental problem in college admission is definitional: No one in college admission can really tell you what they’re looking for in applicants. Oh, we have a general idea, of course: Academically accomplished, intellectually curious, quick on the uptake, creative, a great writer or artist or musician.  You get the picture.  But we can’t really define “it.”

And yet, we have a test to help us find it.

We have other tests, of course: Suppose you are testing for leukemia in children. You know what the “it” that you seek is.  However, the test you design is very different than the one you might design for the academic “it.”  Let me elaborate.

Imagine a grid like this.  For each test you take, there are four outcomes: Either you have the disease or you don’t; and the test comes back either positive or negative.

If you don’t have the disease, but the test comes back positive (Square 1), you usually take another test, or have some additional medical exploration.  This false positive is inconvenient and costly, but not horrible.  Square 2 is bad news, but at least you know about the disease, and can seek treatment.  Square 3 is the most common, and of course the best news: You get a clean bill of health, and in reality, you aren’t sick.  But look at square 4: You have the disease, but the test indicates you don’t.  The “false negative” leads you to believe you’re  healthy, but you’re not.  It is the worst of all outcomes, and people who create medical tests strive to make this outcome as unlikely as possible, even at the expense of accuracy in the other squares.  You can thank our legal system for that.

Because  the academic “it” is nebulous, the creators of standardized tests, of course, have no need to minimize false negatives (presumably, it’s hard to sue when a test doesn’t detect something you can’t define in the first place). But this nebulous thing that the test can’t measure, and that we can’t define, is precisely the thing we’re trying to reward. So, college admissions officers attempt to minimize the risk of a false positive (Square 1): That is, selecting a student who doesn’t have “it,” whatever that “it” happens to be. After all, it’s possible you’ve faked your way through four years of high school by just acting smart, or so some people seem to think.  The test will tell us if you’re the real deal or not.

This is confirmation bias in operation.  Because the tests have low false positives, we see high scorers and believe they must have “it.”  Usually, we’re right (in other words, we don’t know a lot of high scorers who seem intellectually or academically challenged, so a high score confirms the GPA).  The problem comes when we extend that logic, and say that if you don’t score well on standardized tests, you must not have  “it.”  This is called the contra-positive, and it causes people to make logical errors all the time.

Think of it this way:

You can say, “If that’s a poodle, it must be a dog.”  Does it then follow that,”If that’s not a poodle, it can’t be a dog?”  Of course not.  Replace “poodle” with “high scorer” and “dog” with “smart.”  See?  Simple.

And there you have it: The SAT is prized simply because admissions officers believe–and perhaps because daily experience has confirmed to our satisfaction–that it has a very low incidence of false positives. But in trying so hard to avoid false positives, colleges and universities miss an awful lot of talented students who should be in college.

And we think that’s wrong.