I had a boss who once talked about “sending ideas out to the universe.” It’s poppycock, of course, if for no other reason than the universe doesn’t care. But that doesn’t mean it won’t work sometimes, I suppose. Yesterday, I tweeted this:
And this morning, the Twitter-verse unknowingly responded with this:
Which gave me–as only Twitter can–the ability to legally eavesdrop on three colleagues: John Lawlor, Deb Maue, and Chris Lydon. I know exactly what the conversation means: Measuring the quality of the educational products of a college by measuring the freshman class is like measuring how good a basketball team is by evaluating the average height of its players. It might work, on occasion, but that doesn’t mean it’s right. However, I responded to all three, suggesting that in fact graduation rates were inputs, in a weird sort of way.
That probably would have been the end of it, but it got connected to something I frequently see on my walk between Union Station and my office, about a mile each way. And it is especially common as it gets colder and people are eager to get indoors.
Here’s what happens: You’re standing at a corner with twenty other people waiting for the light to change, and someone notices no traffic coming, so they start to cross against the light. Others follow, even though the sign is clearly “Do Not Walk.” People come from the other direction, and, seeing the throngs of people crossing, assume the light says “Walk” and proceed to do so. But by this time, a car or truck or two has come barreling down the street, and I almost witness a pedestrian hit by a vehicle. What happened? It’s clear that the pedestrians have equated large groups of people crossing the street with a “Walk” sign. And usually, that signal is right; occasionally it’s not, and trouble ensues.
Thus, a blog post. And the end of Idea Block. And the head knocking resumes.
If we attempt to measure the value of an institution by its outputs (graduation rates) we are really just confusing signals, like pedestrians on Jackson Boulevard in Chicago on a cold morning. We’ve become so used to the mix up of inputs and outputs that we forget to look at the real signals; because in a way, inputs and outputs are the same things. Don’t believe me?
Take a look at this screen shot of IPEDS Data visualized. It shows test scores on the x-axis, and graduation rates on the y-axis. Note how they line up?
That alone would be enough to make the point, I think. But play with this visualization by clicking here. See if you can find any type of institution where it doesn’t hold: Urban, rural; any region; any religious affiliation; public or private. And notice two more things: The color of the dot (which represents a single institution) shows the percentage of freshmen with a Pell Grant. And the size of the dot shows the rejection rate: More selective institutions have bigger dots. Then, just for fun, filter to institutions that don’t’ rely on student tuition to manage the budget. Just pull the top slider down to 50%. See who’s left.
Then ask yourself: Is a high graduation rate a function of what goes on inside the institution? Or could it be a function of selectivity, test scores and family income (which are pretty much the same thing) and resources?
Would love to hear what you think.