Remember this scene? (Sorry for the ad before the video.  Hang with it.)

A while ago, a piece in the Huffington Post kicked off an interesting discussion about the use of data in college admissions.  One part in particular seemed to be the focus of much of the attention:

A new focus for predicting a student’s level of interest in a particular college is related to the position the student has listed that college within the grouping he or she has identified on the FAFSA. Because students who place a college in position number one on the FAFSA are perceived as those most likely to enroll, they might be the first to receive financial aid awards, or might receive a personal phone call to discuss the award letter. Why? Because big data, which provides us that information, also tells us those “conversion” rates are high.

Well, duh. First of all, this is really not “new.”  People have been looking at FAFSA position long before there was a FAFSA. When I started in admissions in the early 1980’s, we knew it was a powerful predictor; in fact, the FAF (College Board form) competed with the FFA (ACT form), and many admissions officers preferred the former because you got a photocopy of the actual form, showing the colleges listed and the order; while the FFA only supplied the codes of the colleges (you had to look them up manually if you didn’t know them) and they were not in order.

Now, our National Admissions Association, NACAC, has taken the comments of a few concerned members, and told the membership this, via a list-serv post:

The US Department of Education recently announced that it is accepting comments from the public about how to improve the Free Application for Federal Student Aid (FAFSA) process.  Some NACAC members have been concerned that some colleges use the list of colleges that students supply on their FAFSA forms in an attempt to ascertain the students’ likelihood of attending the institution if accepted.

 NACAC is planning on submitting comments, which are  below, with the Department encouraging them to modify the FAFSA in one of the following ways:

1. Refrain from sending students’ college lists to the colleges; or 2.  Inform students that their  lists can be sent to colleges (preferably in alphabetical or random  order), and allow students to either opt-in or opt-out of such a  transmission.

 

To give you an idea of how powerful FAFSA position is as a predictor of enrollment, take a look at this:  These data are at least five years old (sorry competitors), and show our freshman class broken down by need and ability, into 32 groups.  The pies are colored by FAFSA position: Dark orange is first, light orange is second.  What you see is that the non-enrolling student group is all over the place: Lots of orange, of course, but just as much green, and a lot of red, pink and purple, too (7, 8, or 9.)

Non-enrolling students by FAFSA Position (data is at least five years old)

 

 

Now, take a look at enrolling students.  As a reminder, and an important caveat, we at DePaul have never, ever, ever used FAFSA position for any reason other than to project enrollment patterns of the group.  We have never used it to award more or less aid; we have never used it to decide whether or not to admit anyone.

Enrolling students by FAFSA Position (data is at least five years old)

See the difference? Of those who enrolled, the vast majority had us listed 1 or 2 as early as January.  But from a mathematical perspective, there’s something just as interesting going on.  Because the number of students who did not enroll is larger than the number who did, the predictive capacity of the first position is less important than it might seem.  See this graphic, which has three selected, sequential years in columns, and shows the enrollment outcome by FAFSA position.

What you see is that the predictive power of #1 is less important than it used to be; yield rate for those who list us first (the percentage of the pie that is gold) is falling.  You also see that they’ve been pretty stable for the other groups. (And yes, I have considered the possibility that our competitors have been looking at the students who list us first and have been awarding them more aid.  I’ve not convinced myself that it’s true, after looking deeper at the data.  For one thing, students–both those who enroll and those who don’t–are listing more colleges on the FAFSA, because they’re applying to more colleges, and applying for aid at more colleges):

Why does this matter?

Before I answer, there is an important distinction you need to be aware of:

One the one hand, we have in America College Group A: A small set of colleges and universities who are concerned about over-enrolling a freshman class.  These are places you can identify easily: High profile, extremely selective institutions who often require campus housing for all freshmen, and may generate as little as 5% of all core revenues from tuition and fees that students pay, due to rather large endowments.  They are, by and large, the places that get measured by very small changes in the admit rate, test scores, and even USNWR rankings, because the whole world has gone crazy.

Then, there is College Group B: Those colleges and universities who are concerned about under-enrolling the freshman class. They are lower-profile, less selective, more dependent on tuition, with smaller endowments, and they rely on tuition revenue to pay the bills.  Most institutions, in other words, by a long shot, although there is great range within Group B.

The institutions in Group A pretty much universally claim to be “need-blind” in admissions, although I’ve said many times they are not.  Still, let’s assume that they mean–and are completely honest about–admissions people not looking at the FAFSA prior to making an admissions decision. For them, this is not an issue, right?  This story from last year would seem to suggest they’re not looking at FAFSA position when making admissions decisions; and since they mostly claim to meet need, it doesn’t really matter. So, for once, let’s ignore them, if you think that’s even possible.

Suppose you’re working at one of the colleges in Group B.  And let’s play a nice game of chess (far less dangerous than Global Thermal Nuclear War, fortunately): You’re in charge of admissions and financial aid.  How would you react in each of these circumstances?

• You have a marginal admit who lists you first on the FAFSA.  Actually, you have a lot of them, because for your marginal admits, you’re the dream school.  How do you approach awarding them?  Do you give them a lot of aid to reward their interest and lock them up?  If so, you risk overspending on students who were likely to enroll anyway.  Or do you cut back on aid, thinking they’re coming anyway, and you could use the cash to build a lazy river?  If you do so, you risk losing large numbers of them to someone else.
• You have a top admit who lists you fourth.  Actually, you probably have a lot of these, too, because for your top admit, you’re probably more of a safety (this is the way the world works everywhere in Group B.)  Do you bump up money knowing that these students won’t come if you’re the same cost as their dream school?

If you were going to do this, of course, you’d say you need look at other indicators: Things we already refer to as demonstrated interest, another thing we at DePaul do use to project yield, but not to make financial aid decisions.  At a considerable number of schools in the top third of Group B, the importance of demonstrated interest is real (so much so that we now have consultants who coach students on how to demonstrate interest. See Goodhart’s Law.)

Some counselors have attempted to game the system in their own way to take this information away from colleges, by advising their students to list colleges alphabetically.  The result may be that a student who really wants to go to Yale but also applied to Amherst unintentionally signals both institutions incorrectly.

So, take FAFSA position away or don’t; it really won’t matter much in the end from my perspective, but it will kick off a series of actions by colleges that will introduce greater uncertainty in admissions or aid decisions; colleges have shown time and time again that when the game gets tilted, they respond, often in ways out of proportion to the problem.  You think admissions and financial aid decisions appear irrational now? Just wait.

In this movie, the computer will never play Tic-Tac-Toe against itself until it melts down.  It’s smarter than that.

Your move.