Can A Test Show How Purdue Students Grow? And Is It Even Worth It?
One of Mitch Daniels’ pushes as Purdue President has been to try to amass as much data as he can showing his school is, by a phalanx of empirical measures, a quality institution. And so, the man who instituted performance testing for K-12 teachers as Indiana governor now wants to implement a similar measure for college students.
“All we’re interested in, really, is how as a group Boilermakers are growing,” Daniels says.
There are a number of ways to measure that. But one that a few hundred schools across the country have adopted is an exam called the Collegiate Learning Assessment, or CLA+ test. Roger Benjamin runs the Council for Aid to Education – the group that developed the test.
“The student takes it at one time – either as an entering freshman as part of a sample or as an exiting senior,” Benjamin says.
Note the word “sample”. What Benjamin means is that a group of students in, say, next year’s incoming class, take the test. And then, four years later, a similar group – not the same students, mind you, but ones with like demographics – take the exam, to determine how much the class of 2019 has improved in its ability to reason and express itself.
But that notion irks Purdue doctoral student Fredrik deBoer, who’s studying these sorts of exams for his dissertation.
“The test does not compare one group of freshmen to the same group four years later when they’re seniors," deBoer says. "It’s not, by design, a longitudinal study. Rather, it compares the current crop of freshmen to the current crop of seniors.”
CAE’s Roger Benjamin says that’s right, but he advises schools how to correct for it.
“When they are developing or drawing the representative sample, they have to make sure to control for the entering competencies of the students," Benjamin says. "Either by getting their entering SAT scores and their high school grades or to make sure that background demographic characteristics on them are collected.”
So why do this?
“This really has teeth because the Obama Administration has proposed tying the creation of rankings based on value for college -- which would largely depend on these tests -- they would tie those rankings to availability of federal aid,” deBoer says.
Many education watchers see this as the latest front in a nationwide battle to determine which colleges provide the best bang for the buck. The schools that win get the most cash from the feds. So if Purdue is doing a good job, and no one interviewed for this story thinks it’s not, what’s the problem?
“This question of student motivation on the test – this is the fundamental challenge to the validity of the instrument right now," deBoer says. "There’s a lot of people who have come up with issues that they have with the test, but nothing has stuck quite as significantly as the motivation challenge.”
deBoer says at some colleges, the test – which is supposed to be done in an hour – is taking students less than a third of that time. That means the scores probably aren’t accurate, because students don’t feel the need to do well like they do on the SAT, for instance. Daniels has an idea for fixing that.
“One idea I heard that apparently works very well is to say to the student: ‘If you’ll take this short test coming in the door and you’ll take it at the end of your time, if you get a good measure, we’ll put it on your transcript and employers and everyone else will be able to see it. If you don’t want it on there, it won’t go on there, but it’s all upside,” Daniels says.
“I think it’s a little sad that it’s getting to the point where it’s not good enough just to have a degree,” says Roger Malatesta, who runs Wilmington, Delaware-based Professional Recruiting Consultants. He says such a certificate may matter to employers – but he still worries about how companies might read it – especially if they’re unfamiliar with such tests, which he thinks most firms are now.
“I think it could grow into a standard. I think it would be unfortunate unless it was well thought out," Malatesta says. "Because it could really eliminate a lot of people that could probably do the job even better than some of the ones that maybe passed it or did well with the certificate.”
Purdue faculty have asked the Daniels Administration for more time in assessing whether the CLA+ -- or any test – is right for Boilermakers. Daniels has given a couple extra months to a committee studying the issue, but says he wants an answer sooner rather than later.
It seems clear that schools across the country are adopting similar tests all the time. And recruiter Roger Malatesta says companies might soon follow suit for the same reason schools like Purdue are considering the test – because many of their peers are already on board.
“Does it become trendy, does it become ‘if Company A is doing it, do we have to do it?’ You know, one of those things. It could be something that just snowballs,” Malatesta says.
Fredrik deBoer points out Purdue’s four-year graduation rate is just north of 50-percent. That, he says, may be an even bigger challenge to the school generating data that would even be useful for employers.
“You’re going to lost students who transfer out. You’re going to lose students who drop out. You’re going to lose students who take more time in order to graduate. So that is a not insubstantial series of challenges to having a genuine longitudinal study,” deBoer says.
Purdue began piloting its test with a small group of freshman in this year’s class. There’s no word on when such an instrument might be administered on a larger scale.