The Numbers Game In Education

Thomas Friedman, author of The World is Flat and most recently That Used To Be Us, was interviewed by Jim Fleming on NPR's All Things Considered recently and had this to say about American education,

"Well, if you take the PISA test, which is the best kind of international comparative test on writing, math and science for 15-year-olds, we're right there in the middle of the pack with Slovenia.  We are not leaders in these math and science competitions or in the writing competitions.  We quote the latest sort of statistic, there's an annual kind of you know, math genius competition for college students.  And I think we have one school in the top ten, in Michigan.  And you know, you look at any international comparison test and we're now in the middle of the pack.  And a lot of people say well, that's because we have more diverse populations--no, they factor in all of that, okay.  I'm tired of having to explain like why we're in the middle of the pack.  How about we just come out number one and not have to explain anything anymore."

His remark is typical of those who believe we are in an educational crisis in this country. The PISA test in particular has been used  to demonstrate America's falling performance levels. We rank 24th in math and 17th in science out of 30 countries. But as we've already discussed, this is somewhat of a numbers game being played in the ranking.  Mr. Friedman is a smart man and a decent economist so he should be well versed in how statistics can be manipulated to make a point. In his statement he both acknowledges this fact and then dismisses it out of hand because it would be so much easier if we didn't have to explain statistical manipulation.

There are easy ways to achieve his desired number one status by next year. We could require a certain GPA in order to even take the PISA test. That way only our brightest students, who are obviously on the path to college, would take it and our scores would go up. Instead we encourage everyone to take standardized exams like the ACT and SAT and go on to college. In the last four years we have increased the number of students taking the ACT by 17%. Statistics tell us that this should lower the score as less advanced students are now taking the test. The ACT says, however, that the national scores have not changed in the last four years. That means the we actually have more students doing better now than four years ago.

The question he should be asking is why would anyone manipulate the statistics in the first place. Could it be, as Rahm Emanuel now famously said, "Never let a good crisis go to waste?" First you have to convince people there is a crisis and then you can do all kinds of things to fix it.

Such numbers manipulation is done nationally, and it turns out, locally as well. DESE reported earlier this year that Missouri was ranked 26th on our students' average ACT score. That would put us right in the middle of the pack and add to the evidence that our schools are in crisis and need of changing. It was decent support for our state getting into the Common Core Standards.  We were barely keeping our head above water.

It turns out, when you try to compare apples to apples with ACT scores, Missouri ain't doing so bad. In fact, we are 4th in the nation when compared with other states who have a similar percentage of students taking the test. Whew. Dodged that bullet. Let's pat ourselves on the back and call it a day.  Well, not quite.

There is always room for improvement. As Mr. Friedman says it would be nice to just come out number one overall. To do that, we could look at states like Massachusetts which, while only having 15% of their graduating students take the ACT, blew us away on all the measures.  Forty four percent of their takers received benchmark scores on all areas of the ACT (English Composition, Algebra, Social Science and Biology) compared to the national average of 25% and Missouri's average of 27%. Benchmark scores are the minimum score needed on an ACT subject-area test to indicate a 50% chance of obtaining a B or higher or about a 75% chance of obtaining a C or higher in the corresponding credit-bearing college courses. Perhaps we should adopt Massachusetts' standards and curriculum. If only improving test scores were that simple.

At least adopting MA standards would make some sort of logical sense. Look at who's succeeding and do what they do. Instead we have adopted Common Core Standards which don't look like MA standards.  They don't look like anyone's because they've never been done before. They've also never been tested before so we really don't know what they will do for our test scores. But at least we'll be able to compare our students scores against other states on a level playing ground, right?  Not exactly.

Since it now looks like there will be at least two versions of each of the assessments, a short and a long, it will be harder to compared state-to-state. In addition, each assessment is self adjusting, selecting harder questions for those who get answers right and easier questions for those who answer incorrectly. No two students will take the exact same test. That makes comparing student-to-student within the same classroom a difficult challenge, let alone comparing students between different states.

The numbers game in education will continue as long as there is money and power to be gained from it. Test developers, tutoring schools, educational "thought leaders", venture capitalists,  and a host of others who see an opportunity to make money or a name for themselves will continue to feed us statistics that tell us we need them; their money, their intellect, their policies. In the education numbers game, the only winning move, as they said in War Games, is not to play.