Data Driven Decisions For Control or Cover

David Clemens of the National Association of Scholars has a great piece about the new "data driven" culture in education. Like him, I think if I hear one more person tell me that their process is data driven I am going to scream. Data, on it its own, is like the wind for a sailboat. It can be used to move you, but without sophisticated equipment and navigation knowledge it tends to just push you in one random direction. Like the wind, data can be manipulated to move you in any direction you like. The data itself is not as important as the person using it to steer.  When anyone tells you their process is data driven, take a good look at who is at the helm.  (to complete the sailing metaphor) Watch when they pull the tiller hard to windward, you are about to jibe and could get smacked in the head by the fast moving boom.

Higher Education and the Perfect Data Storm

David Clemens teaches English at Monterey Peninsula College, where he founded and coordinates the Great Books Certificate Program. 

Consensus has it that we are living in the Age of Big Data.  When our college president was hired, he declared himself “data driven”; during interviews for vice president of academic affairs, all three finalists announced that they, too, were “data driven” (though none could articulate a clear image of what higher education might look like ten years from now).  So what does “data driven” mean?  Every day, our digital helpmeets dump petabytes of data into our cringing neural pathways.  We are besotted with data; we’ve never had so much of the stuff.  But to be data driven sounds uncomfortably like Captain Ahab (who was whale driven).

The words “data driven” are gang members; when I hear them, I can be sure the words “outcomes” and “a culture of evidence” are slouching around nearby and will shortly make an appearance.  Often, data is announced (as if newly arrived from Mount Sinai) in totals, aggregates, medians, percentages, rates, multipliers—but then the data just piles up in corners and collects under the bed. 

Frankly, I don’t have much confidence in data’s probative value.  Even though digits and stats supply a comforting sense of measurement, certitude, and solidity, data alone is still the smallest particle of information, no matter how much of it accumulates.  Data by itself is inert, like Frankenstein’s monster, patched together and waiting for a lightning bolt.  Sometimes it waits a long time.  It may seem irrefutable, but until data is analyzed, it just lays there.  Remembering Christmas presents from his childhood in Wales, Dylan Thomas recalled receiving “books that told me everything about the wasp, except why."

“Data driven” is a gift from the vocabulary of business.  On his blog, Evan Miller, President, CEO, and co-owner of Hertzler Systems Inc., writes:

As one colleague put it to me recently: “Most people have tons of data everywhere you turn, but most of that data isn’t accessible or usable.” This is an important incongruity: We say we want to be data driven, but most of us are not.

He continues:

Data may be cheap but not usable and therefore of little value. Often we don’t agree on underlying assumptions used to classify or assign meaning to data so the data are not reliable or valid [my emphasis].

In these situations very talented people may spend hours and hours of precious time to cut, paste and scrub data so that it becomes usable. The result is expensive data that appears too late to provide timely guidance.

In education, data always arrives too late, like Inspector Clouseau, blundering into a scene, oblivious to what’s really going on or who the villain is.  The kind of information data yields is retrospective, not predictive.  Correlation, as we know, is not causation.  To this, I would add mathematization is not explanation. I just learned “mathematization” is among the “bottom 20% of lookups” in the online Merriam-Webster’s Dictionary; what exactly does this tell me? 

Education managers and institutional researchers gather bushels of data, look for patterns, devise schema from which they make models that they hope are predictive in order to guide decisions and behavior.  But error in any of these moving parts can create failure.  Incomplete or irrelevant data, patterns that are Rorschach blots, rickety schema, or Rube Goldberg models all leave data driven managers concussed by reality.

Even analyzed data is haunted by forging, fudging, trimming, and cooking, along with confirmation bias and egocentric thinking.  Data at my college concludes that we have a low transfer rate to four year schools, a big no-no these thrifty days.  Turns out the data only includes transfers to state schools. The data is blissfully unaware of transfers to private or out-of-state schools, yet I can name former students currently at Columbia, Saint Mary’s, Shimer, Willamette, Redlands, Mills, and California Lutheran.  So the picture painted by the transfer data is not remotely congruent with reality even though serious budgetary and program decisions will be based on its inaccuracy. 

While knowing full well data’s vulnerability, education managers cannot resist the temptation to be data driven because data absolves them of responsibility; to be data driven lets them say “the data made me do it” (hat tip to Flip Wilson).

As with so many things, Neil Postman was prophetic about the data tsunami.  Even before Big Data, he wrote:

Like the Sorcerer's Apprentice, we are awash in information, without even a broom to help us get rid of it. The tie between information and human purpose has been severed. Information is now a commodity that is bought and sold; it comes indiscriminately, whether asked for or not, directed at no one in particular, in enormous volume, at high speeds, disconnected from meaning and import. It comes unquestioned and uncombined, and we do not have, as [Edna St. Vincent] Millay said, a loom to weave it all into fabric. No transcendent narratives to provide us with moral guidance, social purpose, intellectual economy. No stories to tell us what we need to know, and especially what we do not need to know.

Without such narratives, we discover that information does not touch any of the important problems of life. If there are children starving in Somalia, or any other place, it has nothing to do with inadequate information. If our oceans are polluted and the rain forests depleted, it has nothing to do with inadequate information.

I am going to make a radical suggestion about data and higher education:  colleges and universities will be better served if they avoid kneeling at the altar of data and instead fill key positions with people driven by intuition, experience, values, conviction, and principle.  A good place to start would be looking for leadership guided by a transcendent educational narrative.