David D. Perlmutter

Professor and Dean at Texas Tech University

Admin 101: Which Metrics Are Actually Useful and How Can You Tell?

Full vitae admin101

No one becomes an academic administrator already holding all the skill sets necessary for the position. One novelty for me was figuring out how to use the internal "dashboards." That term, adopted from the business world, refers to the various screens of quantitative data and analytic metrics — the numbers — that help me do my job as dean.

At any given university or college there is a dashboard somewhere that will claim — always keep that qualifier in mind — to tell you everything you need to know about some aspect of the institution. There are dashboards for all sorts of information: the projected enrollment for the fall semester, the faculty’s grant productivity, the list of students who may drop out in coming months, and, of course, the amount of your budget that remains for the year.

Learning how to use dashboards is vital, both to do your work on a daily basis and to ward off problems in the future. For new administrators the greater challenge lies not in learning how to read a particular dashboard but in knowing how to employ it — while retaining a healthy degree of skepticism and keeping in mind context and nuance.

The Admin 101 series delves into both the obvious and the behind-the-scenes aspects of college and university administration. This month’s column turns to the ubiquitous but rarely discussed dashboard as a key tool of leadership — and yet, conversely, a minefield as well.

Don’t expect much training in how to use dashboards. On my campus, like most, new students can go to the library and take a workshop on "how to look stuff up." They can learn how to use databases, online catalogs, and other interactive resources to find pretty much anything they will need for their studies in any kind of course or major.

Unfortunately, I have never found the equivalent, all-encompassing onboarding to train new department chairs and deans on how to find and use the wealth of internal data available about any institution. My discovery of where to look for this year’s budget numbers versus, say, 20-year-old patent information has always been serendipitous and haphazard.

Discovering which dashboards are where does not have to be completely hit or miss, though. As a newbie administrator, don’t be shy about admitting what you don’t know and want to learn. Longer-serving deans, veteran staff members, and heads of support units in finance, facilities, and institutional information and analytics are all tremendous sources. Consult them. Find out what they wished they’d known earlier and where to track that information.

Discern the dashboards that matter. An old line from the detective noir genre goes, "There are eight million stories in the naked city." Likewise, on any campus, there are probably thousands of individual metrics, numbers, statistics, and trends that are being tracked somewhere and can be pulled up on your screen. But some are more important than others. In your first semester as department chair, dean, or beyond, try to identify the primary ones you need to curate and consult.

The criteria for what will make your personal Top 10 List of Dashboards should be determined by the following factors:

  • First, which sets of information will you need to inspect most regularly? For a department chair, the list might include course enrollments and budget figures. If you’re the director of a Ph.D. program, you probably want to look often at the status of applications to your program and the overall picture of degree completion.
  • Second, which dashboards are the ones you are judged on? Everyone in administration serves at someone else’s pleasure. Sometimes, the "boss" is not directly stated but heavily implied — e.g., woe to the department chair who loses the support of his or her faculty. So be aware of the dashboards that the people you serve are watching. For a dean at a research university, for example, you can bet your provost is tracking federal restricted funding brought in by your college as well as your record of private and foundation fund raising.

The point is: Dashboards are not just for your use but for people to evaluate you.

Understand the context behind the numbers. As dashboards multiply, the facts and figures they represent get more complicated and the software itself becomes increasingly sophisticated. But those numbers rarely tell the full story and are only a starting point.

To take a prominent example, our college advising office subscribes to an algorithmic software package that we feel has significantly helped us increase our retention of undergraduates. It allows our academic advisers to track an advisee and receive a range of direct and indirect signs that the student may be in trouble and need intervention. Visually, the level of alarm is signified by red, yellow, and green coloring over the students’ records.

However, our advisers, department chairs, and undergraduate dean and his team do not rely on color coding alone to take action. Many students suffer academic setbacks for reasons that have nothing to do with their academic performance and instead are symptoms of money, family, social, and personal setbacks. The dashboard won’t tell us what those problems are, but it can lead us to start asking questions, get involved, and help those in need.

The dashboard, thus, is extremely useful but only if the teams of people that monitor it can work together to interpret and act upon the available and implied information.

Appreciate that sometimes the numbers are not accurate or useful. I once heard a dean at a major research university make a cynical remark that has turned out to be frequently correct: "No two dashboards tell you the same thing." Not infrequently, you will face contradictory data and interpretations.

I would add that the famous semiotic observation is equally true: "The map is not the territory." Just because something is exquisitely, colorfully represented by tidy numbers lined up in cells does not mean that the information is valid or applicable.

To take a recent example, my college used to use a truly beautiful, high-end — read: very expensive — software program that purported to show us the research and grant productivity of our faculty members in comparison with academics at other units of the university and at comparable programs nationwide. I actually enjoyed playing with the software and creating really interesting charts that tracked our success and progress.

The main problem: The software makers had to maintain such a massive data set that they could not guarantee their results were current within a few years. Even with its vastness, there were also serious questions about how comprehensive it was — in covering, for instance, journals in subdisciplinary fields. All of which is why the college’s faculty vetoed the program’s use in quantifying promotion-and-tenure standards and similar tasks.

The lesson here: In using any dashboard, try to understand how the information is created and where it comes from.

Monitor trade-offs and trends. Dashboards do not have to be merely passive portals that allow you to see what’s happening in some realm. One of the main takeaways of learning how to observe multiple sets of data points over time is that you can see how different decisions influence the outcomes. Does investing heavily in marketing result in a major increase in enrollment or credit hours? Has hiring faculty members in a hot area of research actually increased your grants procurement — beyond what these hires cost the institution in start-up and lab-maintenance expenses? Over the years, you can experiment to see the effects of different stimuli.

At the same time, dashboard will give you yet another lesson in one of the fundamentals of being a college administrator: Everything is a trade-off.

Budget dashboards will show you — in real time and in rectangular boxes — what happens when you spend more money on one thing, leaving you much less for something else. That is probably one of the toughest learning curves for professors moving into administration. As a faculty member you’re focused on your own work and the need for investment in it. In administration you’re giving up that singular focus to oversee dozens or even hundreds of different projects and programs. Dashboards provide an eagle-eyed view, and it may not be a pretty one.

I am not in love with dashboards, but I cannot do the dean’s job without them, or at least not very effectively. And the same goes for any department chair, provost, or vice president.

Universities and colleges today are more complex — and perhaps more fragile — than ever. Our decisions have consequences and must be as fact-based, data-driven, and realistic as possible. Yes, dashboards can be misleading, and we should be wary of being hypnotized by figures and charts that don’t mean much. Yet dashboards can also prove to be powerful tools for making rational decisions on behalf of the individuals, the institutions, and the missions we serve.

David D. Perlmutter is a professor and dean of the College of Media & Communication at Texas Tech University and an executive coach for hospitals and universities. He writes the Career Confidential advice column for The Chronicle. Browse the previous columns in this series here.

 

Join the Conversation

0 Comments

Log In or Sign Up to leave a comment.