Beware the Metrics System

This excerpt is from my regular column, Culture & Kibbitz, at The Clyde Fitch Report. You can read the entire post here.

Last week at a committee meeting (I sit on the board and nominating committee of an arts organization), we were asked what metric we should use to measure our progress on equity, inclusion and diversity. The organization strives to serve the entire complexity of its field and with a limited number of board seats to replace each year, assembling a proper slate can be likened to a jigsaw puzzle, where pieces can form many pictures instead of just one.

Metrics qua metrics are tricky: by trying to capture a complex phenomenon in a simple number, they tend toward the reductive and are more subjective than we think. The Czech economist Tomas Sedlacek has cogently critiqued the idea that economic measures are objective, because they are actually normative -- related to good and evil because we apply value judgments to a measured number; surpassing the desired metric is success, not meeting or surpassing it is failure. And to paraphrase Nate Silver in his consideration of data, metrics need a direct connection to specific strategic goals or they will be less effective and we may miss an understanding of the impact our actions have. Moreover, relying solely on supposedly objective metrics often incentivizes behavior in unintended ways that do not advance the underlying imperative.

Metrics to measure success at building diverse communities may quantify how a “picture” is changing, but, in and of themselves, metrics do not effectively measure success as they do not get to the heart of the matter. These issues require an ongoing and evolving conversation because what underlines them are questions of community -- from who participates to the way the group treats its members. Therefore, the snapshot of a simple metric, at best, can only indicate the state of a group at a given moment in time. While a snapshot can be informative, -- and when compared to past and future snapshots it can document change -- it exists outside the context of a continuum that has a starting point and a goal. Standing alone, a metric is only a data point floating at sea.

For this reason, I suggest that any metric aiming to measure the success of a group’s efforts toward equity, inclusion and diversity must begin with a clear articulation of the intentions and goals of that group. As with any strategic imperative, a clear concise and comprehensible statement of the organization’s goals and intentions should provide sufficient guidance for an institution if it persistently measures and evaluates this statement through self-examination and adjustment—which metrics can aid. Unless considered in this way, metrics are potentially harmful, becoming fixed goals resulting in judgments rather than signposts along a journey towards organizational fulfillment. While metrics can play a role in organizational dialogue, it is the dialogue that matters, not the metrics, especially for a strategic imperative like equity, inclusion and diversity, which is rooted in communal relationships and dynamics.

Read the entire post in Culture & Kibbitz at The Clyde Fitch Report here.