Panel Paper: The Challenge of Collecting Better Data for Better Management & Leadership Strategies

Thursday, November 2, 2017
Hong Kong (Hyatt Regency Chicago)

*Names in bold indicate Presenter

Robert Behn, Harvard University


Public managers have many responsibilities for which better data would be valuable:

 (1) Improving their organization’s performance and thus producing better results (their core and most important responsibility).

 (2) Choosing or designing the policies that they will implement to produce these results. (Sometimes these policies are chosen by executive- or legislative-branch superiors.) 

 (3) Allocating limited resources (from dollars to people) among the components of their strategy to ensure they work effectively and efficiently both individually and collectively.

 (4) Choosing or developing leadership strategies to mobilize people (employees, collaborators, citizens) to devote energy and intelligence to make their policies contribute to better results.

For numerous reasons, however, it is easier to collect and employ data to make better policy decisions—(2) & (3)—than for leadership responsibilities—(1) & (4).

 This is because policy decisions can be based on data drawn from experiments that directly compare the relative effectiveness of different policies: Define a specific policy, implement it with fidelity for a collection of randomly selected individuals or groups, do the same for a different policy (the current policy or other potential policies), and compare the results: Which policy do the data reveal was most effective in producing better results?

 We might be able to do the same for leadership strategies. Define a leadership strategy, implement it in one or several randomly selected public agencies, do the same for a different management strategy (the current strategy, or for multiple, potential strategies) and compare the results. Which strategy do the data reveal was most effective in mobilizing people to produce better results? For at least two reasons, however, it is difficult to employ a RCT to determine the comparative effectiveness of different leadership strategies:

 (A) Because a management strategy is not an HR system, but a concept—an idea—it must (almost by definition) be adapted to differences in (a) organizational context, (b) political circumstances, (c) policy details, and (d) the desired results. This “imperative of adaptation” undermines a manager’s ability to implement the strategy with “fidelity.” How can a leadership strategy that a manager of an urban public-health agency employed effectively be also employed by a manager of a state agency providing energy assistance to the indigent, or by a commander of an air force base. (At a minimum, the definition and nature of such “fidelity” is, for each strategy, open to much disagreement and debate.) Even a manager who has effectively employed a leadership strategy in one agency will need to adapt it to a new agency, purpose, circumstances.

 (B) Two different leadership strategies might be based on the same insights about human behavior and motivation. The macro explanations of both strategies might be clear, easy to explain, and appear quite similar. Yet the specifics of implementation—as employed by two different managers (who each are unfamiliar with the other’s approach)—could be quite different yet effective.

Thus, this paper will explore and analyze strategies for collecting better data about the effectiveness of leadership strategies—data that can help public managers produce better results.