Why Your Leadership Development Programs Don’t Work

In my last blog, I briefly listed some of the causes of a failed leadership development program. Other than blindly chasing trendy practices, not using evidence-based developmental initiatives and trying to use “a one size fits all” approach for various organizational needs, one important reason many leadership development (L&D) efforts fall flat is because they fail to accurately access the effectiveness of the programs.

Without proper evaluation results, it is hard to convince stakeholders that these programs are essential. Meanwhile, due to a lack of visible outcomes, leaders are often unmotivated to participate or allocate limited organizational resources to L&D efforts.

However, when conducted incorrectly, the evaluation results can be misleading or simply not convincing enough. Thus losing the organizations precious opportunities to adopt effective L&D programs.

To evaluate the effectiveness of an L&D program, the key question that should be asked is: “What standards are we using to evaluate and measure success?” The choice of evaluation criteria can directly influence the types of evaluation outcomes, which can in turn lead to vastly different opinions on the overall effectiveness of the program.[1]

Professor Kirkpatrick, best known for his extensive work on evaluating training programs, brought up four training evaluation criteria: reaction, learning, behavioral and results criteria.^[2],[3]

Reaction criteria is about the participants’ overall attitudes toward the program. It is one of the most popular to measure. The attitudes measured within reaction criteria include, but are not limited to, satisfaction with various aspects of the program — such as content, format, venue, instructor/lecturer or amenity. This type of measure is popular for not being hard to come up with, quick to administrate, easy to analyze and simple to understand and communicate with stakeholders.

Although research studies have found a moderate relationship between reaction criteria and one’s subjective judgement of the effectiveness of training programs,1 they found little connection between how the trainees feel about the programs and the other evaluation criteria, such as how much they learned from the program (learning criteria), how well they can utilize what they’ve learned in their work (behavioral criteria) and how much the program will contribute to overall organizational performance (results criteria).[4]

Simply because you liked a training program doesn’t mean it has taught you very much. Neither does it mean that the program can contribute to your job performance or the overall benefit of the organization.

Therefore, to correctly evaluate an L&D program, we need to measure all types of criteria that are important for decision making. Most importantly, we should have these criteria in mind before developing or selecting the program. Conducting a needs assessment is an important first step for designing or choosing an L&D program.

The needs assessment can be conducted through evidence-based and scientific analyses of the organization (e.g., What is the organization’s strategic goal for the next five years? What abilities do the leaders need to have to help the organization reach these goals?), the job (e.g., What types of knowledge, ability, and skills are needed for each job area? What needs to be improved for each type of job within the organization?), and the people (e.g., Who needs to be involved in which type of L&D?). Meanwhile, a systematic needs assessment based on scientifically best approach (see my other blog on the scientific approach to building a leadership competency model) can help make sure the areas of improvement identified are accurate. Only when you know exactly what you need to improve can you know what the best is for you.

Results of the needs assessment can also help the L&D staff to track and quantify change and improvement in behaviors and organizational results over time, therefore providing solid and objective data for the effectiveness of certain programs.

Ask yourself: what kinds of leadership development initiatives is your organization is taking right now? Are they effective? How do you know?

[1] Arthur, W., Jr., Tubre, T.C., Paul, D.S., & Edens, P.S. (2003). Teaching effectiveness: The relationship between reaction and learning criteria. Educational Psychology, 23, 275–285.

[2] Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of the American Society of Training and Development, 13, 3–9.

[3] Kirkpatrick, D. L. (1976). Evaluation of training. In R. L. Craig (Ed.), Training and development handbook: A guide to human resource development (2nd ed., pp. 301–319). New York: McGraw-Hill.

[4] Arthur, W., Jr., Bennett, W., Jr, Edens, P.S., & Bell, S.T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88, 234-245.