The Training Measurement Book: Best Practices, Proven Methodologies, and Practical Approaches

Advanced Praise for The Training Measurement Book "This book is a must read for all learning executives! Derived from years of research with learning analytic best practices. Josh Bersin covers the entire spectrum with a comprehensive, complete, and credible approach to training measurement." — Frank J. Anderson, Jr. , president, Defense Acquisition University "Josh Bersin effectively tackles the hotly debated of area of training measurement, considered by many a topic shrouded in conceptual and statistical complexity. He takes a refreshing clear and simple look at how to effectively measure the impact of training." — Vince Eugenio , vice president human resources, Enterprise Learning SolutionsCardinal Health "From an apprentice to a master in the field, everyone will take away a full historical perspective, thought–provoking concepts, and practical resources. As always, Bersin′s work takes us beyond a popular approach, offers the next step in evolution of a concept at hand, and is relevant, contextual, and based on facts and real–world observable patterns." — Irada Sadykhova , director, Learning Strategies, Microsoft "Josh Bersin provides badly needed clear thinking and practical insights in a muddled field. Training practitioners would do well to heed his advice and stop wasting money on counterproductive and misguided evaluation approaches." — Robert O. Brinkerhoff , professor emeritus, Western Michigan University, and creatorof the Success Case Method "The approach moves training and development from an event–based activity to a continuous, actionable process founded on information—an advancement long overduein the industry." — Michael Echols , vice president, Bellevue University "If you are new to training, The Training Measurement Book gives you everything you need to get started measuring your initiatives. If you are many years into the measurement journey, the book will motivate you to embed measurement as a de facto part of how your training group operates." — Chris Moore , founder, Thinq and president, Zeroed–In Technologies, LLC

Introduction 1 The Challenge of Training Measurement 1 Need for New Measurement Models 3 Structure of This Book 6 Methodology for This Research 9 A Personal Comment 10 Chapter 1: General Principles of Training Measurement 13 1. Measurement Should Deliver Actionable Information 13 2. A Measurement Program Should Not Be Designed to Cost–Justify Training 20 3. Measure Training as a Support Function 22 4. A Measurement Program Must Meet the Needs of Multiple Audiences 26 5. Measurement Should Be a Process, Not a Project 27 6. The LMS Is a Foundation for Measurement 31 7. Dedicate Resources 33 8. Start Simply and Evolve Over Time 34 Chapter 2: The Pros and Cons of Using ROI 37 ROI Analysis Assumes That Training Is Treated as an Investment 38 In–Depth ROI Measurements Are Often Diffi cult to Believe 42 In Concept, the ROI of Training Should Be Extraordinarily High 44 It is Very Difficult to Correlate Outcomes Specifically to Training 46 How Do You Make ROI Actionable? 48 Use of "Potential ROI" During Performance Consulting 48 "Performance–Driven" Versus "Talent–Driven" Training 50 Don′t Let ROI Become "Return on Insecurity" 53 Chapter 3: Limitations of the Kirkpatrick Model 57 The Kirkpatrick Levels Are Not a Complete Model 58 The Model Incorrectly Positions the Levels as a Hierarchy 62 The Kirkpatrick Model Ignores the Role of Training as a Service–Delivery Function 63 Kirkpatrick Misses Other Operational Measures 66 One Alternative: The Six–Sigma Approach to Measurement 68 Another Alternative: The Success Case MethodTM 68 Chapter 4: The Impact Measurement Framework 71 The Business Impact Model 73 The Impact Measurement Framework 87 Summary of the Framework 103 Program versus Organizational Measures 103 How to Use the Measurement Framework 106 Summary of the Measurement Models 106 Chapter 5: Implementation: The Seven–Step Training Measurement Process 109 The Seven–Step Program Measurement Process 109 Best Practices in Implementation 123 How to Start 129 Chapter 6: Measurement of Business Impact 139 Simplifying the Problem 140 Use Line–of–Business–Specific Measures 141 Integrate with Performance Management Processes 148 Chapter 7: Measurement of Alignment 153 Caterpillar′s Planning and Budgeting Process 154 CNA Insurance Training Investment Model 156 How Do You Measure Alignment? 157 Chapter 8: Attainment: Measurement of Customer Satisfaction 163 The Six Sigma Approach 163 How Do You Operationalize the Measurement of Customer Satisfaction? 166 Chapter 9: Measurement Tools and Technologies 169 The Role of the LMS 169 Training Analytics and Advanced Reporting 172 The KnowledgeAdvisors Solution 174 CLO Dashboard by Zeroed–In Technologies 175 Chapter 10: The Journey Forward: Focus on What Matters 179 Appendix I: Case Study A: Randstad Measures Onboarding 185 Randstad Overview 185 The Business Problem 186 The Solution 187 Results 191 Lessons Learned 196 Next Steps 196 Appendix II: Case Study B: HP Develops an Integrated Measurement Process 199 HP Measurement Goals 199 HP Systematic Approach 200 Designed to Deliver Actionable Information 202 HP Solution: Standard Evaluation and Indicators 203 Lessons Learned 206 Appendix III: Research: The State of Training Measurement Today 209 What Do Training Organizations Consider Important? 209 What Do Training Organizations Actually Measure? 211 Budgets for Training Measurement 212 The Importance of a Business Plan 214 How Do Organizations Measure Alignment? 215 The Role of Performance Consulting 217 Satisfaction with LMS Systems 218 Is Enough Being Spent on Training Measurement? 219 Demographics from the Survey 220 Appendix IV: Examples of Learning Measurements 223 Caterpillar University ROI Analyses 223 DAU: Business–Focused Approach 223 Cisco Sales Training 226 Microsoft Learning 226 Sprint University 229 Appendix V: Specific Learning Measures 233 Appendix VI: Training Analytics Specifications 235 Introduction 235 1.0. Business Measures 235 2.0. Dimensions 239 3.0. Hierarchy and Aggregation 242 4.0. Computational Flexibility 242 5.0. Security and Roles 243 6.0. Dashboards 244 7.0. Alerts and Report Subscriptions 244 8.0. Extract, Transform, and Load 245 9.0. General 246 10.0. Creating Demonstration Scripts 246 Index 249 About the Author 259

Josh Bersin is the president and founder of Bersin & Associates, a leading industry research and advisory firm in enterprise learning and talent management. Bersin is the author of The Blended Learning Book from Pfeiffer, and The High–Impact Learning Organization and High Impact Talent Management .