It starts with a question. The search for an answer leads you to LinkedIn, Facebook, blog postings and association discussion boards. There are endless articles and stories with “the answer”. Some are easy to identify as “too good to be true” while others are convincingly written. What should we make of all these claims for the latest and greatest methodology?
I have stumbled into this dilemma a few times myself. Recently, I ran across this article: posted by Valerie Strauss and written by Paul Thomas. This article provided some excellent questions to provide a critical eye to the claims made (in that case for education) when you hear about the latest and greatest in training and development.
Here are the slightly altered questions to focus on training & development of employees and am including a few illustrative bullet points on each one.
Are the claims and counter-claims framed within the perspective of the person making them?
Is there author bias? (example: writes about the greatness of micro-learning/ sells a micro-learning system)
Who sponsored the research?
Are training & development claims framed as “miracles”?
“This training increased sales by 300%”
Are the claims of educational quality expressed in terms of correlation or causation?
Increase in sales may increase with the training activity - correlation. A new bonus structure also went into effect - correlation. Find out what caused sales people to reach goals - causation.
Do the claims address participant populations being addressed?
The results of programs in a company flush with cash and resources that focuses on services may be very different from a manufacturing organization recovering from bankruptcy.
Do claims of training success address issues of scalability, selection, attrition, stratification, re-segregation of participants and away from business factors?
Example: A leadership program implemented with 5000 participants may have opportunities for projects, promotions, mentorship that an organization with 100 participants may not have.
Do counter-claims about current training start with fair and accurate characterizations of the position being debated?
Does the article spend more time painting a poor picture of a current state to make theirs appear better?
What are the experiences and credentials of the person making the claim?
It is like a resume - “worked for a Fortune 100” company may mean anything. I did telemarketing in high school for a Fortune 100 company - does that count?
Are claims supported with evidence - citations, hyperlinks or both?
The availability of independent research on claims, independent sources that are in agreement.
Is the only source the white paper written by that same person?
Great ideas and innovations are out there. We need to challenge ourselves to get past the hype and marketing to ask ourselves some of these fundamental questions before we ride the next big trend.