Thanks Bill, much appreciated comments. I see Kirkpatrick’s and Phillip’s ideas as a useful framework for thinking about testing the effectiveness of a learning intervention. It simply is common sense at one level – a heuristic for thinking about why, what and when I might ask questions to evaluate. It’s precision as a ‘scientific’ model suffers from many different complexities, but its usefulness, in challenging the naievity of simple ‘reaction’ evaluation and offering other areas to consider, is in my view fantastic.

There’s also the issue of transfer of learning to practice/behaviour. It has been found that while 80% may be able to repeat what they’ve earned in a post session test, only about 20% manage to make the transfer. Post learning support, from mentors, coaches or work colleagues is therefor necessary to enable the translation and can move the success rate to 80%. But is that time and effort factored in when determining RoI?

So overall I think you do well to keep clear of the entangling RoI net!