Randomized Trials and Management “Science”
Vaughan at Mind Hacks raises an obvious point about business gurus: they never test their predictions. In this way, they are like the “experts” whose reputations are coming under attack more and more—for instance, in books like Super Crunchers and Expert Political Judgment, and blogs like Overcoming Bias. They are easy targets.
It fact, the field is somewhat worse that he describes, I think, since methodologically a lot of business gurus use the old-school sociological method: the case study. In other words, “I observed these people doing such-and-such. They group also had attribute A.” The enthymeme is: “If you want attribute A, you should do such-and-such.” Of course, biz writers rarely make that premise explicit. Why? Because doing so opens them up to the obvious charge Vaughan makes: “Where’s you’re evidence?”
So far, so correct. And yet, biz gurus have another function, which is to spread around ideas. Yes, they typically, even in the best biz reviews and biz books, given scant evidence for their claims. But the point is really to help people free associate, come up with novel approaches, and so on. This is what “consultants” do. Are they mere parasites? It’s an empirical question and I’ve not seen an answer to it yet. I suspect many are parasites but that from time to time they are extremely helpful.