A new study about the value of financial advice was recently released that apparently demonstrates the causal link between having a financial adviser and having more financial assets. Industry groups have been quick to trumpet the findings, but they should probably hold off on quoting it as the gospel.
"We now have irrefutable, scientific evidence that Canadians are better off financially when advised, and that advice creates lasting and measurable value for those who receive it," proclaims one bulletin to industry participants.
There's just one problem with that statement. It's completely refutable.
The research paper, "Econometric Models on the Value of Advice of a Financial Advisor," put out by the Centre for Interuniversity Research and Analysis on Organizations (CIRANO) examined responses to survey questions and reports that "on average, participants retaining the service of a financial advisor for more than 15 years have about 173 per cent more financial assets (in other words, 2.73 times the level of assets) than non-advised respondents…"
I'm sure that public relations and marketing managers at financial services firms are salivating at the thought of how they can milk that statistic, and indeed some already have.
The authors themselves identify previous studies that reinforce some of the skepticism on the value of financial advice, namely that it might come down to good client selection on the part of the advisers. In other words, people who have attributes more profitable for advisers are precisely the people advisers go after.
Or keep.
"Clients who get poor portfolio results, either because the advisor is poor or because they manage their money poorly and drain their savings, are more likely to split with their advisors. The study lumps these clients whose relationships with their advisors sour into the same category as those investors who have no financial advisor. These problems are serious and could undermine almost all of the study's conclusions," says Michael J. Wiener, a cryptographer and author of the finance blog Michael James on Money.
The CIRANO paper suffers from the same survivorship bias we see in mutual fund returns reporting. If a mutual fund has done so poorly that it gets shut down, its performance drag can also get removed from the group average. This serves to overstate the performance of the group.
In addition, due to the limitations of the depth of CIRANO survey questions, there is no differentiation between the value of advice from different advice sources. Advice from a bank teller is counted in the same category as advice from a financial planner.
I'm pro-adviser, but I'll excuse those who might skim the headlines of my columns and blogs and assume the opposite. I do believe that the vast majority of Canadians need good financial advice and that good advice is worth paying for. I also believe there would be an effect that can be measured, but the CIRANO paper doesn't show it to the degree that some have interpreted.
But don't take it from me. I spoke with Professor Claude Montmarquette, president and CEO of CIRANO and one of the authors of the study, and he indicated that the study "is absolutely refutable given the limitations of the data."
He confirmed that a survivorship bias exists in the study and that he would like to see a more formal longitudinal study intimately tracking the performance of advised versus non-advised groups over a long period of time.
When I asked him what he thought of the industry associations' characterizations of the research, he said, "We need a better study and a better paper before I would be comfortable with the way they are saying what they are saying."