Public editor: The trouble with online surveys

Two readers wrote to take The Globe and Mail to task for its online survey last week on assisted dying. You see these daily questions on the homepage under "The Conversation" on the right-hand side about halfway down. Readers can click on yes, no, unsure/maybe on an issue in the news each day.

A reader in Ontario noted that last Wednesday morning, the question was about assisted dying. When he voted early that day, "it was running 80 per cent in favour after a few thousand votes." That percentage is roughly in line with what published polls have demonstrated in terms of Canadians' views. But by Thursday, he said, the numbers "showed 72 per cent against assisted dying. … I surmise that some group against this initiative got wind of your poll and had its members go to your website to vote. Can't blame them for trying to exert their influence, however if my surmise is correct your polls will fast become useless."

Then on Saturday, The Globe published a small item in the paper's Focus section titled Assisted Suicide that included the results of that survey. It said: "Most pollsters say that Canadians are greatly in favour of changing the law to reflect this point of view. But when we asked our online readers if they agree with assisted suicide, the response was very different, indeed." It said 71 per cent of online readers had voted No, 26 per cent had voted Yes and 5 per cent were unsure.

Wanda Morris, the chief executive officer of Dying With Dignity Canada, was quick to send a letter to the editor titled Responsible Journalism Please! Re: Assisted Suicide [sic] Globe Poll:

"I was appalled to read Saturday's article about the difference in polling results between the Globe's (online) poll and the results of statistically valid polls by reputable polling firms, of which last week's Ipsos Reid poll was the latest example.

"The results of the Globe's uncontrolled online poll do not reflect the opinion of Globe readers (as the article states) but of anyone who votes. Special interest groups routinely skew results by sending out mass e-mails to their supporters encouraging them to vote.

"Sponsoring uncontrolled online surveys and then using these results to cast doubt on statistically valid polls is at best opportunistic, at worst it results in real damage as public policy makers react to a distorted picture of reality. Disparaging Ipsos Reid's poll is like doubting a peer-reviewed scientific article published in Nature because it is contradicted by an entry in a flat-earth blog.

"We understand that the media likes to portray an issue as two-sided, it generates debate and debate sells papers. But there comes a time when truth needs to triumph over spectacle."

Ms. Morris was referring to a story from last week on an Ipsos Reid poll showing that more than 90 per cent of Canadians are in favour of assisted dying. The poll of more than 2,500 Canadians was conducted by Ipsos on behalf of her advocacy group.

The Globe's questions are not online polls, but surveys of readers' views, although both online and in the paper they are not clearly labelled. Online at least, when you see the question and click on your answer pro or con, there is a context that does explain that it is a survey of any and all readers who want to comment.

The paper version lacks that context. Normally that item in Focus includes a headline that says "You Said It..." but not this time. Last Saturday, it was simply titled "Assisted Suicide."

Ms. Morris is right that all online surveys are subject to being flooded by interest groups. Polling companies base their work on science and experience, at times weighting to balance demographic and other factors such as regional differences. The reputation of polling companies is based on their accuracy, especially around political polling done just before a vote. (As a sidebar, watch for lots of municipal polls next week.)

Online surveys allow the readers to state their view, but they are no better than hamburger polls for accuracy and likely even more subject to manipulation because it costs nothing to add your view. Their value is to allow readers the chance to offer her or his opinion on the issue of the day.

On the good side of the ledger, The Globe acknowledged the diametrically opposed difference in the results on assisted dying between its online survey and the Ipsos poll. On the negative side of the ledger, it was not clear in the paper what those numbers meant.

My concern is that the survey and the opinion poll are not equal – not even close – and given the apparent manipulation of the online survey, why publish it at all in the paper?