A medical journal used a questionnaire survey to determine whether a particular change in its format would increase i...

Emil-Kunkin on July 2, 2022

Hi Alicia,

The argument is flawed because the journal made a decision based on a biased sample. They only polled their readers, but to understand if a decision would increase readership, they would also need information about people who do not currently read the publication. C shows that there is no difference between the readers and non-readers. That is, the results they garnered reflect the sentiments of the general population, not just that of their biased sample.

A does not address this sampling bias problem. While a strong response rate would help us to know that our results about current readers are valid, it still doesn't solve the problem that we know nothing about people who are not current readers- and were thus not polled.

Vic04324 on September 28, 2023

Hi Emil,

I was wondering if you (or anyone) could explain how you knew the sample was only given to current readers. The stimulus never states to what population they administered the survey, only that the journal "... used a questionnaire survey to determine whether a particular change in its format would increase its readership." I picked A because I failed to read the other answer choices and identified C as the better answer. However, I wasn't sure if I missed something in the stimulus that would have alerted me to the population that was tested, or if this was deduced by the presence of C which provides better evidence than A by addressing both problems in this question (not knowing what population received the survey and not knowing what percentage of that population returned the survey).

Emil-Kunkin on September 29, 2023

That's a great question, and I realize that I was a bit off in my earlier explanation. We actually have no clue who they sampled from the passage. While I still think it seems quite likely that they did only survey their readers, I am not able to prove that from the passage! This is actually a pretty common reason why people get questions wrong, and a reminder to always be cautious about assuming things not in the passage.

That said, the flaw is that we don't know who they sampled. It's a possibility they only sampled their existing readers. It's also possible the sample was a perfect representation of their total potential readers. However, the fact we don't know leads to a possible issue, that the sample wasn't representative. Here C fixes that problem by telling us that even if the sample wasn't representative, it wouldn't have mattered since both the representative sample and the biased one would have yielded the same outcome.