Two different national organizations that conduct research on higher education did independent surveys asking faculty how well prepared they thought their students were. The results of these two surveys drew attention when they were reported in the Chronicle of Higher Education because the findings from the two surveys were very different. Researchers from Research Foundation A found that nearly 75% of professors said that their students were “seriously underprepared.” Researchers from Research Foundation B found that only 18.8% of the faculty they surveyed said that their students were “not at all prepared.” Survey-research findings can be expected to vary from one survey to another, but the large discrepancy found in these two surveys could make one wonder about the reliability and credibility of survey findings. Before reaching this conclusion, it is useful to consider several details of the two surveys. [Note: This question is based on a report from the NCRIPTAL Update, Spring 1990, Vol. 3, No. 1, pp. 2–3.]
A Who was asked? The original sample for Foundation A included 10,000 college professors who taught undergraduate and graduate students in all types of institutions. Of the original sample, 54.5% responded. Foundation B omitted research universities (25% of Foundation A’s sample). Foundation B had a final sample of 2,311 (62% response rate). Approximately 90% of the final sample were teaching introductory-level students. How might the characteristics of the samples surveyed by Foundations A and B affect the findings obtained in the two surveys?
B What was asked? Foundation A asked its respondents: “The undergraduates with whom I have close contact are seriously underprepared in basic skills such as those required for written and oral communication.” The responses for this statement were: strongly agree, agree with reservations, neutral, disagree with reservations, and disagree. Foundation B asked its respondents: “In their background preparation, students who enroll in this course are most typically. . . .” The response choices were: not at all prepared, somewhat prepared, very well prepared, and extremely well prepared. How might the nature of these questions affect the findings obtained in the two surveys?
C How were the results reported? The findings for the Foundation A survey (75% of students seriously underprepared) were reported in the Chronicle by combining the response categories “strongly agree” and “agree with reservations.” The findings for the Foundation B survey (18.8% of students not at all prepared) represented only respondents who chose the “not at all prepared” response category. How do you think the results might look if the Foundation A estimate included only the respondents who chose the “strongly agree” response?