"In the article "Young Have More Faith in UFOs than Social Security," a poll conducted by a group called Third Millenium was made into a news story to show concern about the Social Security system. Some key information about the poll is included, such as who sponsored the poll, when the poll took place, the sample error and number of respondents. Unfortunately, the article left out other key information that prevents the poll from being accepted as [representative of] popular belief."

"The headline in the Gainesville Sun screams "Young Have More Faith in UFOs than Social Security." The accompanying report offers results from a survey of 500 "Generation Xers" (Americans aged 18-34) and 500 senior citizens.sponsored by an advocacy group called Third Millenium. If the poll results are valid and accurate, the impolications are cause for concern. However, an informed consumer of public opinion would find that the Sun's report calls for more questions, not reactions."

It is not enough to mention these items. They must be explained and the poll assessed against them. What I needed to learn was how the lack of information might matter in interpreting the poll results. For each one of these criteria, there is a possibility of bias--a term that should be defined. For example, we need to know who conducted and sponsored the poll because they might intentionally bias the survey toward their point of view by question design or, if they have little experience, accidentally bias the survey. The use of non-probability methods of respondent selection may produce a biased sample--one that differs systematically from the general population about whom the poll is supposed to speak and is therefore unrepresentative. Even with a probability-based survey, poor implementation of the sample design--a bad sampling frame, low response rate, etc.--may introduce yet additional bias in the sample. We need to know the questions because many aspects of questionnaire design may bias the respondent toward a particular answer. Each method of contacting respondents has its own characteristic bias. The failure to incorporate filters or other devices might produce a high degree of non-attitudes. If many respondents didn't give an answer to a question--the don't knows or no opinions--that might produce misleading percentages.

It helps to provide examples from the survey and from the readings. This survey report made some rather sweeping characterizations of respondents and emphasized bizarre findings. Perhaps the question on the future of Social Security was constructed in a way that evoked skepticism while the UFO question was worded more gently. Perhaps the 46% who believe in UFOs are a small fraction of the entire sample because a large percentage asked the question may not have offered an opinion. There are other specific aspects of the survey that you could have mentioned. (A very good answer would have noted that the story does not define "senior citizens.")