The methods box below contains 13 passages of text that provide valuable insight into the poll and how it was conducted. The ability to identify and understand this type of information will allow you to be a better judge of the validity of a poll.

An explanation for the text displayed in bold is provided below:

Yourtown Daily News:  The organization that paid for or otherwise sponsored the survey. Be suspicious of surveys that do not clearly identify their sponsor, as well as those that are sponsored by special interest groups. This would include political horserace polls paid for by a candidate or party, or a survey on air pollution commissioned by an environmental group, company or industry association. It doesn’t mean that the results are wrong: It just means you need to be extra careful in examining the poll for evidence of bias. Also, there’s some evidence that respondents may answer some questions differently, depending on who is sponsoring the survey. An experiment conducted in the early 1990s by The Washington Post found that black respondents answered some racially sensitive questions about the mayor of Washington differently, depending on whether they were told in the introduction to the poll that the survey was being sponsored by the Post or by the University of Maryland. 

Telephone:   The survey was conducted by telephone. Most media surveys are conducted by telephone and not by face-to-face interviews. Studies suggest few substantive differences in responses to most types of questions, depending on whether they are asked over the telephone or in face-to-face interviews. Of course telephone survey would not reach the 2% of the adult population that has no telephone service. Another 10% rely exclusively on cell phones, with young people comprising about half of this group. The cell phone issue poses a real challenge for pollsters attempting to do surveys of young adults. But the best current evidence suggests that cell phones are not a serious obstacle to conducting political polls and most types of surveys of the general public—at least not yet.

Randomly selected sample: A sample in which every member of the target population—in this case, all adults 18 and older—has a known probability of being selected. Pure random samples are the gold standard of public opinion polling but are more difficult and costly to obtain than less rigorous and less reliable methods. Carefully done random samples are always representative samples; that is, taking into account sampling error, they closely approximate features of the populations from which they are drawn, such as the percentage of men, women, old people, young people, Republicans, Democrats and other groups in the population. Online surveys such as those featured on many internet web sites that invite people to participate may produce highly entertaining results, but they are unreliable at best and notoriously misleading.

1,509 adults 18 and older: The number of people interviewed for the survey, from which is computed the overall margin of sampling error. Most surveys restrict interviewing to people 18 years old or older, though some special projects include people much younger in the sample. How large a sample is large enough? There is no single “magic number.” Generally speaking, very small samples—a few hundred or less--are usually avoided because small samples have large margins of sampling error. It is also more difficult to reliably analyze key demographic groups such as young people, blacks or the college-educated in a small-sample survey.  By convention, major survey organizations typically interview between 1,000 to 1,500 adults for a national poll. Also by convention, few media organizations conduct polls or report the results of surveys with fewer than 500 respondents, though it is occasionally done—always cautiously, of course.
 
Continental United States: Here’s one of polling’s dirty little secrets: Very few national polls are truly national. To save money and time, most “national” poll do not include Alaska or Hawaii. State and local surveys should clearly describe the area and population from which the survey was drawn: residents of the city of San Diego, Dade County residents, registered voters in the state of Illinois, etc.  

English:   Most major surveys in the United States continue to be conducted only in English, which excludes non-English speakers from the sample.  This typically isn’t a serious problem for most political polls where most registered voters speak, read and write English even if it is not their preferred language. On national surveys, about 5% percent of households that interviewers reach on the telephone fail to produce an interview due to language issues, and in some parts of the country this percentage is much higher. It would be problematic, for example, to conduct surveys in parts of South Florida or in certain areas of the southwestern United States without having a Spanish-language version of the questionnaire and Spanish-language interviewers available. Many large survey firms maintain trained and qualified Spanish-language interviewers, and have interviewers fluent in other languages available for special projects.

February 5 to February 11, 2007: The dates the interviewing was conducted, called the survey’s “field period”. Field periods of several days or more allow the interviewing firm to use the best methods to collect a good, representative sample while those conducted in a single night require compromises that may impact the results. But longer field periods also increase the chances that events may occur that change attitudes while the interviewing is in progress—not a good thing, as attitudes before and after the event may differ and these differences would be lost when the results are aggregated and reported together. This is a particular problem in political surveys conducted before primary elections, where a candidate may withdraw and his or her support shifts to another candidate, or a new candidate joins the field. Also, surveys conducted immediately after a significant news event—such as “overnight” or spot news polls--may not reflect changes in attitudes that occur as the public learns increasingly more about the event.

known demographic discrepancies: An oversample of 500 blacks were interviewed, for a total black sample of 673 For this survey, more blacks have been interviewed than would have been questioned by chance alone. Such oversamples are designed to allow journalists or researchers to characterize the results of important subpopulations, such as African Americans. Since about 12% of the population is African American, researchers estimated before the project began that only about 180 blacks would be included by chance alone in the 1,500 national survey. (In this example, a total of 173 blacks were questioned in the national survey.) To boost the black subsample, other methods were used to obtain interviews with an additional 500 African Americans, for a total of 673 blacks. When the overall national results are reported, any oversample is “weighted back” or adjusted so it does not affect the accuracy of the overall estimates. In this example, the additional black respondents are adjusted so that about 12% of the weighted sample is African American.  If this adjustment were not made, blacks would constitute about 30% of the national sample—much too high.  And note: the margin of sampling error is larger for subgroups—in this case, black respondents—than it is for the overall results. Care should be taken in reporting results of subgroups smaller than 100 people—in this survey, college-educated Republican women younger than 25 years old likely would be such a group—as the margin of sampling error for such a small subsample would be very large.

Weighted:  “Weighting” is a statistical procedure used to bring key demographic variables in a sample in line with known parameters.  The most common weighting scheme is called “ARSE” weighting, an admitted unfortunate acronym that stands for the demographic variables age, race, sex and education. In national surveys, these four variables are adjusted so that the final weighted sample correctly reflects U.S. Census estimates for age and education groups, the races, men and women. These adjustments can correct for the fact that some groups, such as women and better-educated people, tend to be slightly over-represented in survey samples because they are easier to contact and interview. Weighting can include more or fewer than these four variables. If data are not weighted, make sure that the sample does not contain too many or too few people in key demographic groups, such as young people, blacks and those with less than a college education.

Plus or minus 2.8 percentage points for the full sample and plus or minus 4 percentage points for data based on blacks: The margin of sampling error for the 1,509 adults in the sample, and the margin of sampling error for results based on the 673 blacks interviewed in the survey. Margin of sampling error applies to the marginal results of the survey. For example, if this survey found that 51% approved of an action taken by Congress while 49% disapproved, the 2.8 percentage points would be added and subtracted from each of those numbers to estimate the range in which we are confident the actual values would fall if all adults in the country population had been interviewed. Note: margin of sampling error is typically reported in percentage points, not percent. There’s a big difference between incorrectly writing that the margin of sampling error for this survey is plus or minus 2.8% rather than 2.8 percentage points. For example, if 51% approved, about 2.8% of that number is 1.4 percentage points—or half the real margin of sampling error of plus or minus 2.8 percentage points.

in 19 out of 20 cases, the survey result will differ by no more than the stated margin of sampling error: This language alerts you to the fact that the confidence level for the margin of sampling error of 2.8 percentage points and a sample size of 1,509  is .95, which is expressed in the box as the equivalent fraction, “19 out of 20”. Together, the confidence level, margin of sampling error and sample size gives you an indication of how confident you can be of the results. In this case, you can be about 95% confident that repeated administrations of this survey would produce results that differ from the “true” value—if it could be known—by no more than the stated margin of sampling error. The .95 is a bit arbitrary… some researchers use a more rigorous .99 or a .90 standard. The more confident you want to be, the larger the margin of sampling error. Conversely, using a lower confidence level produces a smaller margin of sampling error. Either way, there’s a price to pay for the fact that the data come from a sample and not the whole population. Confused? Don’t be. By convention, virtually all pollsters have adopted the .95 standard in computing the margin of sampling error they report to the public. When a different confidence level is used, it should be clearly stated.

only one source of error: Margin of sampling error is not the only source of error in public opinion polls. It is, however, one of the few we have a simple mathematical formula to calculate. Question wording, question order, interviewer effects are other potential sources of error in polls—and sometimes contribute far more variability to survey results than sampling error. Remember: More surveys have been undone by margin of thinking error than margin of sampling error.

Allied Research: Who actually conducted the interviewing. Most survey sponsors typically hire research companies or field services to do the actual interviewing. These firms maintain staffs of permanent or temporary interviewers who are trained and supervised.  One important way to evaluate the overall quality of the survey, particularly one done by a partisan or special interest group, is to find out who did the interviewing. Many of these better-known local and national research firms, as well as survey centers associated with major universities, follow strict guidelines on how surveys may be conducted and reported. Journalists typically can have more confidence in surveys conducted by these firms, even when a partisan or special-interest group sponsored and paid for the poll. But as with any potential news source, take nothing for granted. Even the most respected polling organizations can make mistakes.