Wednesday, February 08, 2006
Garbage In = Garbage Out
Upon entering any graduate program, those seeking a master's degree are required to take a class that deals with the methods of research. Not only does the class teach you how to do research, it also teaches you how to evaluate research. And one of the first things you learn is that shoddy methodology leads to unreliable results.
One of the most popular methods of data collection is the survey. It allows the researcher to gather a large amount of information in a relatively short amount of time. It can be done through the mail, by phone, or in a face-to-face setting. However, if the questions asked are not worded correctly, the data can end up skewed. Survey questions need to be unbiased in order to yield valid results. This is why polling data can vary so much from one company to the next and can often be unreliable. To completely evaluate the validity of a poll or survey, it is necessary to know which questions were asked, who was doing the asking, and how the questions were worded. Only then can a poll be deemed trustworthy.
With this in mind, I would like to take a look at the latest "census" distributed by the RNC. A friend of mine received this in the mail, recently. As it turns out, she is a registered Republican. However, this is only to be able to receive their mailings and to participate in their primaries. In truth, she a solid Democrat and a pretty damned sneaky one at that.
All of the questions on this "census" have three possible responses - Yes, No, and Undecided. Limiting the respondent's answers makes the data much easier to compile, but it often yields results that are less than accurate. Especially if the questions are worded in a biased manner.
The first category of the census contains three questions and is titled DOMESTIC AND INTERNATIONAL SECURITY.
With only three choices, the person answering the survey is put in quite a difficult position. Notice that right off the bat, the first question imply's that initiatives to promote safety and security are the sole property of George W. Bush. Naturally, every American citizen wants to be kept safe, so it's kind of hard to disagree with this one.
With the second question, the implication is made that you have to choose between individuals or organizations committed to further attacks on America or air strikes. Apparently, there is no diplomatic option. Notice also the veiled reference to September 11. By using the phrase "further attacks on America," they RNC is surreptitiosly invoking al Qaeda.
And with the third question, the assumption is made that everyone already supported the security measures taken. But once again, the reader is put in an awkward position. Of course we all want to be safe, but with only three possible answers, we are forced to say yes, I want to be safe; no, I don't want to be safe; or undecided. Not much of a choice when you think about it.
So far, the RNC has made it difficult for the average person to answer anything but yes to any of the questions.
The second section of the census is titled ECONOMIC ISSUES.
The death tax? That's a Frank Luntz creation. It implies that people are taxed just because a relative has died. Of course there's more to it, but the average person prbably doesn't know that.
The next question is another example of a biased question. The wording of it makes it appear as though Bush has sole ownership of job creating policy and it forces the reader to choose between supporting or opposing job growth. Every American I know supports job growth. This is a question worded specifically to yield a positive response and at the same time create a positive image for Bush.
The thrid question is yet another no-brainer. The reader is once again forced into a specific response. Either you're for cutting the deficit or you're for uncontrolled spending. Gee, that's a tough one.
So far, the RNC is batting 1.000.
The next section is titled SOCIAL ISSUES.
This is the red-meat portion of the survey. Notice the subject matter - religion, abortion, social security, and marriage. The RNC is targeting its base with these questions. You've got the religious right, the pro-lifers, the pro-privitization crowd, and the homophobes. All of them are wedge issues that the party has been able to use very effectively with its constituency.
Notice also the subtle inference that Bush is on the right side of these issues. When the questions include the phrase "do you support," it implies George's favorite sentiment: "You're either with us, or you're with the enemy." Of course nobody wants to deny aid to those in need and nobody wants to promote late-term abortions and everybody would like to save Social Security, so once again it makes it difficult to answer no to these questions. The FMA? That's a feeler question trying to determine whether or not Bush should jump on that band wagon.
Next section: DEFENSE ISSUES.
Even more red meat. Notice the wording of the first question - "have to serve under..." because Bush has said repeatedly the he would never give another organization control over our troops. And fighting terrorists? Once again, you're either with us... And finally, the last question. It's obvious that recent developments in Iran influenced the wording of this question. It plays to people's fears about the news coming out of a nation that Bush once declared a member of the axis of evil.
The rest of the questions had to do with party affiliation and donation information.
So there you have it. Sixteen questions, of which, fifteen were clearly biased and one (Federal Marriage Amendment) was attempting to determine popular opinion so that the party knows which band wagon to jump on. Based on the questions, this poll will result in purposely skewed data reflecting positively on the White House's policies and consequently creating a aura of support for Bush, himself. Of course the RNC will release the results with much fanfare and trumpeting and Bush will be hailed as a hero. But when you look at the questions and the choices, any logical person can discern what is really happening. It's just garbage in, garbage out. Don't believe the spin because now you know the truth.
One of the most popular methods of data collection is the survey. It allows the researcher to gather a large amount of information in a relatively short amount of time. It can be done through the mail, by phone, or in a face-to-face setting. However, if the questions asked are not worded correctly, the data can end up skewed. Survey questions need to be unbiased in order to yield valid results. This is why polling data can vary so much from one company to the next and can often be unreliable. To completely evaluate the validity of a poll or survey, it is necessary to know which questions were asked, who was doing the asking, and how the questions were worded. Only then can a poll be deemed trustworthy.
With this in mind, I would like to take a look at the latest "census" distributed by the RNC. A friend of mine received this in the mail, recently. As it turns out, she is a registered Republican. However, this is only to be able to receive their mailings and to participate in their primaries. In truth, she a solid Democrat and a pretty damned sneaky one at that.
All of the questions on this "census" have three possible responses - Yes, No, and Undecided. Limiting the respondent's answers makes the data much easier to compile, but it often yields results that are less than accurate. Especially if the questions are worded in a biased manner.
The first category of the census contains three questions and is titled DOMESTIC AND INTERNATIONAL SECURITY.
- Do you support President Bush's initiatives to promte the safety and security of all Americans?
- Do you support the use of air strikes against any country that offers safe harbor or aid to individuals or organizations committed to further attacks on America?
- Do you continue to support increasing the amount of security at airports, train stations and all government buildings including monuments and museums?
With only three choices, the person answering the survey is put in quite a difficult position. Notice that right off the bat, the first question imply's that initiatives to promote safety and security are the sole property of George W. Bush. Naturally, every American citizen wants to be kept safe, so it's kind of hard to disagree with this one.
With the second question, the implication is made that you have to choose between individuals or organizations committed to further attacks on America or air strikes. Apparently, there is no diplomatic option. Notice also the veiled reference to September 11. By using the phrase "further attacks on America," they RNC is surreptitiosly invoking al Qaeda.
And with the third question, the assumption is made that everyone already supported the security measures taken. But once again, the reader is put in an awkward position. Of course we all want to be safe, but with only three possible answers, we are forced to say yes, I want to be safe; no, I don't want to be safe; or undecided. Not much of a choice when you think about it.
So far, the RNC has made it difficult for the average person to answer anything but yes to any of the questions.
The second section of the census is titled ECONOMIC ISSUES.
- Should the Inheritance or "Death Tax" be permanently repealed?
- Do you support President Bush's pro-growth policies to create more jobs and improve the economy?
- Do you think congress should focus on cutting the federal budget deficit by reducing wasteful government spending?
The death tax? That's a Frank Luntz creation. It implies that people are taxed just because a relative has died. Of course there's more to it, but the average person prbably doesn't know that.
The next question is another example of a biased question. The wording of it makes it appear as though Bush has sole ownership of job creating policy and it forces the reader to choose between supporting or opposing job growth. Every American I know supports job growth. This is a question worded specifically to yield a positive response and at the same time create a positive image for Bush.
The thrid question is yet another no-brainer. The reader is once again forced into a specific response. Either you're for cutting the deficit or you're for uncontrolled spending. Gee, that's a tough one.
So far, the RNC is batting 1.000.
The next section is titled SOCIAL ISSUES.
- Do you support President Bush's initiative to allow private religious and charitable groups to do more to help those in need?
- Do you support the law, passed by the Republican Congress and signed by President Bush, that bans partial-birth abortions?
- Do you support the President's efforts to save Social Security for future generations?
- Do you think Congress should pass legislation on the Federal Marriage Amendment?
This is the red-meat portion of the survey. Notice the subject matter - religion, abortion, social security, and marriage. The RNC is targeting its base with these questions. You've got the religious right, the pro-lifers, the pro-privitization crowd, and the homophobes. All of them are wedge issues that the party has been able to use very effectively with its constituency.
Notice also the subtle inference that Bush is on the right side of these issues. When the questions include the phrase "do you support," it implies George's favorite sentiment: "You're either with us, or you're with the enemy." Of course nobody wants to deny aid to those in need and nobody wants to promote late-term abortions and everybody would like to save Social Security, so once again it makes it difficult to answer no to these questions. The FMA? That's a feeler question trying to determine whether or not Bush should jump on that band wagon.
Next section: DEFENSE ISSUES.
- Do you think U.S. troops should have to serve under United Nations' commanders?
- Do you agree that our top military priority should be fighting terrorists?
- Should the U.S. continue to work on building a defense shield against nuclear missile attack?
Even more red meat. Notice the wording of the first question - "have to serve under..." because Bush has said repeatedly the he would never give another organization control over our troops. And fighting terrorists? Once again, you're either with us... And finally, the last question. It's obvious that recent developments in Iran influenced the wording of this question. It plays to people's fears about the news coming out of a nation that Bush once declared a member of the axis of evil.
The rest of the questions had to do with party affiliation and donation information.
So there you have it. Sixteen questions, of which, fifteen were clearly biased and one (Federal Marriage Amendment) was attempting to determine popular opinion so that the party knows which band wagon to jump on. Based on the questions, this poll will result in purposely skewed data reflecting positively on the White House's policies and consequently creating a aura of support for Bush, himself. Of course the RNC will release the results with much fanfare and trumpeting and Bush will be hailed as a hero. But when you look at the questions and the choices, any logical person can discern what is really happening. It's just garbage in, garbage out. Don't believe the spin because now you know the truth.
