Wisconsin survey questioned for data accuracy

A recent survey conducted by the Wisconsin Manufacturers & Commerce about perceptions people have of Wisconsin has sparked some debate on whether the survey will be able to gather useful data.

Robert Griffin, a professor in the College of Communication, and Jamshid Hosseini, associate professor of management, both have survey expertise, said the survey lacked key tools in gathering quality information.

“It didn’t lend itself to quantitative analysis,” Hosseini said. “(It) didn’t ask for level of education (or) income. This makes it difficult to figure out who the respondents are.”

The survey, called “Wisconsin state of mind: A national perception survey,” is part of The Future Wisconsin Project by WMC. The purpose of the survey is to figure out what people around the country truly think about Wisconsin.

“Getting people’s perceptions of the state of Wisconsin is a worthwhile endeavor so that efforts can be made to attract new residents, students, businesses and tourists,” Griffin said. “Unfortunately, the kind of survey they are conducting will most likely yield misleading information.”

Hosseini said if the survey incorporated more Likert scale questions, the team behind the survey would have gathered more accurate data.

The Likert scale is a 5-point scale that gives the respondent a range of interest from strongly agree to strongly disagree. Hosseini said the scale is used to gather quantitative analysis because it compiles a rating scale for every response. Yes and no or word association questions only allow for limited answers making it more difficult to track.

“The questionnaire itself is okay,” Hosseini said. “But a researcher wouldn’t be able to deeply analyze the data.”

Hosseini said the survey lacked the ability to give useful information to policy makers because it didn’t allow respondents to answer questions about what they didn’t like about Wisconsin.

“What does Wisconsin need to do to make it more attractive?” said Hosseini. “Or what kind of changes are you looking for?”

Griffin said he had an issue with the lack of probability sampling in the survey.

“The biggest problem is the lack of probability sampling because you can’t trust that the people who have answered this questionnaire represent the general public,”  Griffin said. “Biases can occur.”

Griffin said although the survey is effective in that it is quick and simple, it lacks the ability to make a scientific analysis.

“The gold standard for surveys is to use a technique such as random sampling to proactively select some people from the larger population – let’s say the city, state or entire county – to answer the questions posed in the questionnaire,” Griffin said in an email. “The is the basis of a ‘scientific survey.'”

Griffin said the survey doesn’t allow respondents to skip questions, which forces people to answer questions that they don’t have answers to. This doesn’t allow the team conducting the survey to learn about what people don’t know about Wisconsin.

“I want to emphasize that the folks sponsoring and doing the survey seem to have the very best intentions, but my concern is that the data this survey would yield would not really work very well for them, and would probably be misleading,” Griffin said in an email.