A recent survey conducted by the Wisconsin Manufacturers & Commerce about perceptions people have of Wisconsin has sparked some debate on whether the survey will be able to gather useful data.
Robert Griffin, a professor in the College of Communication, and Jamshid Hosseini, associate professor of management, both have survey expertise, said the survey lacked key tools in gathering quality information.
“It didn’t lend itself to quantitative analysis,” Hosseini said. “(It) didn’t ask for level of education (or) income. This makes it difficult to figure out who the respondents are.”
The survey, called “Wisconsin state of mind: A national perception survey,” is part of The Future Wisconsin Project by WMC. The purpose of the survey is to figure out what people around the country truly think about Wisconsin.
“Getting people’s perceptions of the state of Wisconsin is a worthwhile endeavor so that efforts can be made to attract new residents, students, businesses and tourists,” Griffin said. “Unfortunately, the kind of survey they are conducting will most likely yield misleading information.”
Hosseini said if the survey incorporated more Likert scale questions, the team behind the survey would have gathered more accurate data.
The Likert scale is a 5-point scale that gives the respondent a range of interest from strongly agree to strongly disagree. Hosseini said the scale is used to gather quantitative analysis because it compiles a rating scale for every response. Yes and no or word association questions only allow for limited answers making it more difficult to track.
“The questionnaire itself is okay,” Hosseini said. “But a researcher wouldn’t be able to deeply analyze the data.”
Hosseini said the survey lacked the ability to give useful information to policy makers because it didn’t allow respondents to answer questions about what they didn’t like about Wisconsin.
“What does Wisconsin need to do to make it more attractive?” said Hosseini. “Or what kind of changes are you looking for?”
Griffin said he had an issue with the lack of probability sampling in the survey.
“The biggest problem is the lack of probability sampling because you can’t trust that the people who have answered this questionnaire represent the general public,” Griffin said. “Biases can occur.”
Griffin said although the survey is effective in that it is quick and simple, it lacks the ability to make a scientific analysis.
“The gold standard for surveys is to use a technique such as random sampling to proactively select some people from the larger population – let’s say the city, state or entire county – to answer the questions posed in the questionnaire,” Griffin said in an email. “The is the basis of a ‘scientific survey.'”
Griffin said the survey doesn’t allow respondents to skip questions, which forces people to answer questions that they don’t have answers to. This doesn’t allow the team conducting the survey to learn about what people don’t know about Wisconsin.
“I want to emphasize that the folks sponsoring and doing the survey seem to have the very best intentions, but my concern is that the data this survey would yield would not really work very well for them, and would probably be misleading,” Griffin said in an email.
Jim Morgan • Sep 18, 2015 at 11:23 am
I appreciate the professors interest in the project. The social media survey we are conducting is not intended to be a scientific sample that will result in policy ramifications. It is the first step in raising awareness about our state’s perception. We realize it will be anecdotal, and that is how we plan to use it. We do plan to do scientific surveying at a later date , but this is intended to simply be a conversation starter.
Dr. Robert J. Griffin • Sep 18, 2015 at 8:34 am
Another vexing problem with the survey is that people are not told before they begin, as they should be, what their rights as respondents are. This survey appears anonymous, but at the end, people are asked to give their names and email addresses if they want to be entered into a drawing for a prize. How will those data be kept confidential, and not associated with the answers given? People have a right to know that, but they are not given any answer to that.
It is doubly tragic that various university systems in the state are listed on the survey sponsor site as co-sponsors, since universities in particular should be demanding only the highest quality research from the get-go