I’ve been following some of the discussion about the Mormon Gender Issues Survey, and I thought I’d share some of the more interesting things I’ve learned.
Let’s look at the FAQ. Others have looked at some of the questions (and maybe I will later).
Q: Who are you?
While a person’s associations or perspectives shouldn’t automatically disqualify them from research or dialogue, we know that each of us is susceptible to bias. Diversity, of course, is foundational to countering this (which is why, for example, being peer-reviewed is so important).
What of the diversity of this particular group?
An M* blogger did some leg work, and had this to say (emphasis added):
That list doesn’t exactly scream diversity, which suggests to me that there would be trouble avoiding bias that would color the development of the survey, the deployment of the survey, and the use of survey results. After seeing this group, I would be wary of each of those steps in the process, though by itself this doesn’t mean anything is amiss.
I have two other thoughts on this list.
First, the commenter assumes a familiarity with John Dehlin. I wasn’t really aware of him 6 months ago, not having spent much time in those kinds of circles. For those who want to learn more about Dehlin’s Mormon Stories and his dialogue and research tactics, I highly recommend this paper.
Second, there are a number of ties to the OW movement. As you consider whether or not that is material, I suggest that there is an important distinction which the commenter unfortunately doesn’t really highlight. There is a difference between someone who is neutral on the issue of ordaining women and someone who is neutral on the group Ordain Women.
I imagine that most members of the Church would be in favor of the ordination of women were Church leaders to pronounce such a change. Likewise, I imagine that most members of the Church would be against the ordination of women were Church leaders to suggest that such a change is inappropriate.
Q: What do you hope to accomplish with this project?
A: We’re genuinely curious to know how members of The Church of Jesus Christ of Latter-day Saints think about issues related to gender. Yes, part of that is the question about female ordination, but we’re interested in broader issues, like whether members of the religion think some people are treated unfairly, whether something can or should be done about it, and whether they think things are changing or will change in the future.
I’m generally unsatisfied with this answer.
I can appreciate that someone is curious about what the survey results will be, but curiosity is not an end in itself. It’s a means to an end.
A number of people will eventually take this survey, and the researchers will net data in the form of some idea of what respondents think of female ordination and a number of “broader” issues. What then? What is to be done with the results? What is the next step? Emphasizing the original question, what do they hope to accomplish?
The way that this frequently-asked question is side-stepped concerns me, particularly when a great many on this team are associated with groups known for the way they encourage agitation and dissension. I wonder whether TBMs (true-blue Mormons) would be comfortable contributing to efforts at discrediting the Church or criticizing its current positions on those “broader” issues.
Q: Are you allowed to conduct a survey of Mormons?
A: Sure. We obtained IRB approval to conduct the survey. We don’t need the permission of The Church of Jesus Christ of Latter-day Saints since we are not contacting members through wards or official lists. People can choose whether or not they want to participate.
This is an interesting question.
First, of course people can survey anyone they want on anything they want as long as they don’t engage in illegal activity. In this case, there may be additional requirements if they desire to conduct surveys as representatives of the Church, but that doesn’t seem to be this group’s intent.
(By the way, the Church has its own internal research organization. There wouldn’t be any case where these researchers officially represented the Church in their efforts.)
Now, there are certain standards surveyors need to abide by if they want others to take their survey results seriously. I cannot survey my Facebook friend list and make sweeping generalizations about much of anything.
Well, I could, but no one would take me seriously.
Second, they mention their IRB approval as a way to (I suspect) bolster their credibility. I think that is intentionally misleading.
IRB stands for institutional review board. IRBs help ensure that
“appropriate steps are taken to protect the rights and welfare of humans participating as subjects in a research study. A key goal of IRBs is to protect human subjects from physical or psychological harm, which they attempt to do by reviewing research protocols and related materials. The protocol review assesses the ethics of the research and its methods, promotes fully informed and voluntary participation by prospective subjects capable of making such choices (or, if that is not possible, informed permission given by a suitable proxy), and seeks to maximize the safety of subjects.”
The survey has been approved by the IRBs of the University of Tampa and Georgia Southern University. But what exactly is the implication of that? One M* commenter said this (emphasis added):
“That this survey has been approved by the IRBs of The University of Tampa and Georgia Southern University does not mean that the methodology being employed (e.g., wording of questions, medium of data collection) is scientifically sound. All IRB certification means is that the rights and privacy of any human subjects involved (read: survey participants) is protected.”
Another M* blogger added:
“As for the approval of the IRBs, It’s like ISO 5000 – it’s an international standard for processes, and compliance means that your process is repeatable. But if you repeatable process turns out, for example, violin shaped objects without quality tone, then all ISO 5000 certification proves is that you will consistently get VSOs without quality tone.”
Clearly, their appeal to their IRB approval is misleading, at best.
Additionally, if their goal is to survey members of the Church of Jesus Christ of Latter-day Saints, how can they validate their sample if they don’t use official Church lists? The best they could do is to frame their results as being from “respondents who identify as members of the Church of Jesus Christ of Latter-day Saints,” and that is a significant difference.
Q: How do you plan to use the data?
A: Well, we’re academics. We tend to think in academic terms. So, we’re primarily interested in using the data for peer-reviewed, academic publications. But we have already been contacted to consider writing at least one book chapter based on the data and we plan to issue some reports based on the data as well.
Much like the earlier question on what they hope to accomplish, this answer is generally unsatisfying.
For example, what is the subject of the book they’ve been solicited to write a chapter for? Additionally, how can they even know that this will be an appropriate fit given that results aren’t even collected, let alone analyzed? This, perhaps more than anything so far, betrays a clear agenda with the survey,
And what of peer-reviewed publications? I’ll believe it when I see it. Their data collection methods alone would cause issues with that. For example, like we’ve already discussed, there is no way to guarantee that responses represent an accurate sample of Latter-day Saints (even Latter-day Saints in the United States, to say nothing of non-English speaking members or those without internet access). The channels they’re using to promote the survey cast a wide net, and there is no way to prevent the same individual from taking the survey multiple times or to prevent individuals from misrepresenting themselves or their background as active Latter-day Saints.
But then, perhaps appearing in peer-reviewed publications is not really a goal of theirs.
Q: Will you share the data?
A: Maybe… Why do you ask? More seriously, if you want access to the data you’ll need to contact one of the members of the Mormon Gender Survey Group and explain how you plan to use it. The group will then discuss it and make a decision.
This question and answer, more than any other, betrays something more sinister than the researchers let on, and should be alarming. The commenter I quoted earlier said this:
“That should set off all sorts of red flags to any researcher. That they would be unwilling to share their data openly—allowing it to be scrutinized by other academics—says everything. That they would even ask one’s motivations for wanting to look at the data collected (“Maybe… Why do you ask?”) speaks openly to the biases of those conducting the survey (correct or not, I cannot but infer that they would be unwilling to share data with someone whose opinion may differ from their own and who might go about discrediting their methodology and results).”
There are some major issues going on here. I’m not opposed to surveys, or commentaries on modern Mormonism, but these researchers aren’t even interested in scientific validity. They’re interested in a very specific narrative, and in exerting total control over the telling of that narrative.
I don’t even really care about that. I care more about their desire to disguise their intent.
They can wear their shirt, but we all know they’re Spider-Man (oh yeah, totally brought it back full circle).
So what’s the best thing to do in cases like these?
Ignore it. Don’t take it. Don’t share it. Every response, even supposedly favorable responses (which are difficult to achieve, given the question bias), will be used to strengthen that narrative.