Fatal analytical flaw? Use it anyway

I read the blog Box Turtle Bulletin nearly every day and frequently discuss postings by lead authors Jim Burroway and Timothy Kincaid. They started the site many years ago to pick apart scientific research that supports the anti-gay industry. So Jim Burroway spent a day examining a new study by Mark Regnerus that will appear in the Social Science Research journal that compares the grown children of straight parents with those of gay parents.

The good news is the study is based on a "national probability sampled population." This is the best way to get subjects for a study. It doesn't happen much is social sciences because it costs so much money. Grants of more than three-quarter million paid for the work.

And the bad news. Regnerus is horrible at proper data analysis. There is a proper control group of kids of straight parents in which the couple maintain their relationship for the entire time the kids live at home (up to age 18). But the same-sex families are not represented by couples who came together before the child was added to the family and stayed together for the duration. Regnerus felt that pool of respondents was too small.

So he added in the children of any parent who ever had any kind of same-sex relationship. As commenter Straight Grandmother noted, that means it is likely the child started life in a "mixed orientation marriage" where one parent is straight and the other gay. Those marriages rarely last. Which means the child probably experienced his/her parent's divorce. Burroway says that's like comparing apples and elephants. The analysis, and the conclusion, is fatally flawed.

As one might expect from the flaws, this analysis shows the offspring of gay couples is not as good as long-term straight couples and is roughly the same as stepfamilies and single-parent households.

The proper way to fix that flaw should have been to bring more subjects into the study. Yeah, that probably would have required another sizable grant, but as it is now, it appears the original three-quarter million grant was wasted.

Commenters wondered whether the paper was peer-reviewed before publication. I don't know the answer. But commenters went digging and found that the same journal contains reactions by other researchers. Here is one of them, agreeing the analysis is flawed.

Importantly, one cannot clearly link having a lesbian mother (or gay father) with any of these outcomes. As stated earlier, the group is comprised of young adults who experienced multiple family structures, not only a same-sex parent household (indeed, some of the respondents never lived with the mother’s same-sex partner). It is quite possible, for example, that many or most of the negative outcomes result from the divorce of the young adult’s biological parents that preceded the mother’s same-sex relationship.

As you might have guessed, the study was funded by anti-gay groups. Burroway reminds us that the source of funding doesn't always mean the study is biased (though we've seen it frequently is the case). One must look at the actual study and critique it directly -- which he did.

Straight Grandmother wrote directly to Regnerus to point out the flaws in his analysis. It may be a few days before she gets an answer. In his reply the author said the entire raw dataset will be available in the fall and other researchers can do their own analysis.

Even with its flaws, will the anti-gay industry use this study to proclaim that it proves gays don't deserve to get married? Silly question. Already happening.