|
medpundit |
||
 |
 |
|
Wednesday, October 18, 2006Curious about the kind of people who would have the chutzpah to claim to a national audience that this kind of research was methodologically sound, I contacted Johns Hopkins University and was referred to Les Roberts, one of the primary authors of the study. Dr. Roberts defended his 47 cluster points, saying that this was standard. I'm not sure whose standards these are. Appendix A of the Johns Hopkins survey, for example, cites several other studies of mortality in war zones, and uses the citations to validate the group's use of cluster sampling. One study is by the International Rescue Committee in the Democratic Republic of Congo, which used 750 cluster points. Harvard's School of Public Health, in a 1992 survey of Iraq, used 271 cluster points. Another study in Kosovo cites the use of 50 cluster points, but this was for a population of just 1.6 million, compared to Iraq's 27 million. When I pointed out these numbers to Dr. Roberts, he said that the appendices were written by a student and should be ignored. Which led me to wonder what other sections of the survey should be ignored. It's the student's fault! Isn't it always? With so few cluster points, it is highly unlikely the Johns Hopkins survey is representative of the population in Iraq. However, there is a definitive method of establishing if it is. Recording the gender, age, education and other demographic characteristics of the respondents allows a researcher to compare his survey results to a known demographic instrument, such as a census. Dr. Roberts said that his team's surveyors did not ask demographic questions. I was so surprised to hear this that I emailed him later in the day to ask a second time if his team asked demographic questions and compared the results to the 1997 Iraqi census. Dr. Roberts replied that he had not even looked at the Iraqi census. And so, while the gender and the age of the deceased were recorded in the 2006 Johns Hopkins study, nobody, according to Dr. Roberts, recorded demographic information for the living survey respondents. This would be the first survey I have looked at in my 15 years of looking that did not ask demographic questions of its respondents. But don't take my word for it--try using Google to find a survey that does not ask demographic questions. Either Dr. Roberts and his colleagues didn't know how to set up a research survey correctly or they set it up deliberately to be inaccurate, knowing that few in the media or general public would know enough to challenge them. Or maybe they let the student design and carry out the whole study without supervision. Either way, it reflects badly on them. (By the way, my original post has been updated a few times with more links to questions about the survey.) UPDATE: More interesting background on the lead researcher, Dr. Les Roberts, via Tim Blair- he's a very active Democrat. Not that there's anything wrong with being a Democrat. It's just that it raises suspicion about the motives of both his methods and his timing in its publication. Wouldn't it be a different story altogether if the headline were not "Public Health Researcher Estimates Iraqi Death Toll at 650,000" but "Democratic Office Seeker Estimates Iraqi Death Toll..."? UPDATE II: Beware of politicians (and public health types) preaching morals. UPDATE III: There's also this interview with another author of the study, Gilbert Burnham: Burnham: This was a ‘cohort’ study, which means we compared household deaths after the invasion with deaths before the invasion in the same households. The death rates for these comparison households was 5.5/1000/yr. What we did find for the households as a pre-invasion death rate was essential the same number as we found in 2004, the same number as the CIA gives and the estimate for Iraq by the US Census Bureau. Death rates are a function of many things—not just health of the population. One of the most important factors in the death rate is the number of elderly in the population. Iraq has few, and a death rate of 5.5/1000/yr in our calculation (5.3 for the CIA), the USA is 8 and Sweden is 11. This is an indication of how important the population structures are in determining death rates. (You might Google ‘population pyramid’ and look at the census bureau site—fascinating stuff.) PajamasMedia: During the same period, Iraq is at war with Iran and itself. Public-health infrastructure was poor, although perhaps not as poor as today. Does it seem plausible to you that the baseline (or pre-war) mortality rate is accurate? Burnham: Yes as above. Yes as being the right number, and Yes as what we need it for—comparisons in the same households before the war. Not to mention when entire families are wiped out by a totalitarian government, no one is left to tell the tale. PajamasMedia: The Lancet Study comes up with a post-war mortality rate almost double that Saddam’s Iraq. In fact, it is roughly equivalent to the mortality rate in Hungary is 13/1000. Does that rate seem plausible, given Hungary’s superior infrastructure and almost 50 continous years of peace? Is it possible that both the pre- and post-war mortality rates are too low? Why not? Burnham: There are many old people in Hungary , 40% are over age 55 vs. 9.3% in Iraq over 55. That’s the difference. Doesn't 9.3% sound like an awfully small percentage? What happened to all of those Iraqis who would be in late middle age and old age now? Did they emigrate? Or did something more nefarious happen to them? PajamasMedia: Historical comparisons might be helpful here. 650,000 violent deaths is about 150,000 more than the number of soldiers who died (violently and by disease) during the American Civil War, a conflict which involved a population larger than Iraq’s, and lasted a year than the current conflict has been going on. There is nothing in Iraq that looks like Shiloh, Antietam, Gettysburg, Cold Harbor, etc. What makes you believe that Iraq is deadlier than the American Civil War? Burnham: What we are reporting is cumulative deaths over a 40 month period throughout an area of 26.1 million, not a 1-2 day battle field event. Maybe he was tired or rushed when he read that question, but he certainly didn't answer it. posted by Sydney on 10/18/2006 06:16:00 AM 1 comments 1 Comments:
Sydney wrote:
By |
|
