1-1banner
 
medpundit
 

 
Commentary on medical news by a practicing physician.
 

 
Google
  • Epocrates MedSearch Drug Lookup




  • MASTER BLOGS





    "When many cures are offered for a disease, it means the disease is not curable" -Anton Chekhov




    ''Once you tell people there's a cure for something, the more likely they are to pressure doctors to prescribe it.''
    -Robert Ehrlich, drug advertising executive.




    "Opinions are like sphincters, everyone has one." - Chris Rangel



    email: medpundit-at-ameritech.net

    or if that doesn't work try:

    medpundit-at-en.com



    Medpundit RSS


    Quirky Museums and Fun Stuff


    Who is medpundit?


    Tech Central Station Columns



    Book Reviews:
    Read the Review

    Read the Review

    Read the Review

    More Reviews

    Second Hand Book Reviews

    Review


    Medical Blogs

    rangelMD

    DB's Medical Rants

    Family Medicine Notes

    Grunt Doc

    richard[WINTERS]

    code:theWebSocket

    Psychscape

    Code Blog: Tales of a Nurse

    Feet First

    Tales of Hoffman

    The Eyes Have It

    medmusings

    SOAP Notes

    Obels

    Cut-to -Cure

    Black Triangle

    CodeBlueBlog

    Medlogs

    Kevin, M.D

    The Lingual Nerve

    Galen's Log

    EchoJournal

    Shrinkette

    Doctor Mental

    Blogborygmi

    JournalClub

    Finestkind Clinic and Fish Market

    The Examining Room of Dr. Charles

    Chronicles of a Medical Mad House

    .PARALLEL UNIVERSES.

    SoundPractice

    Medgadget
    Health Facts and Fears

    Health Policy Blogs

    The Health Care Blog

    HealthLawProf Blog

    Facts & Fears

    Personal Favorites

    The Glittering Eye

    Day by Day

    BioEdge

    The Business Word Inc.

    Point of Law

    In the Pipeline

    Cronaca

    Tim Blair

    Jane Galt

    The Truth Laid Bear

    Jim Miller

    No Watermelons Allowed

    Winds of Change

    Science Blog

    A Chequer-Board of Night and Days

    Arts & Letters Daily

    Tech Central Station

    Blogcritics

    Overlawyered.com

    Quackwatch

    Junkscience

    The Skeptic's Dictionary



    Recommended Reading

    The Doctor Stories by William Carlos Williams


    Pox Americana: The Great Smallpox Epidemic of 1775-82 by Elizabeth Fenn


    Intoxicated by My Illness by Anatole Broyard


    Raising the Dead by Richard Selzer


    Autobiography of a Face by Lucy Grealy


    The Man Who Mistook His Wife for a Hat by Oliver Sacks


    The Sea and Poison by Shusaku Endo


    A Midwife's Tale by Laurel Thatcher Ulrich




    MEDICAL LINKS

    familydoctor.org

    American Academy of Pediatrics

    General Health Info

    Travel Advice from the CDC

    NIH Medical Library Info

     



    button

    Wednesday, October 11, 2006

    Lancet Strikes Again: I admit, this headline caught my eye. 655,000 dead in Iraq is an impressive number. Then I read the first sentence and saw that the number was gathered by public health researchers and it lost some credibility. The American public health community has a decidedly left leaning cast to it. It is more politically homogenous than any other medical specialty. How homogenous are they? Well, you won't find statements like this on the website of any other medical speciality. One is obliged to assume that the researchers started with a bias.

    Then I read that it was published in The Lancet and I lost all interest. This is the journal that gave us the infamous MMR-causes-autism study and that published a similarly discredited tally of Iraqi casualities before the last American election. In the ranks of medical journals, I place them on a par with The Guardian.

    This time, however, the media is on to them:

    Robert Blendon, director of the Harvard Program on Public Opinion and Health and Social Policy, said interviewing urban dwellers chosen at random was “the best of what you can expect in a war zone.”

    But he said the number of deaths in the families interviewed — 547 in the post-invasion period versus 82 in a similar period before the invasion — was too few to extrapolate up to more than 600,000 deaths across the country.

    Donald Berry, chairman of biostatistics at M. D. Anderson Cancer Center in Houston, was even more troubled by the study, which he said had “a tone of accuracy that’s just inappropriate.”


    Kudos to the two New York Times reporters for taking the time to run the study by a couple of statisticians.

    Color the Washington Post skeptical, too:

    And neither does Michael E. O'Hanlon of the Brookings Institution, which also tracks Iraqi deaths.

    "I do not believe the new numbers. I think they're way off," he said.

    Other research methods on the ground, like body counts, forensic analysis and taking eyewitness reports, have produced numbers only about one-tenth as high, he said. "I have a hard time seeing how all the direct evidence could be that far off ... therefore I think the survey data is probably what's wrong."


    The full survey is here. The researchers spent two months canvassing households in various regions of Iraq asking about deaths in the family. Sometimes they were able to confirm the reports with death certificates, sometimes they weren't. They didn't ask if the dead were combatants or non-combatants. They were afraid to ask that question. Afraid for themselves and for those they were asking. They interviewed 40 households in each of their selected regions, then extrapolated the 600,000 figure from the number of deaths they had recorded in their interviews. The margin of error of +/-200,000 speaks for itself. It's not reliable.

    And sorry, but the defense that it's as soundly designed as can be expected for these kinds of public health surveys is a weak one. Retrospective, interview-based studies like this are poor designs. It may be the standard way of gathering data in the public health field, but that doesn't make it the best methodology, and it certainly doesn't make its statistics sound. For too long the field of public health has relied on these types of shotty shoddy numbers to influence public policy, whether it's the number of people who die from second hand smoke or the number who die from eating the wrong kinds of cooking oils.

    But what do the Iraqi's think? Here's one who is particularly livid:

    I wonder if that research team was willing to go to North Korea or Libya and I think they wouldn’t have the guts to dare ask Saddam to let them in and investigate deaths under his regime.

    No, they would’ve shit their pants the moment they set foot in Iraq and they would find themselves surrounded by the Mukhabarat men counting their breaths. However, maybe they would have the chance to receive a gift from the tyrant in exchange for painting a rosy picture about his rule.

    They shamelessly made an auction of our blood, and it didn’t make a difference if the blood was shed by a bomb or a bullet or a heart attack because the bigger the count the more useful it becomes to attack this or that policy in a political race and the more useful it becomes in cheerleading for murderous tyrannical regimes.


    UPDATE: From Dani in the comments section, the editor of The Lancet, expressing his opinion, to which he is certainly entitled. However, his obvious passion (is it necessary to shout when using a microphone?) casts more than a shadow of doubt on his ability to be unbiased in selecting articles for publication that cover the same topic.

    UPDATE II: The Lancet podcast defending the survey.

    UPDATE III: Much more via Tim Blair, including Lancet editor Richard Horton's assessment of peer review, the annals of The Lancet's various controversies, a statistician's analysis of the study, criticism from anti-war epidemiologists.

    UPDATE IV: More here.
     

    posted by Sydney on 10/11/2006 10:51:00 PM 35 comments

    35 Comments:

    aww you forgot to mention all the statisticians in those articles who support the methodology and the numbers. aww poor baby with a short memory.

    By Anonymous Anonymous, at 10:34 AM  

    So Iraq has almost the same rate of population loss as Japan had during WWII? Considering the last study these folks did right before the 2004 election, this is beyond BS. It borders on election tampering.

    G. Hamid

    By Blogger G, at 10:55 AM  

    Given that conflicts are not evenly distributed, and that the independent variable wasn't really independent (households on the same street are likely to have many casualties if they have any), the external validity of this study is near zero.

    By Blogger BizDoc, at 11:01 AM  

    What a lucid summary of everything that's wrong with the Lancet's latest attempt to meddle in American politics. (And, yes, I am one of those who thinks that the timing makes this a political attack.)

    By Anonymous Anonymous, at 11:44 AM  

    One question.

    650,000 dead - where are the bodies?

    John Fembup

    By Anonymous Anonymous, at 12:34 PM  

    Gawker,

    I criticized the reasons they gave for support, though I didn't quote them directly.

    By Blogger Sydney, at 12:56 PM  

    1) The Lancet claims that their active survey is more accurate than a “passive” system of counting media reports, morgue reports or other lists of the dead (and generally totalling about 50,000 dead in Iraq), which are often grossly incomplete in a war zone. This seems reasonable.

    2) To make sure people weren’t making things up, The Lancet asked for and received Death Certificates 80% of the time. Also reasonable.

    3) But The Lancet fails to see that if 80% of the deaths were recorded by Death Certificate, then a passive accounting of Iraqi Death Certificates should give an accurate accounting.

    4) The 95% confidence intervals are pretty tight for this study (meaning no more or less than that random sampling error is small). So could there be any explanation other than stupidity or bad faith to explain how their death figures could be an order of magnitude higher than the passive counts?

    By Blogger DWPittelli, at 1:15 PM  

    Silly, hey...anyone have any thoughts on this. My best friend sent it to me and I'm curious. Sorry to get off topic, didn't see anything from Sid on the subject.: http://www.technologyreview.com/read_article.aspx?id=17597&ch=biotech&ch=biotech

    By Anonymous Anonymous, at 3:14 PM  

    Sorry, the link appears to be giving me trouble. It's about a new nano blood clotting technology. See below.

    http://www.technologyreview.com/
    read_article.aspx?id=17597&
    ch=biotech&ch=biotech

    By Anonymous Anonymous, at 3:16 PM  

    One question.

    650,000 dead - where are the bodies?

    John Fembup

    According to the CIA world factbook, roughly 2,465,149 americans died last year. One question: Where are the bodies?

    Ha! Put that in your pipe and smoke it Mr. Statistical Researcher Epidemiologist.

    By Anonymous Anonymous, at 3:20 PM  

    g
    It is election tampering. The lead authors of the 2004 study are the same. One of them came right out and said the timing of that article was deliberate, according to the AP. Does anyone read The Lancet for its science anymore? Thought not.

    By Anonymous Anonymous, at 4:16 PM  

    So this is where we're at today: Johns Hopkins/MIT release a peer-reviewed study in the one of the oldest and most respected peer-reviewed medical journals in the world, using standard accepted best practices. They study comes to conclusions that are somewhat counter-intuitive, and very politically unpalatable.

    And what do we get?

    Does anyone read The Lancet for its science anymore? Thought not.

    Got another question: Did Dani ever "read the Lancet of its science?"

    Classic stuff, really.

    By Anonymous Anonymous, at 4:37 PM  

    "Then I read the first sentence and saw that the number was gathered by public health researchers and it lost some credibility."

    Yes, we must have only proletarian science, comrades!

    Regards,
    T. Lysenko

    By Anonymous Anonymous, at 4:41 PM  

    Who mandated John Hopkins U to estimate the death toll of the war in Iraq? Can't Iraqi count?

    By Anonymous Anonymous, at 4:57 PM  

    It is election tampering. The lead authors of the 2004 study are the same. One of them came right out and said the timing of that article was deliberate, according to the AP.

    I followed the link and read the story. It doesn't appear to say what you claim. Where do one of the authors say the timing was deliberate?

    By Anonymous Anonymous, at 5:08 PM  

    Red right hand,

    The ninth paragraph down:

    The work updates an earlier Johns Hopkins study _ that one was released just before the November 2005 presidential election. At the time, the lead researcher, Les Roberts of Hopkins, said the timing was deliberate. Many of the same researchers were involved in the latest estimate.

    I remember him admitting it at the time, too.

    By Blogger Sydney, at 5:23 PM  

    Yeah, I saw that later after a second read. I posted a comment to that affect, but for some reason, it didn't show. Here's the mea culpa:

    Ah...that was the 2004 study. Sorry Dani, my bad.

    By Anonymous Anonymous, at 5:32 PM  

    The researcher sounds like a man with a small n.

    By Blogger kentuckyliz, at 5:57 PM  

    I have to say I quit reading the Lancet when it got into the habit of releasing press reports before the actual articles. I would say this was around ten years ago or so.

    Nothing like coming in to the phone ringing when one has no idea what it is patients really want to talk about. Of course, they are not the only journal to do this, only the one that seems to like the press the most.

    Why would you ask if I had ever read the Lancet, exactly?

    By Anonymous Anonymous, at 6:40 PM  

    Also, since I am in a surgical specialty, the Lancet became less relevant to me as the years rolled by. In fact, I just finished up my board recert, and not one article came from there. Out of scores of articles.

    By Anonymous Anonymous, at 6:43 PM  

    Wow. This is a rather good selection of bogus arguments, considering the small number of comments. We've got the 'it's a biiig number!', 'where're all the bodies', 'previous article was discredited', ...

    By Anonymous Anonymous, at 8:53 PM  

    Here is another example of why the Lancet is more than a little biased. Watch the editor, Robert Horton, at his finest. Seriously, we're supposed to think this article is legitimate?

    By Anonymous Anonymous, at 9:15 PM  

    Dani, beat me on that YouTube video of Richard Horton's raving at an antiwar rally three weeks ago.

    Everyone, please listen to the EDITOR IN CHIEF of The Lancet. You will never read this journal the same way again.

    PS: do UK doctors have a code of ethics? Does it condone this kind of circus act?

    By Anonymous Anonymous, at 10:21 PM  

    Also, since I am in a surgical specialty, the Lancet became less relevant to me as the years rolled by.

    If you can't pound the facts, pound the table. Congratulations on your surgical career. I'm a bricklayer. Both you and I are about equally qualified to address the study's methodology. So do you have something interesting to say about the actual study or is this all political posturing?

    By Anonymous Anonymous, at 2:25 AM  

    Well, ibc, if you couldn't assume I knew nothing becaue of my cutesy moniker, now you can assume I know nothing because I am a surgeon!

    The point of my posts has been to highlight the poltical orientation of the Lancet, a position which is both unethical and unparallelled among modern peer-reviewed journals, and as such makes all of their articles suspect. Instead of responding to the evidence I have provided, you attack me.

    Thus strengthening my argument that the evidence is not what the Lancet (or evidently, its supporters) really cares to examine thoroughly, but rather, they see what they wish to see in order to support their version of the truth.

    By Anonymous Anonymous, at 9:12 AM  

    The point of my posts has been to highlight the poltical orientation of the Lancet, a position which is both unethical and unparallelled among modern peer-reviewed journals, and as such makes all of their articles suspect. Instead of responding to the evidence I have provided, you attack me.

    Sorry, but ad hominems, bluster and hand-waving do not a rebuttal make. If you want to provide "evidence" you should at least demonstrate that you've a) read and b) understand the study. That would be more illuminative than posting youtube links of political rallies.

    Attack the study--it's not perfect by any means--not the messenger.

    By Anonymous Anonymous, at 1:21 PM  

    ibc

    I get to choose what issue I wish to bring up- to whit; that the Lancet is biased. At 4:37 yesterday, you decided to address me regarding this topic. You have disregarded my claims, but you have not refuted them.

    Creating a strawman argument regarding the merits of the study itself does not confuse me, nor anyone else reading this thread. The Lancet has problems, and it is perfectly reasonable for me to attack the messenger.

    By Anonymous Anonymous, at 3:32 PM  

    The Lancet has problems, and it is perfectly reasonable for me to attack the messenger.

    Look, this is just silly. You've attempted to discredit the study based on an ad hominem attack on the Lancet and one of the study's principals. You've made no verifiable claim on the study. The fact that you *think* you have says a lot.

    Of course, this is pretty much par for the course for ideologues on either end of the political spectrum, and one of the reasons most on the far-right here in the US won't pick up a newspaper or read a contemporary work of non-fiction that wasn't written by Ann Coulter. Funny how the list of "acceptable" sources of information just keeps getting narrower and narrower.

    Hell, you probably think Chomsky's work in linguistics is rubbish just because he's a frothing loon on any other subject. Or Teller's work in physics, &tc...

    I'm sure you're a very nice person, and a hard-working surgeon. But your "argument" is a bouquet of fallacies.

    By Anonymous Anonymous, at 4:03 PM  

    "According to the CIA world factbook, roughly 2,465,149 americans died last year. One question: Where are the bodies? Ha! Put that in your pipe and smoke it Mr. Statistical Researcher Epidemiologist."

    Ha??

    Are you laughing because you think no one will notice that you chose to attack the questioner, not respond to the question?

    First, the answer to your question is that each deceased American can be found - and counted - right where they were buried. The count is a matter of public record, all the gravesites can be inspected, and there is no need to hire researchers who will use a sample of 2,200 gravesites in order to "extrapolate" the total of 2,465,000 dead.

    BTW, if there were 650,000 dead in Iraq, there would certainly be multiples of that number wounded. Maybe I missed the part where Johns Hopkins counted perhaps 1,500 actual wounded and extrapolated oh, say, 1.7 million suggested wounded. Or maybe that calculation was out of scope.

    My opinion? I think you are entirely too trusting of credentials and process and are not exercising your common sense about the purported result. You do get points for your vehemence however.

    John Fembup

    By Anonymous Anonymous, at 5:01 PM  

    Well, this pediatrician trusts The Lancet a great deal more than I trust you, and The Lancet is a great deal less biased than you are.

    However, a surgeon--a practitioner of a specialty that famously disregards evicence-based medicine in favor of anecdotal tall tales--would be the person one would expect to slag a public health study.

    Congratulations for talking over your own head.

    By Anonymous Anonymous, at 5:47 PM  

    Doc Amazing,

    Not every study that sees the light of publication is well designed. This is true even with the top journals such as JAMA, The New England Journal of Medicine, The British Medical Journal, and yes, even The Lancet. It's important that physicians read the articles that appear in their journals with a critical eye rather than swallowing their press-released claims with credulity the way the lay press does. I'm sure that you can think back to studies that appeared in Pediatrics only to be refuted a few years later by studies of better design, and perhaps less bias. (Everyone has a bias, even researchers who aren't politically motivated or pharmaceutically compensated.)

    By Blogger Sydney, at 8:05 PM  

    In case anyone hasn't re-read the original post, Sydney has put up some updates.

    By Anonymous Anonymous, at 10:53 AM  

    "Given that conflicts are not evenly distributed, and that the independent variable wasn't really independent (households on the same street are likely to have many casualties if they have any), the external validity of this study is near zero."

    I think your intellect is near zero. This is EXACTLY how cluster based sampling works, and this is EXACTLY how it has been used in Kosovo, Angola, the Congo etc. Of course it can be used in such a situation, because you have 2 levels of randomization, of the clusters and the population within the clusters.

    By Anonymous Anonymous, at 3:59 PM  

    I'd like to say that I truly feel sorry for anyone defending the methodology of this survey.

    They evidently missed the part where it has a FIFTY PERCENT MARGIN OF ERROR.

    If the survey were in fact reliable, and the methodology used in fact equally reliable, you'd be looking at a margin of error more along the lines of FIVE percent.

    And for those of you extolling the virtues of cluster-sampling, let me ask you a related question. How often do you see later facts prove a cluster-sampled survey to have had a margin of error greater than reported by the survey?

    That's a serious question, not a talking point. I'm curious.

    This survey seems to me invalid by its own admission - reporting the median figure of a fifty percent margin of error as "the outcome" seems absurd on its face.

    Since so many seem to be so vehement in their defense of this survey, and the journal - which I freely admit I don't read - which published it, would any of you like to explain to me why, exactly, we should consider a study with that great a variance in its possible results to be valid, or even worthy of note?

    I'm honestly curious; I'm not a statistician, but I don't recall ever having heard of a study with that big a margin of error that was given any credence whatsoever, or considered to have any statistical viability. I'd really like to know what evidence, exactly, you're using to justify your belief in the validity of a survey with such an enormous range of possible results; more specifically, why you agree with the median figure, as opposed to the lowest figure, which seems at least marginally believable.

    By Blogger Xeno, at 5:36 PM  

    Xeno: I'm curious.

    Lot's of good stuff to beat your head against here, including what a confidence interval is, and what it signifies.

    By Anonymous Anonymous, at 12:58 PM  

    Post a Comment

    This page is powered by Blogger, the easy way to update your web site.

    Main Page

    Ads

    Home   |   Archives

    Copyright 2006