How Effective are Flu Vaccines?

An article in The Atlantic, based on research by Lisa Jackson, questions the conclusion that flu vaccines work. Here is the essence of her argument from a letter to the editor by Jackson and others in The New England Journal of Medicine:

In an 8-year study of a similar population of members of a health maintenance organization, we found risk reductions among vaccinated elderly persons during the influenza season to be essentially identical to those reported by Nichol et al. (Table 1).1 However, we also found even greater reductions before the influenza season.

Emphasis added. The lack of specificity suggests that those who get vaccinated are in better health to begin with than those that don’t. Other comparisons supported this conclusion.
Thanks to JR Minkel.

Interview with Professor David Jentsch about Not Taking Drug Company Money

Dr. J. David Jentsch is a professor of psychology at UCLA; his research area is psychopharmacology. I contacted him because Aaron Blaisdell told me that he had decided to stop accepting research money from drug companies. This is unusual; I wondered why.

1. What is your research about? What portions of it have been funded by drug-company money?

My own research over the past 12 years has focused on the etiology of mental disorders (how genetic factors influence brain chemistry and behavioral functions) and how psychoactive substances work to normalize behavior through working on those very pathophysiological mechanisms. In particular, I study the brain systems and molecular pathways in control of cognitive functions, with a very specific focus on using that knowledge to generate insights about cognitive enhancement for schizophrenia, addictions and AD/HD. I study rodents and primates.

I have received funds from drug companies for two reasons. 1) The companies appreciated my work and funded efforts to discover new mechanisms that might inform what they ultimately did. 2) The companies provided funds to my laboratory so that I could investigate how novel potential candidate mechanisms that they developed influence cognition in laboratory models.

When one does work like I do, one wants to know that information learned is moving from the bench to the real world. That always requires a connection to a drug company – they make drugs/universities do not. That being said, I’ve always been of the opinion that having the best and most rigorous academic labs undertake these collaborations is in everyone’s best interest (the quality of the work is ensured). In my case, this was always a tiny part of what I did; therefore, the quality was good, my objectivity was unquestionable and the answers were certain.

Because top scientists are increasingly withdrawing from collaborative partnerships (in part because of the negative attitudes about them), this work gets left to less competitive scientists whose objectivity may be less clear because they rely upon this type of support more heavily. I think that is quite unfortunate.

2. How does one get drug-company money for research?

Generally speaking, a company representative approaches you because of your reputation and invites you to propose a study to accomplish a mutual goal (see my answer to #1 above). A study design is drawn up, circulated and discussed and finally approved.

3. How much easier is it to get drug-company money than to money from other sources (for the same research)?

It’s hard to say. Fewer people receive drug company funds. If a company is interested in your work and approaches you, it’s not that difficult to obtain the funds. But it is difficult to be recognized to do this kind of work.

4. When did you start getting drug-company money for your research? If you’re comfortable saying how much it has been over the years (per year), that would help clarify the implications of your decision.

As a graduate student, the laboratory in which I trained participated in some studies. As a faculty member myself, I have participated in two such efforts. The total amount of funding I have received from pharmaceutical companies in all my years at UCLA (a total of 8 years) is less than the budget I obtain in a single year on my RO1 grant. It is not an immense amount, and it certainly is not the kind of funding that I would need to sustain my research program.

Because of the negative perception of these sorts of activities, it is not worth continuing to engage in them. I don’t require those funding sources. That being said, I find it a bit unfortunate. Again, it’s in everyone’s best interest if the TOP scientists did those collaborations to ensure their quality and rigor. When I don’t do them, it is possible that a less objective party does. Second, every concept I have about novel treatments that isn’t pursued because of lack of such a relationship is a potential delay in moving basic science to real use.

5. What are some examples of how the animal-rights activists publicized and complained about your use of drug-company money?

After the bombing [his car was bombed in March 2009], statements were made on the web and in the press by animal rights groups saying that people such as me used animals needlessly in a drug-company-fueled manic process of animal killing in order to get rich. As I already mentioned, this is not the case, if only because people like me often have relatively few such grants, and their size is not large (again, usually not larger than a single year of funding on an RO1 grant). Because of this, I simply decided not to take any such grants in the future.

6. The car bombing (on top of other attacks) led to the decision to stop taking drug company money?

As you can discern from the fact that I only have accepted two such awards in 8 years, I already placed a good number of criteria on accepting them. I wanted them to be only projects that I considered to be of very high scientific merit, and I wanted them to be logically and obviously related to our broader research projects.

Additionally, there is already a good deal of “negative perception” of research funded by drug companies within academic circles, and so I had already batted around the question in my mind about whether I should accept further awards. When the extremist attack on me happened in March of this year (2009), I had not had such an award in some time. That was not because I had taken a decision about the matter – simply that I hadn’t found a situation I wanted to pursue. At that point, the decision solidified.

7. Your decision to not take drug company money — what effect do you think it will have or hope it will have?

I am certain a situation will arise where I will have an idea about a novel therapeutic based upon my research that I will be unable to pursue without such a relationship to a company. What is more, the compounds in development by companies are not being evaluated by me, so they may well be evaluated by someone with a little bit less rigor and objectivity.

I believe strongly that the academic enterprise gives a crucial “objective” check on novel therapeutics when leading scientists who are not “dependent” on drug company money examine them. The alternative is that others who are more dependent, and therefore less objective, will do it.

Med School Interview Questions

Here is what Brent Pottenger was asked during a recent interview at USC medical school:

  • What drives/motivates you?
  • Describe a challenge you overcame?
  • Describe a fulfilling experience that made you want to be a physician?
  • Why USC?
  • What do you bring to the entering class?
  • What area of medicine are you interested in?
  • What would you do for health reform?
  • What do you do outside of school for fun?
  • If you could improve something about yourself, what would that be?
  • What are you looking for in a medical program?

The Price of an Unnecessary Operation

A few years ago, a Berkeley surgeon named Eileen Consorti, to whom I was referred by my primary-care doctor, recommended that I have an operation to repair a hernia so small I couldn’t detect it. I have already written about how she kept saying there was evidence such operations were beneficial but as far as I can tell no such evidence exists. (Dr. Consorti has yet to provide the evidence she still seems to think exists.) Okay, she overstated benefits. What about costs?

During a conversation about whether the operation was a good idea, I said operations are dangerous. I didn’t want to have one unless there was a clear benefit. She replied that nobody had died from anesthesia during one of her operations. But of course death is only one of the things that can go wrong. It turns out the general category of bad things happening during anesthesia is called undesirable events and the rate of undesirable events has been measured. In this study, the rate was 100-150 undesirable events per 1000 hours of anesthesia. My operation was simple; I estimate it would have taken one hour. So my chances of having something bad happening to me as a result of an operation without any clear benefit to me — but considerable financial benefit to Dr. Consorti — was about 10%!

In a discussion of the costs and benefits of the operation, she didn’t tell me this.

How to Eliminate/Prevent a Skin Infection and What It Means (continued)

A brief summary of my previous post is all I needed to do to cure/prevent a skin infection was buy more socks. Instead of buying 5 pairs every 6 months, buy 20 pairs every two years. That’s all. Costs nothing. No drugs. No special treatment of the socks. No special cycle on the washing machine. No following a hundred (or ten) instructions about how to avoid infection. Like my depressed friend, I had the reaction: Why didn’t my doctor tell me this? He didn’t tell me because he didn’t know, I realize. Why he didn’t know . . . is a harder question.

The whole practice of health care is called medicine, so focused is it on cure rather than prevention. There are medical schools, which turn out doctors. Schools of public health are the closest thing we have to schools based on prevention but they don’t even train nutritionists. Nor do they do experiments, in most cases. (They do little data collection besides epidemiology.) And they get much less money than medical schools. Scurvy and Vitamin C are the first examples of the new way of dealing with illness I’m talking about — finding the environmental deficiency and fixing that, which is inevitably extremely safe and extremely cheap. After the discovery of Vitamin C, similar examples were discovered and the broader term vitamin was coined. But I think there is a need for a similar term that includes non-vitamins. It would mean aspects of everyday life, food and non-food, that we need to be healthy.

Like Vitamin C, my discovery that more socks eliminates skin infection points to a cure/prevention agent that is perfectly safe and extremely cheap. So do all my posts about fermented foods. It costs basically nothing to let food ferment. You lose nothing and gain a lot. Yet bacteria are not vitamins — and it isn’t all bacteria we need, just the 99.999% that are harmless. (And other foreign stuff, like bee venom, can substitute for bacteria.) I began thinking there are non-food vitamin-like things (things we need to be healthy) when I discovered the effects of standing on sleep and morning faces on mood. So we need several things to sleep well, including morning light, and at least one thing for proper mood regulation. Insomnia and depression are non-infectious problems, like scurvy. We think of vitamins as preventing/curing non-infectious problems, so the analogy was obvious. And these examples (sleep and mood) involved the brain. So there were vitamins for the brain, you could say. But the socks/foot infection example and the fermented foods/many illnesses example both do not involve the brain and do involve infectious diseases and auto-immune diseases (which, although non-infectious, are quite different from scurvy). So the idea that there are bunch of extremely cheap, perfectly safe things we need to be healthy expands to cover more of health.

Vast amounts of money are spent on health research, much much more on the consequences of poor health, and truly incalculable suffering comes about because we don’t know what these things are. (Depression alone causes vast suffering. Now add to that poor sleep, autoimmune problems, much infectious disease . . . ) Yet because studying these things (a) will make money for no one, (b) won’t produce a steady stream of published papers and (c) is useful (= low status), they are nearly impossible to study.

How to Eliminate/Prevent a Skin Infection and What It Means

Several years ago, during a routine checkup, my primary-care doctor pointed to some white lines on my right foot. (Curiously only one foot had them.) Fungus, he said. I had a fungus infection. What should I do? I asked. He suggested over-the-counter anti-foot-fungus medications, sold in every drugstore.

I tried a few of them. They didn’t work. The problem persisted.

A month ago I noticed the problem had gotten much worse. Yikes. What had gone wrong? I realized that in the previous few weeks I had changed two things:

  • Instead of putting my wash through an extra wash cycle without soap (to rinse it better), I had started doing my wash the way the rest of the world does it. I had stopped doing the extra cycle because I was no longer worried about becoming allergic to the soap.
  • I had bought 5 new pairs of socks and had been cycling though 4 of the new pairs again and again (washing them between wearings, of course), ignoring the rest of my socks.

This suggested a theory: My skin infection was due to my socks. The infectious agents get on my socks and are not completely removed by the washing machine. They survive a few days on the shelf. To wear socks with the infectious agent already present gives the infection a boost. Maybe my new socks supported the infectious agent better than the socks they replaced.

Based on this theory, I did three things:

  • Resumed putting my wash through an extra cycle without soap.
  • Took off my socks earlier in the evening.
  • Bought 12 new pairs of socks and made sure every sock went a long time (e.g., 3 weeks) between wearings.

I saw improvement right away. (The morning after I wore new socks.) A month later, the infection, present for at least several years, is entirely gone. It took about a month for it to clear up completely.

The essence of my discovery is that the infectious agent could survive my socks being washed conventionally (in a washing machine) and live for a few days without contact with my feet. Whereas a few weeks away from my skin killed it. I have been unable to find this info anywhere else. A very minor discovery, but unlike the work that won the most recent Nobel Prize in Medicine, useful right now. Cost: zero. I would have had to buy new socks anyway.

In Cities and the Wealth of Nations, Jane Jacobs tells about a reporter interviewing someone in an oil-rich Middle East country (Iran?). During the interview the interviewee tries to cut an apple with a knife. The knife breaks. We can’t even make knives, the interviewee says. That’s how backward our economy is. To develop economically, MIT professors had advised his country’s government to build a dam, at great expense. The MIT advisors thought that building a dam would be good for economic development. They were wrong, it turned out. Jacobs thought it was telling that after all that money invested, the local economy still couldn’t make something as basic as a good knife. Many industrial processes require cutting tools.

This is the same thing. Preventing and eliminating infection is at the core of medicine, just as cutting is at the core of manufacturing. My discovery reveals that my doctor — and by implication, the whole health care establishment — failed to know something basic and simple about this. If they understood what I figured out, there would be no need for anti-foot-fungus medicine. A gazillion dollars a year is spent on medical research, medical schools and research institutes around the world are full of faculty doing research — and they haven’t figured out something as basic and simple as this.
Gatekeeper Drugs. How to Avoid Infection: Something I Didn’t Know.

Rent Seeking and Our Health-Care System

Does our health-care system (including researchers) engage in rent-seeking when they ignore simple cheap remedies, including prevention?

Here’s a simple example of rent-seeking. Some friends and I went to visit the Great Wall. On the path up to the wall was a man sitting in a chair. He demanded 30 cents to let us pass. There was no gate. He wasn’t a government official — just a man and a chair. There was a path to a goal. It was blocked unless we paid.

In the case of health there are many paths to the goal. Many ways to become healthier — many ways to relieve depression, for example. Prevention is one way, cure another. There are cheap cures and expensive cures. By ignoring prevention and cheap cures, the profession of psychiatry is  blocking those paths (by failing to clear them) and thereby forcing us to take their expensive path (dangerous drugs), usage of which they control. It’s more subtle than the man with the chair but it amounts to the same thing.

Rent-seeking is annoying. I was annoyed by the man in the chair. The rent-seeking of our health-care system is disguised, not easy to make out. This makes it less of problem for health-care professionals, such as doctors; I think few people are aware of it. (For example, most people with acne don’t realize it is probably caused by their food.) But my friend with depression was annoyed, deeply annoyed, when he learned of a simple cheap (partial) solution to his problem.

Gatekeeper Drugs: Drugs that Require Gatekeepers

A friend of mine suffered from depression. Like so many depressed persons, he went to sleep very late — maybe 3 am. I told him that was a very bad sign, no one should go to sleep that late. He starting going to sleep earlier and waking up earlier and felt better. He wondered why none of the many psychologists and psychiatrists he’d seen about his problem had told him what I said. The first time he asked I think my answer was that I cared more than they did about the relation of depression and sleep.

Recently he asked again: Why didn’t they tell him something so simple and helpful? Maybe I learned something in the intervening years because my answer was different. I said all health care professionals — not just doctors, all therapists/healers, mainstream, alternative, Western, non-Western — have no interest in treatments that they are not needed to administer. If all you need to do is to get up earlier in the morning, you don’t need a psychiatrist. Therefore a psychiatrist won’t tell you to do that. The only advice they are likely to give is advice they are needed to administer.

I could give dozens of examples. Does the Chinese herbalist tell my friend with an infection to eat fermented foods to boost his immune system? No, because that wouldn’t involve the herbalist. Instead he prescribes herbs that probably do the same thing. Does a dermatologist tell a teenager that his acne is caused by diet? No, dermatologists make the absurd claim that diet isn’t involved. Because if it were you wouldn’t need them. You’d just figure out what foods are causing your acne, and avoid those foods. Why do medical schools fail to teach nutrition? Because you don’t need a doctor to eat better. Why is prevention almost completely ignored? Because prevention doesn’t require any gatekeepers.

The economic term is rent seeking: health care professionals act in ways that require you to pay them. The usual economic examples of rent-seeking cause a kind of overhead you have to pay but the rent-seeking engaged in by the entire health care industry shortens our lives. Simple cheap safe solutions are ignored in favor of expensive and dangerous ones that don’t work as well. Our entire health system centers on gatekeeper drugs: drugs that require gatekeepers. The usual name is prescription drugs; their danger is part of their appeal to the doctors that prescribe them. Because it makes the doctor necessary.

What the Government — Any Government — Isn’t Telling You About Swine Flu

How weak it is:

By any measure A/H1N1 is a benign flu virus. According to official statements, New Zealand, for example, usually has 400 deaths from flu each year. This year there were 17, so it could be argued that the pandemic has resulted in 383 lives being saved, which makes it more effective than any flu vaccine.

It is always good politics to scare people. Create a danger from which you protect them. It’s such an old and common ploy it’s curious how well it still works. Maybe the gullibility is hard-wired.

Health Care: Why Problems Have Piled Up Unsolved

In an amusing comment on health care, Jonathan Rauch (via Marginal Revolution) imagines an airline system as archaic and inefficient as our health care system.

“Cynthia, I have filled out my travel history half a dozen times already this year. I’ve told six different airlines that I flew to Detroit twice and Houston once. Every time I fly, I answer the same battery of questions. At least a dozen airlines have my travel history. Why don’t you get it from them?”

“We have no way we could do that. We do not have access to other companies’ records, and our personnel have our own system for collecting travel history.”

The health care system, in other words, is full of problems that have built up unsolved. Solutions exist — the problems are not impossible — but haven’t been implemented. Jane Jacobs’s great point, in The Economy of Cities, is that this is what happens when those who benefit from the status quo have too much power relative to those who benefit from change. The stagnation in American health care is profound. It isn’t solved by universal health insurance. There would remain the horrible dependence on expensive dangerous drugs that don’t work very well (e.g., antidepressants, Accutane) and the complete lack of interest in prevention. The underlying problem, the source of many visible problems, is too little innovation.

Anti-Depressants Associated With Birth Defects

In the latest BMJ, a group of epidemiologists reports that SSRI’s (selective serotonin reuptake inhibitors, a commonly-prescribed type of anti-depressant) are associated with a certain type of birth defect when the mom takes the drug early in pregnancy:

There is an increased prevalence of septal heart defects among children whose mothers were prescribed an SSRI in early pregnancy,

We have a health care system built on dangerous drugs — and those drugs are poorly tested for safety. It isn’t in the drug companies’ interest to do so, of course. In this particular case, I wonder if the drugs were safety-tested on pregnant rats and if so what happened.

The Ethical Stupidity of Med School Professors: Plagiarism Very Very Bad, Ghostwriting Okay

Do medical school professors live in a different ethical world than the rest of us? Apparently. A friend of mine just entered grad school at Tsinghua. She was required to attend four different lectures about how academic dishonesty is wrong. (The last one, she said, was good; the speaker told a lot of stories.) China has a huge plagiarism problem, sure, but at least they say that plagiarism is wrong.

Whereas medical school professors haven’t managed to grasp that ghostwriting is plagiarism (taking someone’s words and ideas as yours without acknowledgment). And it happens all the time. NYU med school Professor Lila Nachtigall, as I’ve noted, considered the deed so minor she forgot that she’d done it. Apparently using a different word confuses them. A recent article in Nature reveals the befuddlement of the entire medical establishment about this. We’re not sure what to do about it, journal editors say. As Tony Soprano’s mom would say: Poor you.

What’s so nauseating about this is that ghostwriting is certainly worse than the garden-variety plagiarism that American undergraduates and the odd Harvard professor engage in.  (And at least they are embarrassed, unlike Nachtigall, when caught.) Garden-variety plagiarism is merely self-serving; you save time, get a higher grade. Whereas drug-company ghostwriting makes drugs appear better than they are. Which harms millions of sick people.

Although American universities publicly condemn plagiarism and other types of cheating, in practice they allow them. (Believe me, I know. When I tried to stop cheating in my Intro Psych class at Berkeley, the chairman of my department told me, “We’re not in that business.”) And the student cheaters — having been told by university blind-eye-turning that cheating is okay — grow up to be med school professors who do horrible things routinely. That’s my theory.

Thanks to Dave Lull.

Perverse Incentives in Medicine

In the comments, Timothy Beneke wrote:

My experience with a friend who had unexplained stomach pain was instructive. She saw 6 “experts”, 3 who worked for fixed salaries at institutions (Kaiser, Stanford, etc.) and 3 who were in the marketplace getting paid based on what they brought in each year. The three who were on fixed salaries were professionally cordial, and openly admitted that they could not say with confidence what was causing her pain. The three who were not on fixed salaries were very touchy-feely and charming and spoke with complete confidence about the cause.

Wow. This reminds me of my surgeon, Eileen Consorti, telling me that the operation she recommended would help me, that there was evidence for this, and then — when I couldn’t find any evidence — telling me she would find it and never doing so. She would have gotten thousands of dollars for that operation. It also reminds me of my dermatologist prescribing a medicine that didn’t work and, until I did an experiment that showed it didn’t work, having no idea it didn’t work. He got paid in any case.

Does H. Pylori Cause Stomach Ulcers?

In a previous post I said that the Nobel Prize to Barry Marshall and Robin Warren — for supposedly showing that H. pylori causes stomach ulcers — was a mistake. Because half the world has the bug in their stomach, and only a tiny fraction of them get ulcers, the true cause of those ulcers lies elsewhere, probably with an impaired immune system. Marshall famously drank a flask full of H. pylori and didn’t get an ulcer, yet took this to support his theory. A classic example of self-deception.

Recently Lam Shiu-kum, a former dean of medicine at the University of Hong Kong, was convicted of a giant fraud. He siphoning millions of dollars of medical fees into his own pocket:

Dr Lam, 66, brought a 39 year association with the university, his alma mater, to an abrupt end in March 2007 when the investigation into billing irregularities began. He is a distinguished gastroenterologist who conducted pioneering research into chemoprevention of stomach cancer through the eradication of Helicobacter pylori. His team also conducted the first double blind, controlled study into curing peptic ulcers by H pylori eradication.

I suppose this supports my case. As far as I know, almost all doctors and med school professors believe H. pylori causes stomach ulcers; I have never heard dissent about this.

More. What goes unsaid, and maybe unnoticed, in the debate about health care, is that it is hard to have decent health care (that is, decent health) when those in charge don’t know what they’re doing. The stomach-ulcer-etiology problem is a small example of a big thing. In case I’m not being blunt enough, let me be even more blunt: This example illustrates that the average doctor, the average med school professor, and at least two Nobel-Prize-winning med school professors (not to mention those who award Nobel Prizes) have a lot of room for improvement in their interpretation of simple facts. My previous example of the infectious-disease expert (a med school professor) who overlooked the immune system is another example of vast room for improvement. It’s hard to get good health care from people whose understanding of health is terribly incomplete yet don’t realize this.

Med School Profs As Drug Company Lackeys

What a cesspool. I mean the dirty work medical school professors do for drug companies. The profs make the drugs appear better than they are. Let me count the ways:

1. I blogged earlier about Duke professor Charles “Disgraced” Nemeroff taking huge amounts of money — which he then failed to disclose — to encourage doctors to give dangerous poorly-tested drugs to children. Nemeroff is (or at least was) considered a top psychiatry professor!

2. When the practice of drug companies ghostwriting articles for professors was revealed, New York University professor of obstetrics and gynecology Lila Nachtigall, the nominal author of a ghostwritten article, told a reporter (contrary to evidence supplied by Wyeth) that she had written all of her 1000 articles and 3 books. And she said this:

If they [Wyeth] came up with the idea or gave me an outline or something, I don’t remember that at all. It kind of makes me laugh that with what goes on in the Senate, the senator’s worried that something’s ghostwritten. I mean, give me a break.

It made her laugh. Yes, why should anyone care about the dishonesty of med school professors? What cave has Nachtigall been living in?

3. About half of published clinical trials were not properly registered, a new study showed (abstract here). A large fraction of these studies were drug-company-funded, I’m sure. (More than half were “industry” funded.) And the authors were often med school professors. Failure to register your study means you can distort the results to make them closer to the outcome you prefer by changing the “endpoint” (the dimension you use to measure whether the drug worked). Even among the registered studies, one-third used a different endpoint than the registration said. It is hard to avoid the conclusion that a lot of misleading results — making drugs look better than they really are — are being published. The level of cheating appears to be incredibly high — perhaps more than half of published papers.

Yes, Canker Sores Prevented (and Cured) by Omega-3

Here is a comment left on my earlier canker-sore post by a reader named Ted:

I found out quite by accident WALNUTS get rid of [canker sores] quite quickly. The first sign of an ulcer I chew walnuts and leave the paste in my mouth for a little while (30 seconds or so).

The first time was by accident, my ulcers disappeared so quickly I knew it had to be something I ate. And the only thing I had eaten differently the past day was walnuts.

Flaxseed oil and walnuts differ in lots of ways but both are high in omega-3. My gums got much better around the time I started taking flaxseed oil. I neither noticed nor expected this; my dentist pointed it out. Several others have told me the same thing. Tyler Cowen’s gums got dramatically better. One reader started and stopped and restarted flaxseed oil, making it blindingly clear that the gum improvement is caused by flaxseed oil. There is plenty of reason to think the human diet was once much higher in omega-3. All this together convinces me that omega-3 can both prevent and cure canker sores. Not only that, I’m also convinced that canker sores are a sign of omega-3 deficiency. You shouldn’t just get rid of them with walnuts; you should change your diet. Omega-3 has other benefits (better brain function, less inflammation, probably others).

Let’s say I’m right about this — canker sores really are prevented and cured by omega-3. Then there are several things to notice.

1. Web facilitation. It was made possible by the internet. My initial interest in flaxseed oil came from reading the Shangri-La Diet forums. I didn’t have to read a single book about the Aquatic Ape theory; I could learn enough online. Tyler Cowen’s experience was in his blog. Eric Vlemmix contacted me by email. No special website was involved.

2. Value of self-experimentation. My flaxseed oil self-experimentation played a big part, although it had nothing to do with mouth health. These experiments showed dramatic benefits — so large and fast that something in flaxseed oil, presumably omega-3, had to be a necessary nutrient. Because of these results, I blogged about omega-3 a lot, which is why Eric emailed me about his experience.

3. Unconventional evidence. All the evidence here, not just the self-experimentation, is what advocates of evidence-based medicine and other evidence snobs criticize. Much of it is anecdotal. Yet the evidence snobs have, in this case, nothing to show for their snobbery. They missed this conclusion completely. Nor do you need a double-blind study to verify/test this conclusion. If you have canker sores, you simply drink flaxseed oil or eat walnuts and see if they go away. Maybe this omnipresent evidence snobbery is . . . completely wrong? Maybe this has something to do with the stagnation in health research?

4. Lack of credentials. No one involved with this conclusion is a nutrition professor or dentist or medical doctor, as far as I know. Apparently you don’t need proper credentials to figure out important things about health. Of course, we’ve been here before: Jane Jacobs, Elaine Morgan.

5. Failure of “trusted” health websites. Health websites you might think you could trust missed this completely. The Mayo Clinic website lists 15 possible causes — none of them involving omega-3. (Some of them, we can now see, are correlates of canker sores, also caused by lack of omega-3.) If canker sores can be cured with walnuts, the Mayo list of treatments reads like a list of scurvy cures from the Middle Ages. The Harvard Medical School health website is even worse. “Keep in mind that up to half of all adults have experienced canker sores at least once,” it says. This is supposed to reassure you. Surely something this common couldn’t be a serious problem.

6. Failure of the healthcare establishment. Even worse, the entire healthcare establishment, with its vast resources, hasn’t managed to figure this out. Canker sores are not considered a major health problem, no, but, if I’m right, that too is a mistake. They are certainly common. If they indicate an important nutritional deficiency (too little omega-3), they become very important and their high prevalence is a major health problem.

Suppose You Write the Times to Fix an Error (part 2)

The Roberts-Schwartz correspondence continued. I replied to Schwartz:

“Dining establishments”? [His previous email stated: “Four restaurants simply cannot represent the variety of dining establishments in New York City”] I thought the survey was about sushi restaurants. Places where raw fish is available.

Quite apart from that, I am sorry to see such a fundamental error perpetuated in a science section. If you don’t believe me that the teenagers’ survey was far better than you said, you might consult a friend of mine, Andrew Gelman, a professor of statistics at Columbia.

John Tukey — the most influential statistician of the last half of the 20th century — really did say that a well-chosen of sample of 3 was worthwhile when it came to learning about sexual behavior. Which varies even more widely than sushi restaurants. A sample of 4 is better than a sample of 3.

Schwartz replied:

The survey included 4 restaurants and 10 stores.

The girls would not disclose the names of any of the restaurants, and only gave me the name of one store whose samples were not mislabeled. Their restaurants and stores might have been chosen with exquisite care and scientific validity, but without proof of that I could not say it in the article.

I wrote:

I realize the NY Times has an “answer every letter” policy and I am a little sorry to subject you to it. Except that this was a huge goof and you caused your subjects damage by vastly undervaluing their work. Yes, I knew the survey included 4 restaurants and 10 stores. That was clear.

As a reader I had no need to know the names of the places; I realized the girls were trying to reach broad conclusions. They were right not to give you the names because to do so might have obscured the larger point. It was on your side that the big failing occurred, as far as I can tell. Did you ask the girls about their sampling method? That was crucial info. Apparently The Times doesn’t correct errors of omission but that was a major error in your article: That info (how they sampled) wasn’t included.

He replied:

I could have been more clear on the subject of sample size, but I did not commit an error. Neither do my editors. That is why they asked me to write a letter to you instead of writing up a correction.

I don’t feel I have been “subjected to” anything, or that this is some kind of punishment. This is an interesting collision between the precise standards of someone with deep grounding in social science and statistical proof and someone who tries to write intelligible stories about science for a daily newspaper and a general interest audience. But I am not sorry that you wrote to me, even a little sorry.

i wrote:

“I did not commit an error.” Huh? What am I missing? Your article had two big errors:

1. An error of commission. You stated the study should be not taken seriously because the sample size was too small. For most purposes, especially those of NY Times readers, the sample size was large enough.

2. An error of omission. You failed to describe the sampling protocol — how those 10 stores and 4 restaurants were chosen. This was crucial info for knowing to what population the results should be generalized.

If you could explain why these aren’t errors, that would be a learning experience.

Did you ask the girls how they sampled?

His full reply:

We’re not getting anywhere here.

Not so. After complaining he didn’t have “proof” that the teenagers used a good sampling method, he won’t say if he asked them about their sampling method. That’s revealing.

Something similar happened with a surgeon I was referred to, Dr. Eileen Consorti, in Berkeley. I have a tiny hernia that I cannot detect but one day my primary-care doctor did. He referred me to Dr. Consorti, a general surgeon. She said I should have surgery for it. Why? I asked. Because it could get worse, she said. Eventually I asked: Why do you think it’s better to have surgery than not? Surgery is dangerous. (Not to mention expensive and time-consuming.) She said there were clinical trials that showed this. Just use google, you’ll find them, she said. I tried to find them. I looked and looked but failed to find any relevant evidence. My mom, who does medical searching for a living, was unable to find any completed clinical trials. One was in progress (which implied the answer to my question wasn’t known). I spoke to Dr. Consorti again. I can’t find any studies, I said, nor can my mom. Okay, we’ll find some and copy them for you, she said, you can come by the office and pick them up. She sounded completely sure the studies existed. I waited. Nothing from Dr. Consorti’s office. After a few weeks, I phoned her office and left a message. No reply. I waited a month, phoned again, and left another message. No reply.

More. In spite of Dr. Consorti’s statement in the comments (see below) that “I will call you once I clear my desk and do my own literature search,” one year later (August 2009) I haven’t heard from her.

Evidence Snobs

At a reunion of Reed College graduates who majored in psychology, I gave a talk about self-experimentation. One question was what I thought of Evidence-Based Medicine. I said the idea you could improve on anecdotes had merit, but that proponents of Evidence-Based Medicine have been evidence snobs (which derives from Alex Tabarrok’s credit snobs). I meant they’ve dismissed useful evidence because it didn’t reach some level of purity. Because health is important, I said, ignoring useful information, such as when coming up with nutritional recommendations, is really unfortunate.

Afterwards, four people mentioned “evidence snobs” to me. (Making it the most-mentioned thing I said.) They all liked it. Thanks, Alex.

Stoplights, Experimental Design, Evidence-Based Medicine, and the Downside of Correctness

The Freakonomics blog posted a letter from reader Jeffrey Mindich about an interesting traffic experiment in Taiwan. Timers were installed alongside red and green traffic lights:

At 187 intersections which had the timers installed, those that counted down the remaining time on green lights saw a doubling in the number of reported accidents . . . while those that counted down until a red light turned green saw a halving in . . . the number of reported accidents.

Great research! Unexpected results. Simple, easy-to-understand design. Large effects — to change something we care about (such as traffic accidents) by a factor of two in a new way is a great accomplishment. This reveals something important — I don’t know what — about what causes accidents. I expect it can be used to reduce accidents in other situations.

It’s another example (in addition to obstetrics) of what I was talking about in my twisted skepticism post — the downside of “correctness”. There’s no control group, no randomization (apparently), yet the results are very convincing (that adding the timers caused the changes in accidents). The evidence-based medicine movement says treatment decisions should be guided by results from controlled randomized trials, nothing less. This evidence would fail their test. Following their rules, you would say: “This is low-quality evidence. Controlled experiment needed.” The Taiwan evidence is obviously very useful — it could lead a vast worldwide decrease in traffic accidents — so there must be something wrong with their rules, which would delay or prevent taking this evidence as seriously as it deserves.