In this blog post I shall look at alternative medicines in terms of their efficacy and the standard of evidence required to prove this.
Many experiments have been carried out on alternative medicines. However, unfortunately a large number of these tests do not have good methodology. As a bare minimum a good test should use a large group of test subjects; neither the patients nor the experimenters should know who is and who is not receiving the actual treatment & which patients receive the actual treatment should be random.
There are good reasons for these methodological restrictions: If there are not enough people in the study its statistical power can be very low. If researchers know who is receiving treatment they may not report their observations as accurately (this may be unconscious bias, wishful thinking or unwitting cherry picking of results). Similarly if patients know whether they are receiving the test treatment this may effect them psychologically and effect their reports. In addition there is an effect known as the placebo effect in which patients fair better off if they believe they are receiving a treatment whether or not they are actually receiving it. The mechanism for the placebo effect is poorly understood but may involve an increase in endorphins changing the way the immune system prioritises different tasks. Such biases can ruin even an otherwise very careful study.
There are many scientists who falsify or exaggerate their results in a bid to better secure funding or other benefits. Papers claiming significant new results should be carefully checked for evidence of fraud (think human cloning!). This is not a fringe problem! Many well known experimenters fudged their results to make them look more convincing. Examples include Mendel, Mickelson or Morley.
These two problems are problems with scientists and with their experiments. The next problem is a problem with the system of publishing in peer reviewed journals:
Suppose you have a good quality paper by reputable scientists at Harvard with a novel result (say that scratching ones ear decreases the risk of cancer). If 1000 groups also performed this experiment and found no result then you should take that into account when deciding how much credibility to grant this new experiment. The problem is that most if not all of those other groups won't publish (who wants to read a paper saying that there's no evidence that scratching ones ear decreases the risk of cancer?). Given that, what you'll see is one or two experiments including the Harvard one you're likely to inaccurately deduce that there is some evidence for a new cheap preventative remedy for cancer.
Of course all of the above problems apply to standard medical therapies just as much as alternative therapies. They show that one should be generally sceptical of new studies and should always replicate them before rushing to the media! The next problem is one specific to fringe claims:
If a study that is performed gives 95% statistical confidence that X is an effective therapy that does NOT mean that the chance it isn't an effective therapy is 5%. Rather one must weigh the study against other relevant studies that have been performed and against your prior estimates. A claim may be unlikely given other experiments that have been performed:
Homoeopathy takes a solution of a chemical and dilutes it to the point at which you should expect none of the original molecules to be present. Any arrangement of water molecules that may have been produced by chemicals previously present would disappear within a picosecond according to well tested chemical and quantum theory. Quantum theory together with chemical theory predict that homoeopathy cannot work. This means that a test of homoeopathy is effectively a test of these theories as well as biological ones.
These physical and chemical theories have been tested (to the requisite level of accuracy). The likelihood that they are inaccurate (in a way significant to homoeopathy's claims) is probably less than one part in 100,000. A test of homoeopathy that concludes that it is effective with a 95% chance of being right just doesn't cut it against such odds. This doesn't mean that a test of homoeopathy could not show it valid one day. It does mean that this would show us new and important physics. A test of homoeopathy that concluded it worked which had a statistical power of 1 in 1,000,000 would be much more convincing although fraud and malpractice would still have to be carefully ruled out.
Other alternative therapies are often different in this regard. Herbal remedies giving health benefits does not contravene either our understanding of physics, chemistry or that of biology. In this instance we can take the statistical power of studies as being close to probabilities that the treatment is effective. Many of our standard medicines came from plant extracts of one kind or another and it is not implausible that some of these remedies might actually have an effect (however caution is advised, see below). Of course many herbal remedies haven't passed this test in reality.
Acupuncture and therapies involving certain exercises, stretches etc. These therapies actually do something to your body. Claims need to be assessed on the basis of to what extent they contradict current well established theories. The more this is the case the higher a standard of evidence should be required to show the therapies are effective. A claim that acupuncture relieves pain in the area worked on is more convincing than a claim that it help reverse diabetes.
Incidentally we now know that some alternative therapies do have some positive effect. Acupuncture for instance is thought to have certain limited positive effects. A small number of herbal remedies have an affect also.
However, there are a good reasons to oppose many alternative therapies until they have been put through clinical testing: Almost all known therapies have some side effect, some of which can be serious. The more likely an alternative therapy is to have some effect the more carefully it should be put through clinical trials to make sure that this effect is not negative. In addition many alternative therapies are not properly regulated and are hence full of quacks ripping off the infirm. Apart from financial loss this can result in people failing to receive tried and tested therapies that have much more chance of actually helping them.
An example of the former problem is St. John's Wort. A herbal remedy for the treatment of depression, it has been found to disrupt at least one form of the birth control pill. It would be a mistake to assume that 'natural' remedies derived from herbs of plants are safe.
In another blog post I shall look into the difference between reductionist and holistic approaches to reality and explain why I think why holism is consistent with science. I shall also come back to the issue of finding a plausible mechanism for a phenomenon and how this effects statistical calculations regarding detecting such phenomenon.
In summary the main point made in this post is that some scientific claims in one area of science (biology or psychology) have implications in another area of science (Physics or Chemistry). When this is the case we must adjust our initial expectations depending on the other area of science and the standard of proof applied there.