The Role of Amateurs in Science
by Siegfried Othmer | November 19th, 2007
There is one field in which an extensive mutually beneficial relationship has existed between amateurs and professionals. It is in astronomy, and the phenomenon was recently taken up in Science Magazine by John Bohannon (Volume 318, 12 October 2007, pp 192-3). Significantly, this symbiosis is occurring in a science in which we have only limited ability to do experiments. Mostly the science is observational. Most of the scientific observations are specifically targeted and hypothesis-based. They are so numerous that time on the big observatories for each project is scarce and therefore precious. But there is another crucial aspect of astronomy that focuses on celestial events that are not predictable either in time or place. This is mostly where the amateurs come in. They represent a world-wide army of knowledgeable observers that is on watch every night around the globe.
Most comets are first discovered by amateurs. For example, the Shoemaker-Levy comet that crashed into Jupiter in 1994 was first seen by amateurs. The last supernova in our celestial neighborhood, 1987A, was also first seen by amateurs. Word of the discovery was passed on immediately through existing networks to the professionals. And most recently a white dwarf in Andromeda reached its explosive threshold after scavenging matter from its binary partner. This time-critical observation was made by an amateur who rapidly passed on the exciting news.
A significant feature of this relationship is that confirmation of any amateur’s claims is easy to come by. An observation is confirmed by yet another observation. Validation is always close at hand. So in that realm the amateur’s contribution to science will always be secure, yet it will also remain bounded. Increasingly, however, amateurs are also playing a role in more scientific pursuits, as in the long-term monitoring of variable stars, and professionals are coming to depend upon these data. The role of amateurs in the field is increasing over time, and a professional astronomer was moved to say that “The sum total of ingenuity and energy among the world’s amateur astronomers vastly exceeds that of the professionals.”
Other examples of amateurs playing a role within science may be found among birders who participate in annual bird counts, and in amateur collectors of archeological artifacts. Most of the latter may be in it for themselves rather than for the science generally, so it would be a stretch here to highlight only the synergy with public purpose. A slightly better example is offered in paleontology, where some significant finds have been made by amateurs, although the private interest often conflicts with the public interest here as well. The Burgess Shale was discovered by an amateur, for example. Botanists rely on amateurs do to sample collecting for them around the world. As it happens, one of the two species of Bristlecone Pine in the White Mountains was identified by an atmospheric physicist who was also an amateur botanist (the late Dana Bailey). There is no way that taxonomy can be left entirely to professionals. Millions of species have yet to be identified and catalogued.
So the divide between the scientific expert and the knowledgeable amateur is not as absolute in the real world as one might have it. Now with this background, let us once again gaze upon our field of neurofeedback and ask how the interface is faring between the knowledgeable expert, the clinician, on the one hand, and the university-based scientist on the other. It is apparent that there is hardly any interface at all. Very few threads of connection exist in the field between the clinician and academia. This has not been for lack of openness on the part of the clinician. By and large, these people seem only too willing to share the good news about neurofeedback with anyone.
It is the academic scientists who have put up the barriers to communication. This barrier is such that even neurofeedback practitioners holding adjunct professorships at universities are generally not too forthcoming about their work lest they meet a hostile reception. This is such a remarkable state of affairs that we must take up the opportunity to do some remarking. In contrast to the astronomy example, we are not talking about amateurs here. Clinicians are highly trained in their profession, by and large, the majority of them (or at least a very large minority) sporting Ph.D.s. This means that they have even received training in research methodology; most of them have had to do a dissertation involving a research design. Aside from the Ph.D. level, nearly all neurofeedback practitioners have some kind of advanced degree relevant to their field of practice. The vast majority of them is licensed in their chosen professions. This binds them to act within the compass laid out by their colleagues both ethically and practically. They have also had extensive apprenticeships in addition to their academic work.
The second analogy is that in the realm of behavioral and mental disorders we are in an exceedingly complex and multi-faceted observational realm. As in astronomy, observation strongly dominates overt experimentation. The disregulated brain functions in ways that are not always going to be repeatable. Cumulative observations are needed on the same individual in order to bound one’s assessment of the level of function and the character of the dysfunction. The observations must cut across various functional categories. If the only access we have to this rich clinical phenomenology were through systematic research with group studies, under controlled conditions, with limited variable space, then our science would be impoverished indeed. (And that’s just where we are in mental health research, aren’t we?)
The third analogy to astronomy is that both the academic researcher and the non- academic expert have shared goals. In the case of astronomy, it is the advance of our scientific understanding of the cosmos. In the case of mental disorders, it is improved clinical care. Given the fact that the ultimate product of the behavioral health sciences is improved clinical outcomes, and given that these will likely be mediated by clinicians, just how is it that the clinician should be essentially excluded from the ongoing scientific conversation?
The fourth analogy to astronomy is that the collective resources deployed in the clinical community vastly exceed the resources available to the research community even under the best assumptions. The data relevant to the scientific understanding of neurofeedback are already being gathered on a daily basis in thousands of settings. The economic exchange involved in all of neurofeedback this year in the US is of magnitude comparable to the entire budget of the National Institutes of Mental Health. Even if the NIMH were to suddenly get on board and throw massive resources at this field, it would not begin to compare to the resources already deployed in the hands of research- cognizant professionals.
I was set to thinking along these lines by the report that Larry Silver had spoken out rather forcefully against neurofeedback recently. Dr. Silver is a Child and Adolescent Psychiatrist in private practice. He is also Clinical Professor of Psychiatry at Georgetown University Medical Center, and has been Acting Director and Deputy Director of the National Institute of Mental Health. He has also held the Presidency of the Learning Disabilities Association, which has historically been notably non-judgmental with regard to alternative modalities for dealing with learning disabilities.
His vigorous denunciation of neurofeedback at this late date was therefore somewhat surprising. Here’s what he submitted in answer to a question about neurofeedback from a parent who had had a good experience with it, and was curious as to why doctors don’t recommend it more routinely.
Answered Silver: “First, no research has shown that the theory behind the treatment or the treatment itself is effective for people with ADHD. “Second, when children who receive this treatment are studied, most show no benefit. The question for you is if the neurofeedback treatment made the difference, or if there might have been another contributing factor (medication, special help in school, increased parental attention, tutors, etc.). “I will not discourage any parent from trying what they think will help their child; however, I do not condone treatments where the claims and findings cannot be significantly duplicated.” In short, Silver does not condone parents seeking out neurofeedback. Pretty strong stuff. Those familiar with the research in this field know these statements regarding neurofeedback to be entirely false. But there is more. I had communications with Dr. Larry Silver during the early nineties, and his opposition to neurofeedback is much more strident now than it was then. This trend is counter to the trend in research, which has brought ever better studies to bear on the issue of neurofeedback efficacy generally, and efficacy for ADHD specifically. I alerted Silver to our then-new IQ data, which were intended to replicate the earlier published IQ measures of Michael Tansey. Silver was unwilling to support publication of our data because they were not acquired under controlled conditions. That was certainly his prerogative, but at the very worst the data are out there with a question mark remaining. At minimum, two independent sets of such data should have kindled some curiosity in the scientific mind.
Silver’s non-response to these remarkable data renders his own declared objection disingenuous. Confronted with unexplained large IQ gains in a group study, Silver objected that one could not be sure that the neurofeedback was responsible. The usual scientific response to that state of affairs would have been to mobilize an effort to understand what mechanism might have been involved through a controlled study. Silver’s subsequent inaction demonstrates that he simply did not believe that such IQ changes were possible by any method, so what’s to study? The data were simply dismissed. Such a dismissal implies that Silver himself rejects any kind of placebo model as an explanation, a position with which we would hasten to concur. There is no placebo model for 23-point IQ gains. But if there cannot be a placebo explanation, then a placebo also need not be tested for. Hence the data must stand as they are. They don’t have to pass by the way station of a meaningless placebo-controlled study that cannot possibly alter the conclusion we have already reached.
If Silver had followed the field he would have found out that every study of neurofeedback for ADHD in which IQ was measured has shown increases, and by now these findings are no longer scarce. Our early results and Tansey’s were being duplicated, just as his above message calls for. Now in his response to the inquiring parent, Silver is being called upon as an authority on ADHD. And in his role as a scientist, he must give answers that are supportable. How is it possible for him to annihilate all of the cumulative research support for our claims? Why is he willing to risk such a categorical statement that is provably false?
The answer, I submit, lies in the radical discontinuity between the scientific (read pharmacological) and the clinical realms of the mental health enterprise. After I gave up on Silver so many years ago, it is likely that he has not had many fruitful interactions with clinicians knowledgeable about neurofeedback. There is no enviroment for that to happen organically. The rise of clinical neurofeedback has occurred entirely outside of his experience. So Silver no doubt felt secure in the knowledge of where his colleagues stood on the matter, and that was enough. In his stridency, he reflected the atmosphere of intolerance that is now abroad within academic psychiatry generally. Neurofeedback threatens pharmacotherapy; so we must act collectively as if it didn’t exist.
The above explanations are sufficient for the purpose, but I am tempted to indulge a further speculation. There is an incredible arrogance in Silver’s verbal annihilation of neurofeedback. He exhibits that destructive combination of ‘arrogance and ignorance’ to which scientists at the top of the heap are susceptible. But there may also be a rising level of fear in back of this, the fear of a person in high station insecure about his position. Surely the questions about neurofeedback are coming at these people more frequently and more insistently nowadays. Parents who have been helped by neurofeedback are not so easily knocked off their perch. And parents are increasingly making their own decisions in many ways, not only with respect to neurofeedback but also with regard to other alternative modalities. The rising tide of neurofeedback has not been forestalled by the insistent denials of Silver, Jensen, Barkley, Goldstein, Ratey and others. What’s to do? Nothing for them to do except to raise one’s voice even further and to raise alarms even more insistently.
It may appear that in challenging people like Silver I am attacking the enterprise of academic science itself. That is of course not the case. I read the Journal Biological Psychiatry faithfully, and within its confines excellent work is being done. But I also believe that the science of psychiatry has been badly regimented from the top, which in turn has only been possible because it has cut itself off from fertilization from the bottom. With science having become the dominant paradigm, it is only natural that forces both good and bad will cloak themselves in the mantle of science for their own ends.
Another example of this is furnished conveniently by the current news of the resumption of “scientific whaling” by Japan. The agenda is already clear. The whales must be killed in order to be studied, and they will end up on the dinner tables in Japan forthwith. That doesn’t leave much of a window for scientific investigation. But they call it scientific whaling nonetheless. Similarly the pharmaceutical industry is harpooning our children into lifetime drug dependency. Already 1.6 million American children are on more than one psychoactive agent. Such polypharmacy has never been adequately studied in research. Such data as exist have sometimes been discouraging (Ritalin + Clonidine, for example; the addition of anti-depressants to the treatment of Bipolar Disorder). But the impression is given that polypharmacy is scientifically sanctioned nonetheless. Silver is the witting agent of such dependency. He has discredited himself as an arbiter of non- pharmacological alternatives.
So what’s the answer? In this particular microcosm it lies in our establishing links to academia wherever the opportunity arises, to restore a broken connection—even if we did not break it in the first place. If just what we know now about neurofeedback were common knowledge within a university psychology or psychiatry department, it would be revolutionary.
Secondly, we must stop blaming ourselves for the position we find ourselves in. If a person like Silver can survey all of the existing neurofeedback research and yet not see anything, that problem will not be solved by placing even more research before him. To the neurofeedback deniers we must always remain invisible! We are confronting nothing less than counterfeit science, for which honest coinage provides no countermeasure. My hope in this piece is through exposure to render the life of the neurofeedback deniers just a bit more uncomforable. To the public they may still appear as authority figures; to us they appear sullied by what may well rise to the level of scientific fraud.
Siegfried Othmer