CE:s serious couch - Q&A on brains

[C Elegans]

[QUOTE=CM]Basically scientists do alot of studies and do alot of work to explain our environment and its actions and how they affect us. I wanted to know is when is it that a study becomes accepted as legitimate? Ie: Some scientist do a study they have a control group and all. They follow the rules to the T but when does it become accepted as a good and logical piece of science. I am not sure i am making myself clear.

Basically i want to know how do you evaluate a scientific study as being correct?[/QUOTE]

This is an extremly important question which I glady answer, the answer is going to be long though.

Short summary: All scientific studies must be replicated several times by independent scientists, and if possible also with backwards causality. This process often takes 10-20 years.

More detailed answer for those who are interested: A million things can go wrong in science. Very, very rarely it happens that scientists confabulate their results in order to achive grants money or personal fame and honour. I am only aware of a handful of cases, but they do exist and have been discovered by peer-review and the monitor process. Monitoring is a control system that means that an independent expert, usually from abroad, will visit your lab and check all your raw data.

What happens more often though, is that you happen to catch random events and interpret them as general, or there is something wrong with your measurement instruments or data analysis. Two colleagues of mine spent over 1 year full time to collect data about the startle reflex. The experiments where really cleverly set up, and they tested lots of people. When the data collection was finished and they analysed their data, it turned out their data looked really shaky and strange. A further investigation revealed there was a technical error in one of their instruments, so the they discarded the whole data set.
The most common problem though, is misrepresentative samples. This goes for all empirical, experimental sciences, beit medicine, chemistry or physics. When you study something, you can rarely carry out full population studies, ie study all possible individual occurrences of a phenomenon. You have to study a sample, you can't study all stars of this type, or particles of this type, or human beings of this type, you can only study a selected group. Even if you pick your sample according to all the scientific rules, there is always the chance that by coincidence, your sample is not representative for the entire population. Thus, it is very important to replicate (repeat) findings on independent samples. In genetics, you constantly have this problem because the possible variations in 30 000 genes is vast, so if I pick 300 individuals and you pick 300 individuals, chances are big that these two samples will be quite different even if we try to match them for age, gender and whatever.

Aside from independent samples, a study must also be replicated by independent groups. Exactly how many replications that are demanded differs depending on the status of the field. If there is very strong evidence pointing in a certain direction and you carry out a study that goes in line with this hypothesis, then it may be enough with 3-4 replications if the study designs are solid and the findings is very clear. If you discover something new that no current hypothesis has addressed, more replications are needed. Backwards causality is usually viewed as the "final" replication. Let's say I do a genetic study on patients with autism. I compare a group of 100 patients with a group of 100 control subjects, and then I find that the patients with autism have a gene polymorphism (gene variation) at the 5-HTTLTR gene, that normal control subjects have not. Let's say 3-4 other groups replicate my findings in large, independent samples. Then it's about time to make a backwards replication by collecting blood from lots of random people, both patients with autism, patients with other disorders and control subjects and then genotype them blind, ie I don't know who is who, I just group them according to their genotype and when that is done, the code is broken and we identify who is patient and who is control subject. If the finding still holds, my orginal study will be viewed as legitimate and "correct".

An exception from this standard, is the large epidemiological population studies that include full populations. One example is the international twin registers that collaborate to make studies of for instance all patients between age 30-60 with a specific form of cancer in several different countries at the same time. One of my collaborators at the Swedish twin registry has just started to investigate 6 000 twin pairs with schizophrenia. The aim is to investigate all Swedish twin pairs where one or both siblings have schizophrenia. At the same time, the twin registry in Finland, the US, the UK, Japan and Denmark are doing the same thing. Some other countries have indicated they aim to join on the way. In 5-6 years or so, they will have produced data that cover hundreds of thousands of twins with schizophrenia, so replications are not necessary since these are full population studies. So if they say: schizophrenia is heritable to 75% and an infection in the mother during pregnancy increases the risk for development of schizophrenia with 30%, then this will be take as correct.

It is a question of taking many samples from a population and all show the same thing, or taking the whole population. Full population studies are so expensive and takes such an incredible amount of time to carry out, so they are very rare. It's like longitudial studies, where you follow a cohort (large sample of individuals, usually many thousand) over a long period of time in order to decide causality. They knowledge you gain is of highest quality, but nobody wants to pay for something that may or may not show something interesting in 20 years, so it can rarely be done.

Finally, some words about falsification. No scientific knowledge is ever "ultimate" or "absolute"; dogma cannot exist in science. All current knowledge can always be falsified by future studies. Considering the massive evidence and what would be required, it is however very unlikely that the existence of gravity, that all life on earth has common ancestry or that DNA carries genetic information, are going to be falsified as long as humans and earth have the features they have today.

[C Elegans]

[QUOTE=Chanak]I'll PM you my e-mail address. I'm interested. [/quote]

Nice Here is a link to a free mini-review in a very good scientific journal:
http://www.jneurosci.org/cgi/content/full/22/3/612#B14

I'll send you some articles when I get back to work Monday, since I need my institutional subscription to all the journals in order to have access to full articles.

Nerve cells are strange indeed, both the peripheral ones as you describe and the central ones (those in the brain and spinal cord). Almost every day I am amazed by both new fantastic discoveries, and how little we know.