You are intelligent.

You are a scientist.

You, yeah you, the one reading this.

You, the one who failed the chemistry test everyone else easily got an "A" on. Or you, the one who got in way over your head taking Physics AP in high school for some god-forsaken reason. Or maybe you, who tried to major in Biology but couldn't pass Chem 102, coming to the conclusion that you are just shitty at all things science.

...Oh wait, those were all me. 

So maybe I'm not innately talented in passing Chemistry tests I never studied for. Maybe I needed to take a regular-level physics class before trying an AP level. And maybe it's not best to try and learn Chemistry beside 150+ other kids in a giant lecture hall, especially when it's not your strong suit.

In fact, I think that NOT being a natural scientific genius may be why I look for so many different ways to try and explain its principles. Because whatever way they tried in school didn't work for me, exercise did and learning about what are the best legal steroids for bodybuilding is necessary for this lifestyle.

And if you're like a lot of other people I know, it didn't work for you either.

Our brains tend to grasp things easier if we put a human spin on them - if we add a social element to it. My goal today is to explain how many scientific studies are formatted in an interesting and understandable way. I think it will help to shed some light onto some of their more subtle points:

1. Why the context the study was conducted in is very important
2. Why, a good deal of the time, if you're not already in the field the study is looking at, you won't understand it 100%
3. How to work around #1 and #2

So, here's our fake scenario:

You have a fussy 4 year-old who hates going to bed at bedtime. If you want to retain your sanity, you need to figure out a way to get him to go to sleep on the best memory foam mattress  you have consistently and reliably.

Never thought I'd find a reason to post this video on my blog...

Efficacy of Bed-Time Stories on Sleep-Induction in 4-year-old Male

Abstract

Objective: To determine the efficacy of reading stories at bed-time on the time it takes to fall asleep (TTS) in our 4-year-old male subject.

Methods: The subject was exposed to 3 different treatments:

1. Bed-Time Story: Goodnight Moon 
2. Bed-Time Story: Where the Wild Things Are 
3. Control (No story)

Each treatment was done for 2 weeks, followed by a 2-week wash-out period between each treatment where no books were read.

TTS was recorded each night and averaged using the mode.

Results: TTS results varied from 4 to 97 minutes.  The treatment with the lowest average TTS was treatment #2 with an average of 13 minutes.  

Conclusion: Treatment #2 was significantly more effective than treatments 1 and 3 on TTS. However, one of the TTS values for treatment #2 was 97 minutes - the highest value of any treatments.  This suggests that factors outside of the story read and the bedtime may be affecting TTS values.

It is clear that further study must be done on this subject to arrive at the optimal treatment outcome.  

So that all seems pretty succinct, right?  Well let's move onto the FULL study and see if we're missing any crucial context...

Full Study

Background: Having children fall asleep in a timely fashion is of paramount importance for the child's development.  Well-rested caregivers have been shown in previous studies to have higher levels of patience, are better able to work to provide for their children, and are more apt to be involved with their children when they are at home.

Better sleep benefits the child by making them more attentive; performing better at tasks required intellect or motor coordination.  As well, they are better able to express their wants and emotions to their caregivers.  

Some previous treatments have been tried on the subject with negligible improvement. Such treatments included a warm bath before bedtime, use of a nightlight, and turning off the television at 8:00 PM. 

However, other treatment options were promising.  Pushing bed-time back from 8:30 PM to 9:00 decreased time till sleep (TTS) by an average of 19 minutes.  However, benefits reversed after 10:00 PM.  

The treatment with the highest success rate was allowing the subject to sleep with his caregivers.  However, this treatment has dangerous side-effects for the caregivers and thus is not a suitable long-term treatment.  More options must be made available. 

In this study, we hope to combine the proven-successful treatment of a 9:00 PM bed-time with 2 other options in search for the best treatment option for the subject. 

So we've learned two important things from the background of this study:

1. This study is being done to improve the overall well-being of the child.
2. Previous treatments have been attempted, some less and some more successful.  The successful treatments affected what was done in this study.

Why is this important?  Well, it's not hard to imagine someone reading the abstract and completely dismissing the study by saying 9:00 is a horrible time to put a child to sleep, 8:30 is a much better time.  Except by reading the background, you discover that's already been attempted and found not to work.  Whoops!

Can't you imagine this study being posted on Facebook, and someone commenting "Everyone knows a warm bath works best to relax you and put you to sleep. What a stupid study. I can't believe we're wasting taxpayer money on this. Thanks Obama!!" ...except that was also attempted, and found not to work, if you had read the background.

As well, the background gave the "big picture" idea of why this study is being done. If the purpose of the study was to find out what would benefit the caregivers most, you could arguably propose the best treatment would be putting the child up for adoption! But now that you have the context of the study, you know that's not a viable option.

Methods: The subject was exposed to 2 different treatments and 1 control over the course of 16 weeks.  

Potential variables that were controlled for included:

• Time when reading began (9:00 PM)
• Pre-bedtime ritual for up to an hour before bedtime
• Sugar and caffeine intake after 2:00 PM 
• Bedroom temperature
• Bed-sheet color
• Level of lighting in bedroom during readings 
• Caregiver doing the reading 

Each treatment was attempted for 2 weeks, with 2 weeks in between each treatment method so that the subject would not become accustomed to the routine. 

During the control periods, the same caregiver would be present in the room, but without reading a story.

The treatments involved rotating 2 different books: Where the Wild Things Are - a more action-packed thriller compared to Goodnight Moon. We wanted to test if the level of excitement in the books affected TTS values.  We hope that this data will help the caregivers select appropriate bed-time books in the future. 

TTS values were collected every day, for a total of 14 data points for each treatment.  The mode was selected as the average for each treatment, as there were typically one or two extreme outliers for each one. 

The methods section is very important! This shows you how the researchers conducted the study in the first place, which tells you if the study is a good one or not.

If the methods were, "our intuition and common sense told us Goodnight Moon was a superior book choice, then we flipped a coin to decide whether it was better than the control," you'd say it was a poorly-done study, wouldn't you?! I hope so!

We're also shown how the researchers tried to make sure nothing else could possibly be affecting the outcome of the study besides the book being read. Maybe they missed some possible other variables, or were unable to control for some, (such as the time of year, it's hard to get kids to sleep around Christmas!) affecting the validity of the results?

Results: It was found that reading Where the Wild Things Are resulted in the lowest average TTS compared to Goodnight Moon and no book. (Figure 1)  The average TTS values for each treatment were:

1. Goodnight Moon - 21 minutes
2. Where the Wild Things Are - 13 minutes
3. Control (no book) - 27 minutes

These findings show a significant difference between TTS in each group.  

The findings appear to suggest that a bed-time story is a better treatment than nothing, however further research involving a wider variety of stories is necessary to draw any conclusions. 

Results sections can be pretty cut-and-dry. They tell you what happened, which is pretty important though!

Discussion: Reading of Where the Wild Things Are showed a dramatic decrease in the time it took for the subject to fall asleep as compared to Goodnight Moon and Control treatments. As well, aside from boredom on the part of the caregivers from reading the same book over and over, side-effects were non-existent.  

It is believed that the excitement of the story-line, followed by a slow, soothing ending is what made Where the Wild Things Are the best choice. The subject has a history of enjoying action-packed media, so it is possible this history influenced the outcome of the research.

It is possible that pushing back reading time to 9:30 PM would have influenced the efficacy of the Goodnight Moon treatment, when the subject would be more apt to being sleepy, and more receptive to a low-action story. Further research will be necessary. 

There are many limitations in this study. First, only two books were chosen as treatments - one as a more "action-based" story, and one as a "low-stress" story. However, the subject owns many other books and it is possible their outcomes will be unpredictable. 

As well, we were unable to control for the time of year this study was conducted.  The beginning of December occurred around the same time as the Goodnight Moon treatment. The subject loves Christmas, so his excitement may have adversely affected this treatment's outcomes. 

This study only included one subject, which is another large limitation for general use of this research. Further research with more subjects and a wider variety of treatments will be necessary for public relevance. 

Man, we got a load of information from the discussion section! (That one is always my favorite) The researchers owned up to the fact that this study has little relevance outside of the one subject they studied. Even then, there were many possible alternative explanations.

The researchers are basically straight-up telling us that a headline like, say...

"Scientists show storybook, Where the Wild Things Are, a better sedative than a bottle of Benadryl!"

...just may be a tad inaccurate.

We also learned some limitations we would have had NO IDEA about unless we knew the subject well beforehand, such as Christmas-time is a large confounding variable for him. Realize that important contextual data like this is not always included in the study itself.

This means unless you had been following and studying the subject for a long time prior, you wouldn't be able to grasp the importance of everything going on in the study.

This of course doesn't mean that it's pointless for you to read a study.  Hell, if you read any part of the actual study being discussed in the internet article du jour, you're doing better than 99.9% of the people commenting, and potentially even the person who wrote the damn thing.

Being able to point out limitations isn't being critical about what another person believes, it's pointing out how the study they're basing said beliefs on has the potential to be wrong.

At the same time, you need to be able to apply that same idea to yourself and your beliefs.

It's always possible that one study had too many confounding variables and limitations to be useful. So what if we did 99 more studies, and 98 of them showed that Where the Wild Things Are, read at 9:00 PM was the most effective treatment, with just 1 of them showing Goodnight Moon was a slightly better choice?

99% of studies came to one conclusion, 1% came to another. Using this data, which of these statements is more evidence based?

• "Based on the current evidence, Where the Wild Things Are is the best choice for getting this subject to fall asleep the quickest. Should more studies come out showing otherwise, we will reconsider this stance."
• "Where the Wild Things Are shows all manner of unnatural things that are not good for a child's developing brain. Here is a study that shows Goodnight Moon is the best choice. The other studies are funded by lobbyists pushing a pro-monster agenda."

And if you read an article that cited the ONE study out of 99 on the subject showing Goodnight Moon was the best choice, not knowing the other studies existed, you might believe it.

This is why if the scientific community comes to a general consensus on a subject matter, you really ought to lend it some credence. They likely know more than you do because they know:

• The context the studies are done in
• The entire giant SCOPE of research being done on the subject
• What limitations and variables really matter
• The specific scientific jargon used in said studies

If you were the parent of the child in our fake study, you'd roll your eyes at someone telling you that a warm bath is the one true answer. (and maybe start yelling at your television when an ignorant, uninformed pundit started calling you an idiot for believing otherwise)

Now imagine the same thing happening, except you're a scientist who has devoted the past 12 years of your life to studying one specific subject matter for meager pay of about $20,000/yr, and some random person from the internet calls you a Big Industry shill for your findings. I'd probably need to be restrained.

Moral

You use a crude version of the scientific method everyday. Anytime you use logic and information to figure out why your child won't go to bed at night, that's an un-refined version of the scientific method.

When you actually set up an experiment and record the results, that's getting a bit more refined. Trying to control for variables that affect the outcome, considering the context in which you're doing the experiment - that's refining it more and more.

There is A LOT that goes into a full scientific study. You cannot rely only on the abstract of one single study to tell you everything you need to know about a subject.

If anything, read the entire, full study before trying to form any kind of conclusion. If you don't understand the study, don't worry - just don't come to any hard conclusions and be open to the idea that your interpretation is wrong!

Oh, and when the scientific community forms a near-unanimous consensus, you should probably listen to them.