Research and Pirates Who Participate

So there’s this study, someone says. It proves this fantastical thing!

Ummmm. Okay.

What does that even mean? Well, science doesn’t prove things.  Proofs are, after all, a mathematical concept. But what makes a good study? Below, I talk about designing a good Participants section. In honor of Wednesday’s Talk Like A Pirate Day, they’re based around a fictional study involving pirates who hate chocolate and a new drug you created that might cure them of a hatred of chocolate.

This is a large hat, but it still only fits one pirate.

Are the pirates the same or different?

Where did you get the pirates from? If they were all from Somalia, you have problems. What if they all have a special kind of Chocolate Hate that is culturally-specific to the area? Then you won’t know if your new drug cures Chocolate Hate for everyone, or just for pirates of Somalian descent. You want to end up with a random assortment of all the eye-patched, peg-legged privateers you can have. In a theoretical sense, you want to take all the pirates, but them in a very durable hat, shake it, and blindly draw 10 of them.

In a practical sense, this isn’t always possible. (Hats are usually too small for more than one pirate, after all.) You can’t round up every possible person who would be eligible for your study. So, the best thing you can do to make up for this is to be aware of the problem, and try to get a variety of people to sign up. This means you should advertise your study in multiple places. (Though, of course, sometimes you do want a specific subset of people in your study. For instance, if I wanted to test a drug that only cured Chocolate Hate in pirates with Spontaneous Hook Growth, I’d advertise in Neverneverland.) You can also take your participants and randomly assign them to different groups within the experiment. (More on that later)

I guess I’ll just make a bunch of movie references, then. So here’s the Sorting Hat.

How did the pirates get sorted?

So you have these pirates. You want to see if your new miracle drug to cure Chocolate Hate works. So, you take half the pirates and give them the drug. The other half recieves a sugar pill. Everyone sits around and swordfights for a few days. Then, you make each group try chocolate, and see the results.

But, how do you decide which participants go in which group?

Well, theoretical hats again. This time, you can just put the names of the participants in a hat, shake it, and draw them out. The first name can go in the Drug group (or to use a more technical term, the Drug Condition) and the second can get a sugar pill (the Sugar Condition). The third name will go into the Drug Condition, and on and on. Doing this just is to remove any bias you could have. Even if you were trying your best to assign people randomly just by looking at names and grouping them, there’s a high chance that some bias (maybe you put everyone with green eyes together?) of yours could effect the study. The idea is to remove any possibility of bias that could cast doubt on the ability of your drug to cure Chocolate Hate.

How many pirates do you have?

For nearly every kind of research, having more participants is better. If you have only ten pirates, your research isn’t going to be as solid as if you had 100 pirates. This is a mathematical thing.

Say there were only 200 pirates in the world. But fifteen of them had a rare condition that meant they could never be cured of Chocolate Hate. If you randomly draw ten pirates out of all two hundred, you might end up drawing only pirates that can’t be cured. Then you’d think that your drug (which might work on everyone else) didn’t work at all. But, if you tested the drug on 85 pirates, things are different. Even if you managed to pick all of the incurable pirates, you’d cure some other pirates–and know that your drug works at least some of the time. So, bigger is usually better. It’s not always possible, but it’s a good goal.

And with all those things in mind, conduct your study! Select the pirates, sort them, administer the drug, and give them chocolate. And if it doesn’t work? More chocolate for you!

Yummy, yummy skepticism.

Images from here

Previous post

Podcasts, Glorious Podcasts!

Next post

Science Sunday: Why The Natural History Museum Is Awesome

Kate Donovan

Kate Donovan

Kate is an outspoken atheist, feminist, demisexual, stigma-busting student in Chicago studying psychology and human development. She juggles occasionally, would knit you something warm if she knew you, and reads anything she can get her hands on. She was raised believing alternative medicine worked, and now spends her time making skeptical faces at it. You can find her on Twitter at @donovanable

No Comment

Leave a reply