Wednesday, February 22, 2012

How to Design the Rice Experiment

"The first principle is that you must not fool yourself—and you are the easiest person to fool."
– Richard Feynman

The rice experiment, as popularized by businessman Masaru Emoto, is a good example of how not to design a scientific experiment. I will explain why at the end. First, I will explain how I would design a rice experiment. I am not a scientist, but I try to stay scientifically literate. If anyone has suggestions on how to improve this design, let me know.

To be clear, I am not planning on doing this experiment, as I will explain afterward.

My Hypothetical Rice Experiment:

1: Prepare good words and bad words on opaque adhesive labels. The labels need to be opaque enough so that they cannot be read through the backs of glass jars.

2: For a control, I would prepare labels with no words. I would also prepare labels with neutral words. I would also prepare good, bad, and neutral words in a foreign language that I don't understand. All the labels would be prepared under the same hygienic conditions and cut in identical shapes. This is the heart of the experiment. Let's pause for a moment and ask why this is so important...

A controlled experiment should answer the question, "Compared to what?" If, all other things being equal, some interesting variable changes, what will happen? That's the question. The tricky part is to make all other things be equal.

For a wonderful description of eliminating variables, listen to or read "Cargo Cult Science" by Richard Feynman. Now back to our hypothetical experiment...

3: Have all the words recorded separately in a ledger. This will help keep me honest.

4: Sterilize dozens of jars, seals, and lids. This will zero out the bacteria count. Let them dry.

5: Have someone other than me apply the labels to the jars. This is the start of a double-blind.

6: Have that person cover the labels with identical strips of lightly adhesive opaque paper. At the end of the experiments, these covers will be removed. This will double-blind the experiment.

7: Have a third person rearrange the jars before delivering them to me. This will randomize the experiment and complete the double-blind.

8: The ledger from step 2 records what words are used, though I don't know which ones are on which jars. The words should be categorized at the outset: good, bad, neutral, or blank. Words should also be categorized as English or foreign. (The foreign word might well be kept secret from me, just in case.) The word categories have to be established at the outset to prevent fudging afterward.

9: Set the labeled empty jars in a relatively non-hygienic place so they can attain similar levels of light contamination. Totally sterilized jars may preserve rice indefinitely.

10: Cook some rice and put the same amount in each jar. A few ounces on the bottom will do. All we want is to be able to look inside the jar without the labels and their covers blocking the view.

11: Set the jars in an array that I can check every day. The jars would be numbered so I can track the progress of each jar individually.

12: See which jars get moldy first. Keep watching as other succumb. I would set a deadline of maybe 60 days.

13: The reveal. After the 60 days, look at the jars and their corresponding labels. Compare with the ledger and mark each jar as good, bad, or neutral. If the results are:
- 12 good English words and 12 good foreign words = all pristine
- 12 neutral English words, 12 neutral foreign words, and 12 blank = all somewhat moldy
- 12 bad English words and 12 bad foreign words = all very moldy

This, or something close, would be an extremely significant result. But I expect the onset of mold will be random and will not track with any good word or bad word labels in any statistically significant way. Mold will slightly favor one category of words over another, just as a matter of statistical noise. The math for this is well worked out. The more jars I use for each category, the smaller this statistical noise becomes. If I do the experiment with 30 jars for each category, I would get very high resolution, low noise results.

14: Submit for peer review. I would explain the process described above. My test would satisfy the basics of what we want from a well designed experiment: it's double-blinded, randomized, controlled, and uses an OK sample size. Negative results would be expected and not terribly interesting. (Sometimes negative results are groundbreaking, like the Michelson-Morley experiment that set the stage for Einstein's Relativity.) In the rice experiment, a positive result would be extremely surprising. The way this is designed, a positive result would have a rock solid foundation. Just one more step would be needed:

15: See if anyone reproduces the results under similar experimental conditions. If no one can reproduce my results, there's a good chance I falsified my data or was just plain sloppy.

16: If the results are positive, conduct the experiment for the James Randi Educational Foundation (JREF). If it does show evidence of paranormal activity that can be verified under scientific controls, I will win $1,000,000. And that's a lot of money! I would like to have that prize! But it's been available for decades and no one has won it yet.

How to Backpedal:

Let's say I was invested (monetarily or emotionally) in the results coming out positive—but they came out negative. There are some tricks, fallacies of special pleading, I can play on myself. These might help me to dismiss my own results or fudge them in my favor:

Anomaly hunting: Maybe some seals were red and some were beige. Maybe the red ones were moldier to a slightly higher degree that statistical noise would predict. Maybe the vibrations inherent in color caused the differences in moldiness. Of course, that's not what we were testing, that's a patterns identified after the fact. If you want to test for color, put that in the ledger at the outset. Don't shoehorn a pattern after the fact.

For some real adventures in anomaly hunting, look at the number juggling people apply to the Egyptian pyramids. You can take a rich batch of numbers and combine them into all sorts of flukes that match physical constants or astronomical distances. James Randi shows how you can do the same anomaly hunting with the Washington Monument in his great book Flim-Flam.

Blame science or Western thinking: This is the common tack of accusing the skeptical mindset of spoiling the results. The experiment above is designed without appealing to any particular cultural heritage. The design is based on me preventing myself from introducing bias. If scientific thinking is such a party-pooper, how has it been so successful in shaping every little bit of technology we use?

Science, skepticism, critical thinking—these have produced plenty of reliable results, like cars and air travel. Telekinesis, for example, has not delivered comparable goods for human transport.

Those YouTube Videos and Why I Will Not Conduct My Own Experiment Design:

The rice experiment, as popularized online, has no controls, no blind (let alone double-blind), and operates on the smallest possible sample size. It is a race to see which rice gets moldy first. If the bad word rice gets moldy first (it's a 50-50 shot) a naïve person might claim confirmation. If the good word rice gets moldy first, a naïve person might think, "I must have done something wrong," or "I got so impatient waiting for mold, maybe my impatience threw it off." Such a person may be less likely to post their results on YouTube.

Now, I have no plans to conduct my hypothetical experiment. It's a lot of work putting together a well-controlled study. And I'm very confident the results would be uninteresting. You might say, "Put your money where your mouth is. Do the experiment!" In a sense I am putting my money where my mouth is. If I'm wrong, I am giving away, for free, a great way to win a cool million from the JREF. Have at it.

17 comments:

Doug said...

Love the article Pat. You sure put some thought into your study! And I of course love the Feynman quote :^)

Doug W.

Anonymous said...

IMO the rice experiment's goal, is to determine whether or not focused and continual "intention," can affect the outcome. IE can we create reality, or affect it in some way through thought?

If that is the purpose, I don't see it included in your experiment. I see every attempt to hide each labeled rice, from the particular intention the rice should be inheriting. The labels aren't the intention, they are an identification for appropriate application.
Also IMO, I see an overtly complex attempt at exposing variable, that is more likely to produce an unknown/unrealized amount of them.

Thought experiments are always trapped by their own presumptions. Simplicity and first hand accounts/experience, are the true data pool. Mastering an entire experiment yourself, targeting every "conceivable" bias, is in fact bias itself. Although asking for suggestions, slightly mitigates this.

The best way to perform this, is through massively harvested data. When we see personal bias / error from each particular accounting, we can be assured that the OVERALL study hasn't utilized "bias." By admitting it, rather than omitting it. No excuses. If the numbers land at or around chance, then we have our answer. If they are overwhelmingly polarized, we have quite another answer. Intriguing at least. Nice blog, and great application of thought/logic. That's my 2 cents.

McLir said...

I wrote this in response to a friend who had described the rice experiment as proof of "the power of words." So I structured the experiment to test the written words.
Instead, we might build an experiment that scientifically tests the intentions of words on rice decay. How can such a test be done as a double-blind, controlled, randomized study (DBCRS)? And (this is tricky) how can we know when to reveal the results?
I have an idea of how to do this. But I'd be curious how the previous poster would go about it.
The post claims that my method is "overtly complex." But I would argue it just satisfies the basics of a DBCRS. An intentionality test would need to have similar rigors.

The writer claims:
"Thought experiments are always trapped by their own presumptions. Simplicity and first hand accounts/experience, are the true data pool."
Can we agree that our results should be based on the highest quality evidence? That is the point of my essay. Also, what I offered is not a thought experiment. It is a proposal for how to design a real-world experiment--further an experiment design I am willing to give away to any fortune seeker (don't forget about that cool $1,000,000!).
"Mastering an entire experiment yourself, targeting every 'conceivable' bias, is in fact bias itself. Although asking for suggestions, slightly mitigates this."
I don't pretend to target every conceivable bias. I'm just controlling for the biases I can anticipate. The idea that limiting bias, itself, constitutes a bias--this is not explained. I don't get what the writer is saying.
The writer goes on, "The best way to perform this, is through massively harvested data. When we see personal bias / error from each particular accounting, we can be assured that the OVERALL study hasn't utilized 'bias.' By admitting it, rather than omitting it. No excuses."
I don't understand this. Let's say a study is "admitting" bias rather than "omitting" it. What does that mean? What would such a study look like? Would it have any controls? Would sloppy experiments be included? If so, why?
Again, I do have an idea of how to test the intentions of words on rice decay. If you have a design proposal, you can post it here. Or, even better, you can conduct the experiment and show the process and results. If true, it would be a great discovery!
And the JREF has a million bucks at the ready, to sweeten the deal.

Tyler Bitton said...

Dummass... try it! It frikkin works??

Tyler Bitton said...

Everytime...

Justin Bowditch said...

Is it possible that your thoughts during this experiment sabotaged the point of it. The experiment is not only about the words you use but the power of the thought behind it. If you go into something like this with a blind sense of skepticism, then you are not conducting the experiment as it was described. I too am skeptical, but the experiment clearly states that along with the words you must also emanate the thoughts and the feelings that correspond with the words. Failure to do that and you are not following the instructions correctly and thereby sabotage it.

Anonymous said...

Excellently put @Justin Bowditch

Aldebaran said...
This comment has been removed by the author.
Aldebaran said...

Hey Justin, I think the point of his very specific application and proposed execution of the experiment is to lay a strong foundation that eliminates the little idiosyncrasies that can cause a result where an unrelated conclusion could be arrived at, and attributed to falsely, without realizing it.

If we want to find out, and not just fool ourselves into believing in, whether or not the world can be affected by thought/intention alone, we MUST be rigorous, right? We are dealing with the fact that many of us would rather fool ourselves than be corrected, and that can be a very powerful "intention" in and of itself.

If "all" variables are controlled and the experiment STILL works, over and over, by different groups of folks, in different places and cultures in our world, THEN we'll have something promising.

Again I say controlling the variables allows the proper place for "intention" and "emanated feelings" to show their true power.

"But the experiment clearly states that along with the words you must also emanate the thoughts and the feelings that correspond with the words. Failure to do that and you [...] sabotage it."

You also sabotage the experiment by ignoring the fact that

a. mold grows everywhere

and

b. people see what they want to see, often in spite of wisdom and truth.

The popular little rice experiment, done in someone's dirty kitchen, only serves human pride.

joel plante said...

i would perform the experiment described here as well as one designed to test the intentions then i could not only prove / disprove the intention experiment but use this as a secondary set of data to compare after all if both experiments are positive then that in itself tells me that perhaps it wasnt intention and that it was something i missed in the setup of the experiments

Anonymous said...

the point of this experiment is lost in this persons inability to have an open mind. most likely caused by the fact that he paid money to a school to tell him what is always correct.
you have to see the words.. one jar can say "I HATE YOU" and another can say 'YOU ARE BEAUTIFUL".
the point is that the rice in each jar be subjected to intentions of the words.. the true mental thought the person directs at each individual jar.
the correct way is to do this 1000 times and see how often and/or fast the jars with the people constantly saying bad things to it molds and how often and/or fast the jar constantly being told bad things molds.
i think the person that wrote this is afraid to do it this way. and if they take down this post, that's proof enough for me that i'm on to something.
but forget the jars of rice.. you don't need a phd in science to know that good intentions and bad intentions have an effect on the physical world, you just need a little common sense and experience.
we all know of a person that has been encouraged their whole life to do well and has been given a good home and awesome parents with the best of loving intentions.. and we've all seen a person like this do very well in life. most of us have seen the exact opposite happen to another person too.
so if you don't believe the rice experiment, just look around you. it's right there.
and on an even broader scale you can look at the greed of our capitalist leaders of industry and know there intentions are essentially selfish and tainted by ego and lust and their actions of their intentions have brought about great physical change to our landscape and environment. and on the other side of that you have all the green people with intentions that are out of love and goodness and are against greed and ego and their intentions when applied in actions also have physical effect on the physical world.
and those who can not accept what i just said are "blind" and those who can't even accept my words as a possibility are "double-blind". peace and love everyone.

Christopher Gumb said...

Hey Anonymous! Great job giving an example of "how to back pedal" as explained in the article that you obviously read from start to finish! Here it is again for anyone else who may not have been so thorough:
"Blame science or Western thinking: This is the common tack of accusing the skeptical mindset of spoiling the results. The experiment above is designed without appealing to any particular cultural heritage. The design is based on me preventing myself from introducing bias. If scientific thinking is such a party-pooper, how has it been so successful in shaping every little bit of technology we use?"


You may also be interested in the equally valid truths of pyramid power and mystic crystals. Just do your research and I'm sure you'll agree. And by research, of course, I mean just decide to believe it.

Belles said...

Seems to me that the people who feel your *super-complicated* study proposal missed the point, have generally missed the point of scientific experimentation.

I had never heard of this rice-experiment nonsense before, but I find the attempts to redefine the scientific method in these comments both hilarious and sad.

I'd imagine the comments on YouTube would be 10x more painful.

Guga said...

I love when somebody says "you don't need science, you just need common sense". Common sense is what led people to think the Sun was revolving around the Earth.

Anyway, strangely enough, those who get upset at skeptical thinking (as if it were something bad, "we don't have an open mind", instead of "we just want to be sure we're not fooled") always find some tweaks that justify its not working (e.g. "you have to think those words!") that are ALWAYS in conflict with the scientific method.

Sorry guys, but Science is what is letting you vomit out this stupidities on open mindness and the power of words. Science is what gave us computers, the Internet. If you REALLY believe this idiotic rice experiment, why aren't you commenting on this blog by the mere power of your brains? Come on, don't tell me you don't believe in telepathic blogging... it's totally a thing, I saw it on youtube. One must have an open mind for it to work...

Anonymous said...

It's seems evident enough to the scientifically-minded that you aren't at all familiar with the purpose of the Emoto experiment. So be a skeptic and try it for yourself- many times. It's been easy to replicate for me and my classrooms. Then, if you are worth a grain of rice in the science realm- report your results. The true scientist is a seeker at heart- not a pseudo wannabe know it all.

Anonymous said...

FTR many of the most respected and powerfully impacting scientists of humankind relied on abstract theories that seemed dubious to those less willing to imagine something beyond the tangible or known. Emoto's experiment, while simple, does reliably illustrate human 'intention'. Apples fell off trees long before Newton called it gravity. Try it for yourself. In fact, send the hate jar love and the love jar hate just for giggles. The experiment will still work- regardless of the label you slap on it.

Patrick McComb said...

The two previous comments seem to be from the same person. You sound as though you successfully performed the "rice experiment" exercise several times. Can you provide a link to your methods and results?
If you can document statistically interesting results under proper experimental conditions, you could win the $1,000,000 I mentioned in the post. Just contact the James Randi Educational Foundation.
(BTW, "pseudo wannabe know it all" might qualify as a double-negative, but I'm not sure.)