New Study: Vibram 5 Fingers' Effect on Bone Edema

I know I said today I'd talk about Chris McDougall and barefoot evangelism, but the results of a new study were released today, and I wanted to comment on that first.

The study results were reported on MSN.com and other news outlets. MSN pitched as a study that suggested minimalist runners should eschew the performance benefits of the running style, and instead should stick to traditional shoes because of the impact on the bones in their feet. This seemed off to me; barefoot running is not usually undertaken in search of better performance, but rather as a means to avoid injury and run more naturally. The same runner shod will torch his times barefoot. Of course, if wearing shoes gets him injured all the time, it might not be worth it, and that's always been my interpretation of the barefoot philosophy. So the MSN article doesn't make a lot of sense, and sounds like it was written by someone who doesn't know what they're talking about. 

Of course, I don't much care what MSN thinks about the study. I want to read it myself. So I followed their link to the actual study. Unfortunately, due to copyright restrictions, I couldn't access the full paper. I could only read the abstract written by the study authors, which is helpful, but prevents me from really digging into their methodology, results, and statistical interpretations. 

Click Read More here or below for a more detailed discussion of the study, but here's the bottom line:

The Bottom Line: The MSN article is terrible and should be ignored. The study itself is sound methodologically, but suffers from some flaws in qualitative value. We don't know how the Vibram runners transitioned, so we don't know whether they did it the right way. We already know that switching from traditional to minimalist/barefoot too fast is stupid and will get you hurt. The p-value of the data is pretty good (p=.009, according to the abstract, which is within the traditional p=.01). This means there's almost a 99% chance these results proceed from an actual physical reality that Vibrams were causing bone bruising. This is a good start, but a lot more research needs to be done. The study's conclusion is sound: if you transition to minimalist/barefoot running, GO SLOW. 

Methodology

From the abstract, I can say that the study seems sound methodologically. There was a control group of traditionally-shod runners, and an experimental group of runners wearing Vibram 5 Fingers. All of them were experienced recreational runners, and they had MRIs before and after a ten week period, during which the experimental group attempted to transition to the Vibrams. 

So far, so good. But there are some still structural problems here.

Sample Size

Apologies for going Stats-101 on you here, but the sample size is too small to be drawing any firm conclusions. There were only 36 runners in the study- 19 in the Vibrams, 17 in traditional shoes. That's not many people, and doesn't tell you a whole lot about broader trends. This small of a sample population could return almost any result, and none of it would really mean very much. This study is a good start, but researchers need to do this over and over again, with much larger samples, before you can draw any meaningful conclusions from it. 

Training Regimens

The study is well-designed in that it assigns numeric values and explicit descriptions of the injury level of the runners post-experimental period. A "0" meant no edema was revealed in the post-MRI; a "4" denotes a full-on stress fracture. So we don't have a problem with the runners' opinions about their injuries. We have solid definitions of the injury evaluations, and I have no complaints. The study authors did a good job of using measurable criteria that was as objective as possible. 

What we don't have is any idea how these people transitioned. Say what you want about the barefoot/ minimalist community, but you have to admit that they are very concerned about new people transitioning from traditional shoes too quickly. No minimalist or barefoot runner will fail to caution you to take it easy or to tell you that you're essentially starting to run for the first time in your life. These aren't reckless people, and they don't throw newcomers to the wolves. 

But not everyone takes their advice. Maybe the full study contains information about this, but at least from the abstract, I have no way of knowing whether the new runners transitioned slowly, or if they strapped on some Vibrams and headed out for a 10-miler. That matters, because if they went too fast, no one should be surprised that they were getting hurt. 

We also don't know their footstrike tendencies. Did they try to transition to midfoot or forefoot striking, or did they continue to heel strike with abandon? You can still heel strike in Vibrams, I'm told. We have no way of evaluating the quality or the quantity of the transition exercise regimen, and that's a major problem with the study (with the caveat that this issue may be addressed in the full paper, which I cannot see). 

This is also only a 10-week snapshot. It's entirely possible that, after a 10-week transition, the edema problem will abate as the foot becomes stronger. The study was not designed to answer that question, but it's an important consideration. More studies are needed to determine how the transition affects runners on longer timeframes. A short burst of bruising would absolutely be worth it if it resulted in longer term health. This study, again, wasn't designed to figure that out, so I'm not criticizing the study, but only suggesting that this is not the only consideration. 

Statistical Significance

This will get a little math-ish, but I'll keep it conceptual. The p-value of the data, according to the study, is .009. The p-value is just a numerical expression of the chance that the data produced by the study was the result of random chance. If p=1, the data is completely random and meaningless. If p=0, the data is completely the result of the experiment, and there is no chance that randomness could produce the same results. Needless to say, p never equals 1 or 0. Traditional p-value tests of significance occur at p=.01 and p=.05, which is to say, data are considered "not random" when there is a 1% or 5% chance the experiment would have produced the results it obtained even if its hypothesis is, in reality, incorrect. In this case, it means that the data recorded in this study has a 0.9% of being produced even if wearing Vibrams and transitioning to running in them has no impact whatsoever on bone edema in the feet. 

That's about as close to bulletproof as you can get in statistics. What that means, though, is still not much. It can be said with a great deal of confidence that the data is not random chance. The possibility still exists, however, that these people transitioned wrongly, or just, by chance, happened to have weaker bones in their feet. The p-value can't tell us that. All we can say from it is that these runners probably did experience bone bruising as a result of their Vibram usage.

Bottom Line (Longer Version)
It's important to remember that 10 out of the 19 Vibram runners experienced more bruising than did the control group runners. That's an important consideration: the traditional runners probably had bruising too. It's just that half of the Vibram runners had more bruising than the control group. We don't know how far the experimental group ran, or if that compares to the mileage totals of the control group. We don't know how fast either group ran. It's likely they were told to do what they wanted, but that introduces a lot of variability into the study, and that shouldn't be overlooked. There are degrees of transition that can produce a wide range of outcomes.

The bottom line, again: Go Slow when you transition.