The worth of a medal?

During a beer tasting which I attended last week, which included a number of the medal winners from the recent Beervana, the discussion turned to the usefulness, or otherwise, of such awards for the ranking of different beers. Do the rankings that come out of such shows really tell us much about the quality of the beers? Now this is not just a problem for beer awards, another obvious example of the issue is wine awards and there is some research that suggest that the rankings that come out of a given wine show aren't worth that much.

There is an interesting article, "An Analysis of the Concordance Among 13 U.S. Wine Competitions" by Robert T. Hodgon, in the Journal of Wine Economics, Vol. 4, No. 1, 1-9. The abstract reads,

An analysis of over 4000 wines entered in 13 U.S. wine competitions shows little concordance among the venues in awarding Gold medals. Of the 2,440 wines entered in more than three competitions, 47 percent received Gold medals, but 84 percent of these same wines also received no award in another competition. Thus, many wines that are viewed as extraordinarily good at some competitions are viewed as below average at others. An analysis of the number of Gold medals received in multiple competitions indicates that the probability of winning a Gold medal at one competition is stochastically independent of the probability of receiving a Gold at another competition, indicating that winning a Gold medal is greatly influenced by chance alone.

I would be willing to bet that beer wards are not that much different to wine awards in this regard.