There are, however some significant criticisms of the systems in which these lists are generated. Whilst many of the criticisms have been covered elsewhere, the one that I wish to pursue is the significant lack of transparency. In the information age, simply creating a statistic and publishing the results is not enough. Without access to the formulas used in determining the "worlds best batsman," one cannot have faith that the statistic is going to give an accurate ranking. Further, one is unable to modify the formula and thereby improve it.
Certainly, Wisden may feel that formulas used to construct its rankings are proprietary information. Whilst this is a valid concern, it reduces the public faith in the statistic. For example, if I was to say that Ricky Ponting scored 9.3 on the CaptainTron and Steve Waugh scored 6.1, whilst it would be debate worthy (and wrong) without knowing what determined the CaptainTron scores it would be hollow.
Wisden could learn from the baseball community (decades ahead in the use of statistics), in which the best statistical methodologies are made freely available to the public. Anyone who is interested in the process whereby some of the fantastic statistics are created would be well served to have a look at the freely available work at Fangraphs or TangoTiger.
So where does this leave us? Over the next couple of months, this blog is going to have a bowl at constructing a tool for player valuation. What's more, each step of the modeling procedure will be posted in stages. As a guy with a fairly solid metric background I am quietly confident that we will come up with something that is not only more useful than the Wisden stuff, but you will be able to see what goes into it rather than the Wisden Black Box.
This doesn't mean that my summer/autumn/winter of Ponting Baiting is at an end. There is always time for that.
4 comments:
Interesting idea professor. Looking forward to it. Despite the gross hypocrisy on my part to do so, I do have a complaint about the incessant ranking focus that cricket statisticians involve themselves.
The guys over at It Figures have this awesome statistical collection, yet almost all they do is rate things (normally badly, though David Barry is a notable exception on both fronts). There are so many interesting empirical questions that could be asked that aren't. The sorts of questions that sabermetricians are very good at, that test received wisdom empirically.
Actually Russ,
The ranking stuff was more a complaint aimed at the Wisden folks than the goal of what I'm doing. Rankings aren't really my bag though.
The project that I think I have the data for is trying to (over a considerable period) look at ODI win shares. Or at least put a framework together to do so.
Russ, could you write a list somewhere of questions that should be tried to be answered by "cricket sabermetrics"?
Not that I'll answer any of them soon (if nothing else, I'm in France and away from my cricket database for a few weeks) but it'd be nice to have collection of open problems in cricket stats somewhere.
While ranking can sometimes be a pointless exercise, it is worth pointing out that with a player transfer market in T20 leagues, there will be a very practical application of working out the overall value of each player.
I don't believe anyone who is interested in cricket gives a toss about ODI win shares. I don't believe anyone who is interested in cricket gives a toss about ODI - period.
What is desperately needed is a better means of ranking first and test class cricketers, like a cricket sabermeterics.
Post a Comment