Following a contentious report from a workshop on handicapping issues held in November 2013, public submissions to the NHA were invited in June 2014.
KAREL MIEDEMA heeded the call and supplied this submission. It is intended to provide a rounded commentary on issues relating to the interpretation and use of NHA Rule 47.3.2, the definition of a handicap.
First, several opening statements are made, intended for further evaluation in the submission. These opening statements are not made lightly and deserve careful reflection by the reader. The content of this submission is based on four decades of hands-on experience with the subject of handicapping.
Athletic ability is innate (you either have it or you don’t), and it is measurable. There are parallels in this between human track athletes and racehorses. Once the inherent level of ability has been established it cannot improve.
Ability is expressed by equine athletes through racetrack performance, and is influenced by
- physical development (for a whole population there is an average development cycle related to ageing and aptitude, while noting that parts of the population may be precocious, others late-maturing);
- outside influences such as training for optimal race-fitness, and racing circumstances which include distance suitability, going, pace of a race, etc.
Weight-for-age is the expression of the average physical development made over time by equine athletes towards maturity, at which stage ability is established.
MR (Merit Rating) is a measure of ability expressed in terms of weight, showing the ability of a horse compared to that of the average horse in a population of the same age and sex.
The handicapper’s task is to
- set handicap weights for future races, i.e. predict future performance levels of equine athletes, based on historical data
- work according to Rule 47.3.2 and 54.1.1, the essence of which is that all horses in a race should have an equal chance of winning, assuming that suitable outside influences exist for all horses in the race (racefitness, distance, going, racepace, etc.)
- keep the level of Merit Ratings (i.e. the rating of the benchmark average horse) the same from year to year
The essence of a handicap race is:
- close finish (accuracy of handicapper’s prediction)
- open betting market (public perception of handicapper’s prediction)
How’s the handicapper doing?
To test whether the handicapper’s task is suitably fulfilled at present in terms of Rule 47.3.2, Merit Rated handicap races for the period 1 August 2013 to 31 May 2014 were evaluated.
A seemingly suitable measure is to see how far the fourth finisher is behind the winner (the first four likely to be horses which should be running close to their predicted ratings; four places also reflecting all the betting opportunities available).
There were 897 MR-handicap races contested on turf in the major centres (Jhb/Vaal, KZN, CT) during the period, varying in class, from MR 62 upwards.
The first four finished within one length in 7% of races, within two lengths in 30% of races, and within three lengths in 58% of races. Almost 90% of the time there was 5 lengths or less between the first four finishers.
All percentages held their own when races were split by the age of the winner (3yo vs older horses). This seems to suggest a high level of competitiveness, one length on average representing 2 MR points.
To get another perspective, the same sample period for other than MR-races (and excluding 2yo races) shows the fourth finisher is within 1 length 3% of the time (MR races is 7%), within 2 lengths 14% (MR 30%), within 3 lengths 32% (MR 58%), and within 5 lengths 66% (MR 90%).
The total number of races considered is 1185 (MR races is 897).
It seems that the application of weight and WFA, in a structured way, more than doubles the chances of the fourth finisher to be within a length of the winner compared to races where other methods are in place. And similarly, more than doubles the chance of finishing within 2 lengths, almost doubles the chance of finishing within 3 lengths, and increases the chance of finishing within 5 lengths by 50%.
It seems that MR handicapping (with WFA) has a significant effect of bringing horses much closer than the alternatives, not including 2yo racing.
Who earns the money?
To test the correlation between Merit Rating and actual earnings four samples were taken. The crop of horses born in 2008 (current 5yo’s), with earnings against MR, and against AR (the MR equivalent used by Sporting Post, to give a level of comparison).
Then the same for the crop of 1996, reflecting earnings which includes a period where at ages 2 and 3 MR-handicapping did not yet exist.
Does Weight-for-Age do the job?
Below are the bench-mark MR Averages, by age and sex, as at July of the years shown.
The table shows the average MR, split by sex, as at the end of the racing year for the specific ages – in other words, top-left are the 2yo’s with MR’s at the end of their 2yo racing-year in July 2010, shown in red; to see how the ratings progressed move to the 3yo’s of 2011, with ratings at the end of their 3yo career, also shown in red.
Similarly, follow the 2yo’s of 2011, in purple, or those of 2012, in black.
The numbers reflect the average MR for the age and sex, for horses that were in the handicapper’s active racing datafile at end-July of each year– meaning that from year to year the rating averages are for horse populations which are similar but not exactly the same (some 2yo’s don’t appear on the 3yo active file, and some 3yo’s did not race as 2yo’s).
A bottom cut-off point of MR 55 was used, to avoid the long tail of useless performers to unduly influence the averages.
For weight-for-age purposes, the 2yo and 3yo ratings are most relevant (2yo ratings are in fact 3yo start-of-season ratings, 3yo ratings are 3yo end-of-season ratings, therefore covering a year of change). When looking at older horses, there is a significant drop off in actual number of horses of an age, as a result of the poor ones being weeded out.
Do remember that Weight-for-Age is built into the Merit Ratings as shown.
The official Weight-for-Age table shows that 3yo’s are expected to improve by between 7kg to 9kg (depending on distance aptitude) during their 3yo year.
This equates to 14-18 MR points.
It follows that if 3yo horses, on average, did not make the assumed WFA progress then a substantial change should be seen in the averages from one year to the next (2yo benchmark rating to 3yo benchmark rating, as shown).
The table shows that changes are marginal, indicating that on average WFA fulfils its assumed role more than adequately.
To complete the evaluation, the progression was measured of individual horses from the 4-year sample above. Of main interest is the change from 2yo to 3yo (meaning end of 2yo career to end of 3yo career), since Weight-for-Age influence counts mostly during that period. The change from 3yo to 4yo (and further ages) is added for interest.
It looks like this:
All age-groups except the 2-3 year age group behave in the same way, the majority showing an increase of about 2 MR-points. The 2-3 year group (blue) is much flatter, indicating a larger number of changes, both up and down. The flattening is most marked in the range from -6 to +4 MR-points.
Explanations for this could be that some exposed 2yo’s do not make the assumed WFA-progress (2yo-precocity a likely reason), or that 2yo’s who were lightly raced at the end of their 2yo season had not yet found their level yet.
To test these assumptions, the mid-year MR of 3yo’s was taken (January) for each of the years, and compared to the 3yo end-of-season MRs.
The result below shows that each of the 3 years (blue, green and red lines) behaves in a similar manner, and that their graphs now much closer resemble those of the older age-groups shown above (the purple line is the 2-3 year one from above, included here to illustrate the change).
Matters raised in the Workshop
A substantial number of matters were raised in the workshop discussion report, many of these controversial in nature. They deserve comment (the reference numbers below relate to those in the document headed “Matters raised in the workshop”, with Workshop notes in italics).
2.2 Slippage and Uppage
- Handicapping with merit ratings (or equivalent) world-wide, tends to produce a gradual increase in benchmark ratings (average for the age groups) from year to year. Wanklin commented that in ‘many countries overall ratings are reviewed twice per year, to contain slippage and uppage’.
I take this to mean that population ratings are adjusted periodically to keep benchmark ratings in line. This also common practice with Timeform, the world’s oldest and most respected source of ratings.
Adjustment is one thing, but determining the cause of the creep is another. That cause is entirely due to the Weight-for-Age table, which in sections is too generous for younger horses.
I discovered this more than two decades ago. We use Weight-for-Age allowances which differ from Official ones, notably for 2yo’s and over a measure of ground for older horses. After adjustments, the earlier creep disappeared from our year-on-year benchmarks, to the extent that we now experience a slight decrease in benchmark ratings from year to year.
Since our methods did not change, the only likely cause for this reversal was the change in Weight-for-Age allowances.
Ratings for 2yo’s are an extremely important ingredient for containing creep. It is essential that a correct benchmark is established for 2yo’s, when (after racing mainly against their own age at 2) their ratings are finalised and integrated into the mother population of ratings.
Fortunately, rating 2yo’s is the most straightforward of all, so that integration can be seamless.
This leads to a comment made elsewhere in the documentation (see Philosophy, Aims and Objectives; point 1.5 MR as a global system – Initial Ratings). It reads:
- “MR’s are published at the end of the juvenile season, or after the first 3 starts. The entry point is a critical component for young horses. If the horse performs well in its first start, it may receive a high rating which hinders its prospect of winning again.
- The 2012-13 crop of 2yo’s had the highest ratings seen in a decade, leading to the notion that the ratings had commenced too highly, or the crop (current 3yo’s) were better than any other generation seen in more than 10 years.”
Two unsubstantiated statements in as many paragraphs.
Two-year-old ratings remain fluid during the 2yo season. Only when it becomes possible to establish a sensible benchmark do ratings become ‘official’ – in the SA scenario this is at the end of the 2yo season. The contention that if ‘a horse performs well in its first 2yo start it may get a high ratings which hinders its prospect of winning again’ is senseless – there are no 2yo handicaps, so ratings are irrelevant from a winning-races point of view at 2. Alternatively, if a horse wins that first start as a 3yo, it will get rated against existing ratings of horses already in the population. The statement that this might hinder the winner of winning again is a clear misunderstanding of the workings of Rule 47.3.2.
When 2yo ratings become official at the end of the season, the benchmark for the crop determines how far above that mark the best horses will get rated.
From the table shown earlier in this document, the benchmark for the 2yo’s of 2013 was not out of line, and also not the highest of the four years of 2yo’s in the table (2010 was; for interest, this was the year of Variety Club et al).
To give perspective, the highest rated of the 2013 crop were Captain Of All (114), Kochka (111), Willow Magic (111), Forest Indigo (110), Red Ray (108) – with the next highest on 105.
Captain Of All ran only once at 3, winning in November. Kochka did not race at all. Willow Magic only had 3 starts (in Oct/Nov). Forest Indigo raced five times between Nov to March (his rating went from 110 to 104 in December, possibly as a result of the reversal of the 6 point increase for the whole population). Red Ray had six starts and is currently rated 111.
2.6 Penalties in Graded Races
The heading of this item is unfortunate – horses are not penalised in terms of Merit Ratings, they are re-assessed, which is a major difference.
- Smith (chief handicapper) noted that ‘in Gr2, Gr3 and Listed races the local Rule allows the handicapper to only adjust ratings for the first four horses, while in Gr1 races this extends to the first five horses, but not to further unplaced horses’.
This, and the host of other arbitrarily designed local rules that affect rating of races, restricts the handicapper from performing his duties as dictated by Rule 47.3.2.
If this NHRA Rule is to be adhered to, as it surely should, all such restrictions must be removed forthwith.
Restrictions amount to no less than a wilful manipulation of the system – some horses will benefit to the detriment of others they will race against in future. Such a Robin Hood attitude must not be condoned.
2.7 Application of MR system in SA
This section consists of trainer’s comments.
Pertinent statements were made, many highly specific, without any attempt to quantify or substantially support these with some form of evidence.
Until such time as this is done, there is no reason to attach any weight to these comments.
2.8 The Weight-for-Age scale
Several observations made here deserve comment.
- Graeme Hawkins believed that the majority of young horses do not improve according to the WFA scale.
Evidence from the tables above is to the contrary. In addition, Hawkins did not substantiate his belief.
- Mike De Kock noted that horses’ actual birthdays are different, and the scale assumes every horse is born on 1 August, when in fact they are born between 1 August and 31 December. The difference in the scale between these dates is 4kg, so assuming all horses are born on 1 August is a bad assumption, as those born in December will automatically be 4kg over handicapped.
De Kock’s observation implies that he believes WFA works.
There is an error of thought, though, which is best explained by way of an example.
Two 3yo’s, one born in August and the other in December, meet early at 3. They carry level weight, run a dead-heat, and therefore are rated the same.
When they meet again, 6 months later on the same terms, they should again dead-heat, as the progression of the WFA scale shows the same rate of improvement for both. Their ratings are still the same.
Another six months on, and now the older 3yo horse has come to the end of its WFA improvement in real terms, because of the August birthdate. The younger 3yo horse, however, still has 4kg improvement to come in real terms because of its later birthday. Next time, he’ll beat the older 3yo horse handsomely, and his rating will be raised to reflect his real ability.
The reward comes, be it belatedly!
This phenomenon is why late-born 3yo’s are able to show sudden improvement late at 3 or early at 4, not because they improved in ability, but purely because of WFA progress.
De Kock is right when it comes to 3yo Classic races (or other races where horses meet on level terms) – there the difference in birthdates can be regarded as a genuine disadvantage, but that’s just the way the birthday-cards fell. Revenge can be had at a later stage.
- Statement: The WFA scale is built into the ratings and it was suggested that a fundamental change that might be considered was to handicap horses off net-ratings and then carry those ratings forward, ignoring WFA improvement.
- This fundamental change would help owners by permitting horses to win their way through.
- Also noted that it appeared common place for 2yo’s who had been accorded high ratings after a couple of starts, had been unable to maintain their ratings.
The suggested change might be a quantum leap for young horses and is worthy of comment and consideration.
No evidence is offered about the young horses ‘unable to maintain their high ratings’. Analysis shown elsewhere in this document appears not to support the hypothesis.
To handicap horses off net-ratings, ignoring WFA improvement, has implications.
For one, it would make the handicapper act in contravention of NHRA Rule 47.3.2. Secondly, in practical terms, how will the handicapper rate horses?
This is best illustrated by way of example.
A field of horses aged 3yo and up meets. The handicapper rates them, ignoring WFA.
The same field of horses meets 3 months later, with weights based on the ratings from the outcome of the first encounter. The result will be substantially different. The older horses may reproduce their comparative ratings, but the 3yo’s will run above theirs if the older horses are used as benchmark. So their ratings go up, even if they couldn’t run in the first three.
Or conversely, if the 3yo’s are used as benchmark, the older horses must be dropped, even if they finished 1-2-3.
And that’s from just one race, without any criss-cross formlines.
The system will likely degrade into an untangleable mass of contradictory information, which will be virtually impossible to un-do.
2.9 Rights of Owners and Progress of Young Horses
- It was noted that in the population of horses, there were more 7-time winners rated below 70 than rated above 70.
- This statistic appears to be an indictment of the MR system actually rewarding merit by permitting prizemoney to be distributed according to ability. If this is correct, it goes to the heart of subsidising mediocrity at the expense of ability; it is counter productive to the selection process for improving the breed and it favours punters above the rights of owners.
- Furthermore, it also means that the MR system advantages the retention of weak horses at the expense of better horses, which is surely not fair.
It is also an indictment against including WFA in the penalty system for young horses as it forces young horses up the scale too quickly. As a result it is commonplace in handicaps every day for young 1- and 2-time winners to have to concede weight to much older, tougher campaigners who have been racing for years and worked their way down to the lower echelons of handicaps where they have an advantage over their more talented younger rivals. The question must be asked is this desirable.
- A potential solution may be not to include WFA improvement in penalties on young horses. By de-linking WFA, it may be that younger horses will be less adversely affected. What is the feeling for this assertion?
Whatever happened to NHRA Rule 47.3.2?
Best of all is the opening statement, of there being more 7-time winners rated below 70 than above. Reality is somewhat different.
At the time of the workshop, the handicapper’s active database contained precisely 149 winners of 7 races or more. Of these, 129 had ratings of 70 or more (and 120 were 75 or more, and 103 were 80 or more).
The 20 horses rated below 70 consisted of 3x 4yo, 3x 5yo, 5x 6yo, 5x 7yo, 3x 8yo and a lone 10yo. Not a single one of these had earned less than R250k, but they were a burnt-out motley lot, way removed from their earlier lofty career-high MR’s.
So these twenty are going to stop the better younger horses from earning?
As far as the MR system subsiding mediocrity at the expense of ability, that notion has been dealt with in detail earlier in this submission.
It might be instructive, though, to see the full list of 7+-time winners, with earnings, career high MR and current MR at the time of the workshop.
This list is included as appendix.
2.11 Line Horse Fixation
- Lips expressed the view that ‘the handicappers must be allowed to assess a race openly – and not to be tied down to assessing ratings only in relation to the line/yardstick horse.
- He made two points
– the line horse is just a starting point
– more important is the interpretation of the result, considering all other factors: distance, pace, draw, tactics, interference, jockeyship, track conditions, etc.
- Try to find something in the puzzle of results that makes sense.
- He believed that criticisms of handicappers that arise are due to the fact that handicappers are not being allowed to interpret and are forced into a rigid approach. He recommended more freedom for the handicappers.
This goes to the heart of the matter. The handicapper is the hands-on expert, with Rule 47.3.2 as his leading light.
There should be absolutely no restrictions on how the handicapper achieves his goal, as long as he applies the tried and tested methods used by expert handicappers world-wide. These unquestionably include the use of Weight-for-Age.
Rather than tie the handicapper down and force him to break Rules, give him a better toolbox.
Sectional timing, for one, is one of the most important tools that ought to be available – as anyone who has experience with it will attest to (we use it as a matter of course, by hand timing all recorded races to evaluate pace of the race – probably the most misunderstood effect on the outcome of races).
Horse racing must be one of the few sports in the world today which is not embracing the runaway progress of technology, and doesn’t answer the call of providing the information-hungry market with what it needs to survive. Certainly, this is the case in South Africa, where untapped markets await if technology is applied to its fullest extend.
One item here is worth of comment.
- If the general benchmark of a Gr1 winner is 115, why should 2yo’s be less? The answer is that generally, on such ratings, competing against other age groups, they can’t win. Higher ratings given last year for 2yo’s have had to come down as 3yo’s.
Unsubstantiated and wrong – again.
There is no general benchmark for a Gr1 winner – that is cart before the horse. The rating is derived from the benchmark rating for the whole population as an entity. Winners of Gr1 races then make up a group, showing what kind of rating is expected to win such races.
The statement that they can’t win against other age-groups is unsubstantiated.
When highclass 2yo’s turn three, they have the Classics in focus – there are few opportunities at Gr1 level to compete against their elders. Historical data of such races (at WFA), if anything, shows more 3yo’s winning – possibly a result of the WFA scale being too generous.
The statement that last year’s high ratings had to come down is incorrect (and explained above under ‘Slippage and Uppage’).
There is nothing to suggest that the handicapping methods used in South Africa are not working in the way they should.
Which is to be fully expected from a system that is in use world-wide, both officially and more notably commercially (Timeform, Racing Post/Raceform, Sporting Post). It is interesting to note that Tattersalls sale catalogues now display the official BHA (British Horseracing Authority) ratings for sales where such ratings are appropriate (notably horses-in-training sales, to indicate current ratings). As an aside, in South Africa the Equimark sale catalogues went even further, a decade or so ago, by showing MR wherever possible in the pedigrees – this was abandoned in later years, because of complaints by breeders who did not want the full picture to be shown! Sporting Post still produces a buyer’s guide for current yearling sales, where dams and siblings reveal their ratings.
Tampering with Weight-for-Age, both in its general application or adjustment of the scale, is highly unadvisable.
Consequences are not easily predicted and are likely to surface only in time.
Once the system is infected, a cure will be hard to find.
When MR handicapping was introduced in South Africa in the late ‘Nineties, it had never been the intention for the old system of rating by ‘races won’ (Racefigures, Plate-racing) to be abandoned to the extend it was (I was a member of the sub-committee at the time, and this is to my best recollection). This may have been misunderstood by the programming committees, or more likely, there were not enough opportunities in terms of available races to do both.
The best racing-diet would include a well thought-out mixture of handicaps and conditions races.
If the aim is to allow the better young horses to reach their appropriate earnings level faster, then a solution could be to keep ages apart, especially when maturity development matters most – in the first half of the 3yo career.
That suggests 3yo’s should race against 3yo’s, with a leaning towards having more conditions-race opportunities in the first half of that season.
I note that in the UK most handicaps are reserved either for 3yo’s, or for 4yo’s up – keeping the ages apart, in other words.
Ideally it should be possible for horses to race frequently, while able to avoid a category they do not want to be exposed to – realistically, though, there simply may not be enough races available to program this way.
One solution could be to program conditions races in lieu of the lowest grade MR handicaps, which serve little purpose to anyone – not punters (the horses are inconsistent), not owners (earnings will remain ad-hoc for the many stragglers), not breeders (if bad horses fall off quicker, more new ones can be sold), not statisticians (earnings of bad horses distort many categories, notably breeding statistics and catalogue information – of the R88 million allocated to MR races in the major centres this season, R15 million, or 17%, went to earners in races of MR70 and below and, strangely, of that R15 million more than R9 million was for races restricted to fillies, further influencing breeding stats).
Some pertinent comment on Weight-for-Age appeared in the Timeform Annual of 2011, in the essay for Canford Cliffs. The extract is quite long, but I left it like that so as not to get things out of context. I have highlighted the most relevant bits in italics.
“The weight-for-age scale is often nowadays a topic for heated debate at around the time of the Sussex Stakes at the end of July, and the latest season proved no different. Three-year-olds have a very good record in the race and Frankel was the twenty-sixth to win in the thirty-seven runnings since another Two Thousand Guineas winner Bolkonski became the first of the six winners of the Sussex saddled by Frankel’s trainer (the race was opened to 4yo’s in the ‘sixties, and to 5yo’s and older in Bolkonski’s year). No horse has won the Sussex more than once, and Canford Cliffs was the second very good 3yo winner of the race in succession to be beaten by a 3yo when attempting to follow up (Canford Cliffs himself beat 4yo Rip Van Winkle, who recorded the best performance in the race in thirty years when beating top-4yo Paco Boy the previous year).
The official weight-for-age scale is arguably a little too generous for 3yo’s at a mile upwards after the middle of summer. They receive 6lb in the Sussex Stakes on Timeform’s scale, compared to 8lb on the BHA scale). However, there can be no such thing as the perfect scale; all are bound to have imperfections arising from the fact that all racehorses are treated for weight-for-age purposes, within their own age group, as if they were exactly the same age (even if the weight-for-age scale was tailored to actual foaling dates, some horses mature faster than others). There are too many variables for one weight-for-age scale to be proved categorically right and another categorically wrong. What can be said, however, is that if the scale used by the Timeform handicappers was a long way out, 3yo’s racing against older horses would run to different ratings – either consistently higher, or lower – than those achieved in races against horses of their own age. As it is, their Timeform ratings are generally compatible. The International Federation of Horse Racing Authorities decided towards the end of the latest season – in which the 3yo’s did well in top open-aged events – to set up a sub-committee to conduct an in-depth investigation of the present official scale which was last modified in 1995. Significant amendments do need to be made to the BHA’s 2yo weight-for-age – which is far too generous – but it will be a surprise if any more than minor adjustments are recommended elsewhere when the sub-committee reports back some time in 2012.
Calls are sometimes made for weight-for-age to be abolished in championship races on the grounds that no other sport gives younger or more immature performers allowances against the best. The benefit for racing having weight-for-age and sex-allowances is that they intensify competition between age groups, which genders public interest, as well as allowing direct comparisons to be made between successive classic crops. The 3yo’s as a group would not hold their own against their elders at level weights, certainly not at a mile and upwards, and abandoning weight-for-age would turn back the clock to the days when the classic crop rarely raced against horses from outside their own age group – the days before championship races like the Prix de l’Arc and King George and, at a mile, the Sussex, Prix du Moulin and Queen Elizabeth II began to challenge the supremacy of classic races by bringing together the best of different generations. Races like those mentioned soon became established as racing’s most informative championship races, and amongst its biggest attractions. In addition, nearly all the major global events have been open to 3yo’s and older horses, giving the potential for challenges between the generations which provide the big events that the public, and the media, want and will support.”
Winners of seven races or more, present in the MR database at the time of Workshop (November 2013), in order of MR, shown alongside the career-high MR (MRhi).
|What A Winter||5||15||3322088||121||121|
|Cherry On The Top||3||8||1867175||112||112|
|Tales Of Bravery||6||9||1351706||113||110|
|Blaze Of Fire||4||7||805775||107||107|
|In A Rush||4||7||557425||105||105|
|Bois De Var||6||7||475037||105||103|
|Halfway To Heaven||5||7||443350||103||100|
|S’Il Vous Plait||5||7||555950||99||99|
|Shades Of Indigo||7||7||689325||102||96|
|Lion In Winter||6||8||738364||112||95|
|Across The Ice||6||8||859377||107||93|
|Where’s My Chop||6||10||586600||90||88|
|Raise The Bar||5||8||708500||103||85|
|Follow The Piper||5||10||795750||105||82|
|Ryan’s A Lion||8||10||814950||87||80|
|D J Detzky||6||8||553200||99||79|
|Eye Of The World||6||14||810450||102||74|
|Campo De Santana||5||8||463725||91||68|
|Copa De La Rey||10||11||620995||107||66|
|Ace Of Hearts||8||7||455450||84||65|
|Made Of Magic||7||7||552375||87||65|