_ Broken Classifiers

[shred]

I've been taking a look at some classifier data, just for the heck of it (note to USPSA HQ-- these do not reveal HHF or anything else that can't be directly obtained from the public web site, but if you want 'em yanked, let me know)

So, there's been talk of 'easy' and 'hard' classifiers...

Here's three graphs to check out. The purplish bars are the actual scores a whole bunch of random open shooters shot the classifier in. The blue line is the national % of open shooters in each class. Call this line "about where the scores should be"

Here's one of SG's favorites . The distribution at the low end is probably caused by 'screwup' runs, and there are a few too many GM runs, but the rest looks ok, so I'm going to call this one "HHF about right". About 15% shot M or GM scores and there are about 12% M's & GM's:

And here's what I'll call "HHF too low" (no kidding, almost 40% of the shooters shot M or GM scores)

And here's "HHF way too high" (check this out.. no GM scores, no M scores and only a few A's-- from another source, I hear the highest score of any GM is 83%). Remember all three graphs are approximately the same set of shooters:

So what can we do about it? You'd think USPSA HQ would raise the HHF on "HHF too low" by now-- existing scores won't change, which could be problematic, but the policy has been in place for years.

For "HHF too High" (which I'll reveal is 99-59), they have to revisit the HHF.. If none of the top GM's can beat 83%, something's wrong with it.

These vary a bit Open to Limited (and L-10, Prod & revo), but the summary's the same.

[Ron Ankeny]

Roy:

Just curious, how did you go about getting scores for a large enough population to make your stats valid? Seems like you would need to have all of the scores or a truly random sample. Did you go to the top 20 listings and get the scores from there?

I agree that the classifier is problematic if none of the top dogs can muster anything better than 83%. On the other hand, the HHF is calculated from the top scores of all shooters so we need to consider all of the grandbaggers and reshoots.

(Edited by Ron Ankeny at 9:24 pm on Sep. 18, 2002)

[Flexmoney]

Sounds to me like the office need to apply some basic statistics to the classifer data.

[shred]

My selection method was two-fold.  I picked a lot of USPSA numbers at random (there was some bias towards shooters that had shot a lot-- for example I didn't go above A40000), and batch-added numbers from online match results that showed they'd shot an "interesting" classifier)

Then the scrapers looked up the %'s on the uspsa web and mashed the data.  While some classifiers are more popular than others, there are ~80 datapoints in the least popular one here.  USPSA.org doesn't say how often each classifier has been shot, so who knows how many more scores are out there, but I believe the sample is fairly random, maybe biased a bit towards higher-classified shooters.

As for 99-59, a while back somebody searched all the GM's and M's they could find and found Ron Avery's 83% was the best, and only, score above 80%.

If USPSA would let me at the raw data, I'd love to do these right.. as it is now, there's enough manual work to do that I'm stopping.  After all, it started as a web-scraper exercise, and I think they work.

(Edited by shred at 10:22 pm on Sep. 18, 2002)

[Flexmoney]

Shred,

Were you ever able to get any info out of Sedro Wooley?

[wide45]

Re 99-59

They made a big change in the stage procedures, when it was rewritten for 99. If current scores are being ranked against the pre- 99 scores, it's no wonder that the %'s coming back are junk.

Check it out

http://www.uspsa.org/classifiers/cm98-18.gif

http://www.uspsa.org/classifiers/cm99-59.gif

[shred]

That could be why this one's screwed up so badly. There are some others that aren't quite as bad, but anybody that's ever calculated a HHF knows there are some that are way too round numbers to be an average of the best ten runs.

On the progress with Sedro, my reply was lost in the board conversion, but I got some of them interested back right before they got distracted by Shooters.com going under, and I've heard some analysis is happening, and have volunteered to do some more, but haven't heard anything for the past couple weeks.

[kaiserb]

I have been looking at some of my HF vs percents. It seems to be pretty easy to find out the HHF with a bit of quick math. However I noticed when I took some other club scores in the same division from various classifiers the HHF was different depending upon the shooter etc. Is the HHF based on a sliding scale, i.e. logarithmic? I just assumed it was based on simple linear formulas.

Any Ideas?

[Flexmoney]

The HHF is billed as an average of the top ten hit factors.

This average becomes the HHF. Everything else is a straight percentage of the HHF.

[ErikW]

http://64.62.172.100/~brianeno/index.php?a...62&hl=factor&s=

a discussion of how the high hit factors are made

[kaiserb]

If it is straight percentage I must have some erronious data from other sources.

I will work some more on it and come up with better data.

[shred]

For those people that didn't figure it out "HHF too low" up there is 99-32, now quit asking

[davecutts]

when our club shot 99-59 in a special classifier, the prez (MV) shot a 88.9. no one else even close.

[shred]

When was that? That would beat Ron Avery's 83% record..

[davecutts]

may or june.

[davecutts]

may or june, and I just got my new classification book, not ten minutes ago, 99-32 is gone, but 99-59 is still there.

[SRT Driver]

Just found a new tool. Here are the percentages that are needed to break into the top 5% of each division.

Open = 87.4%

LTD = 81.8%

L10 = 76.3%

Prod = 75.0%

Revo = 68.9%

The HHF formula seems correct for all but revolver when HHF's are calculated not averaged.

[Rob Boudrie]

Shred,

Were you ever able to get any info out of Sedro Wooley?

Why bother Sedro Woolley? Drop me a line and I'll get you what you need, or if it would be easier, I can put together some on-line code to extract the data you seek - just let me know exactly what stats you need in CVS ot TSV format and I'll see what I can put together.