What makes 03-03, 99-11 and 99-12 hard in CO?

[lstange]

I noticed that these three classifiers ("Take 'em Down", "El Presidente", and "Take your Choice") have lower high hit factors in Carry Optics than in Production. All other classifiers have the same HHF in CO and Production. This leads me to believe that CO HHFs were initially copied from Production, then later (when enough data has been collected) these three were adjusted down.

 

I'm still surprised by the direction of the difference, though. In Steel Challenge peak times are the same between CO and PROD, except for Accelerator where CO is half a second faster. This makes intuitive sense, but why would a red dot make it harder to shoot a certain USPSA classifier?

[ATLDave]

Small sample size.  The data used for the revised HF's had lots of data of highly skilled shooters in Production, while relatively few such shooters were part of the CO data at that point.  

 

Yet another example of the failings of the (nevertheless still roughly workable) classification system.

[daytona955i]

2 minutes ago, ATLDave said:

Small sample size.  The data used for the revised HF's had lots of data of highly skilled shooters in Production, while relatively few such shooters were part of the CO data at that point.  

 

Yet another example of the failings of the (nevertheless still roughly workable) classification system.

That's the only thing that would make sense. Most classifiers are shoot reload shoot, those three aren't much different and the dot with the same scoring (minor) should make the hhf better for CO.

[ATLDave]

A continuously-updating percentile system would quickly sort out this kind of weirdness.  The percentage system's sole focus on the far right end of the curve means that the presence or absence of world-class shooters and world-class efforts in the data pool dramatically skews outcomes throughout the whole range. 

 

But, it works approximately good enough for the low-stakes game of classification.  So HQ won't undertake the relatively modest task of coming up with a more sensible system.

[motosapiens]

It's not just a small sample size, it's poor math skills. The classifier hhf update was done in a way that doesn't make mathematical sense.

[lstange]

1 hour ago, ATLDave said:

Small sample size. 

Maybe for 03-03 and 99-12, but surely El Presidente is shot often enough.

 

It's also strange that those two relatively obscure classifiers have tailored CO HHF, while 75 others still borrow Production HHF.

[ATLDave]

You're looking at that backwards.  Precisely because those are classifiers that are shot fairly frequently, they likely had enough reported scores to meet some arbitrarily-set number of instances to have the CO data set used for its own HHF.  

 

There were people shooting those.  It was just disproportionately not the serious shooters at that point.  Again, USPSA's HHF percentage system means that it doesn't matter what the average shooter does on a stage, nor what most shooters do on a stage.  It only matters what the best shooters do on a stage - or, rather, have done up to the point the HHF gets set.  Early in a division's life, it is pretty reasonable to expect that few of the truly top shooters will be racking up classifiers in that division.  

 

So those were right in the sour spot - enough people shooting those classifiers in CO to get CO's own data used, but not enough serious shooters in CO there to set the HHF scores at places of similar skill to those in other divisions.

 

Again, this is mostly a consequence of the dumb percentage system.  It seems sensible because it seems similar to how we do match scoring, but since the purposes and processes are totally different, it's a dumb choice.  Have continuously updating percentiles and this weird condition doesn't last very long.

Edited April 12, 2019 by ATLDave

[pjb45]

7 hours ago, motosapiens said:

It's not just a small sample size, it's poor math skills. The classifier hhf update was done in a way that doesn't make mathematical sense.

Truth

 

Yes I am a dickhead but with 12 hours of graduate level statistics and research methodology and 8 years in research, I might have a tad bit of knowledge.  When a large amount of Production data is used, it overwhelms the additional new data associated with CO.

 

I would suggest, with data analysis, that data from two years of Nationals and Area Matches would radically change the classifier system.  USPSA should require at least 3 classifiers in those matches.  OOPS, I would suppose there would be a significant reduction in the percentages----bad PR. No ego stroking.

 

[MemphisMechanic]

2 minutes ago, pjb45 said:

I am a dickhead.

 

I would suggest... at Area Matches... USPSA should require at least 3 classifiers in those matches.

 

Yup. Most of the guys spending $600 to go visit an Area match for a weekend are gonna call you a dickhead if you try to stick them with extra classifiers, instead of sexy field courses.

[lstange]

1 hour ago, pjb45 said:

When a large amount of Production data is used, it overwhelms the additional new data associated with CO.

There is actually not that much difference now. CO attendance is already at about 75% of production, and the gap keeps shrinking.

 

If you go back more than a couple of years you run into data relevance issues. There's been quite a lot of progress recently, especially in training methodology.

[pjb45]

1 hour ago, MemphisMechanic said:

 

Yup. Most of the guys spending $600 to go visit an Area match for a weekend are gonna call you a dickhead if you try to stick them with extra classifiers, instead of sexy field courses.

 

$600?  Isn't that match fee for the Desert Classic?  But you are so correct.  The High Desert Classic in NM is a favorite because 90% of the stages were 29-32 rounds.  

 

Still, the data is the data. GIGO, egos aside.

[MemphisMechanic]

@pjb45 a lot of guys like to eat food several times a day, they regrettably have to pay for gas and other travel, and pretty much no one likes sleeping in their car.

 

500 rds of high-quality extra precise reloads aren’t free, even when you do the assembly yourself. Despite how we like to think about it.  

 

Edited April 13, 2019 by MemphisMechanic

[lstange]

All three were corrected in May 1st, 2019 high hit factor update and are now the same as in Production.