ATLDave

Tons of them here in GA.

Uh, no. Scores will continue to be added at all levels. If you want to exclude scores from those people who have posted fewer than 6 or 8 scores, or for the first 6 months after issuance of a first USPSA number, you could do that. That might be sensible... that way, you'd be measuring against the population of people who are at least somewhat committed, not the mere tourists who shot 3 matches then quit. HQ, of course, should have the data on what fraction of scores are posted by tourists and noobs. That would be good data to know/see before deciding what kind of gating is needed on the front end before letting data into the measured-against population.

Sure, but the differences identified in big matches can be stage planning, movement, prop manipulation, endurance/focus over multiple days of shooting... all the stuff the classifier system doesn't look at. Or it can be pure shooting. Or both. The classifier system just looks at shooting and gun-handling skills. A percentile-based system would allow the classification system to better identify the differences between top shooters in just the shooting stuff. Which may not matter much to them. See my earlier analogy to PGA tour pros not caring that much about handicaps... they're all playing heads-up on Thursday-Sunday. Same with the "real GM's."

I do not concede that the percentile system would be "flawed." It would be materially and fairly obviously better in a bunch of ways. No system is perfect, but it seems clearly superior to what we have now. The current system is, however, good enough to get by.

And on some they were getting lower. I've seen GM's and M's shoot certain classifiers very well, and in a way that they were pleased with - and then seen those show up as some ~80% in the system. You probably don't see that as often, because pretty soon MD's hear from their shooters that those classifiers are a waste of time, and then they don't get shot very often. It's too bad, because there are a couple of those that are kind of fun as stages. We bumbled into one of those as a match I MD'ed; the old HHF was jacked up (no idea what it is now, or if it even survived last weeks' purge), and basically nobody's score counted in the system - it was all excluded as "below" except for a few unfortunate souls near the top of a classification range who had it count to bring down their average. But, despite those classifiers, and the few where the HHF was "too easy," the system overall worked pretty well. Stoeger is definitely a smart and analytical guy. He's addressed the classification system a few times in the past on his podcast/youtube channel. IIRC, he identifies some problems with it, but mostly shrugs and says "it seems to work OK in the end." Incidentally, a percentile system could actually make the classification system interesting to those top level guys again, too. Whereas the current system "clips" at 100, a percentile system wouldn't... and could help identify the real differences between the top 1% and the top .01%.

Agreed. The shooters who are truly competing for national titles spend about as much time worrying about the classification system as PGA tour golfers spend worrying about handicaps - they're all playing off scratch in the big games, so WGAF? The classification system (like the handicapping system in golf) is about the non-world-class players, helping them identify a competitive peer group, and generally helping them track progress (or lack thereof) in basic shooting and gun-handling skills.

Good enough. Make 98.5% the boundary for GM, then. Unlike PCC? The significance is only in that division. All of this being in the open would further expose those divisions where the competition is currently so weak that they should perhaps not exist or be recognized in most matches.

I get what you're saying, but what was the dire consequence of not adjusting? I think there has been broad agreement that people are generally appropriately classed/grouped, with exceptions primarily for new divisions or for shameless re-shooters/grandbaggers.

Sure. If that data were available, anyone curious (and adept at Excel or another tool) could do that for themselves.

That's true - but remember the official USPSA position, which is that different divisions don't compete with one another. Since L-10 guys are only competing with other L-10 guys, that's the relevant population to compare against.

None of that is percentile. You are still being measured relative to a single point, fixed by the top HF on a stage. Percentile ranks places you versus the whole population. Which would be dumb at the match level - not enough data. But for national classifications (as in standardized testing), percentile makes more sense.

We could either do as moto suggests, or we could keep the same boundary lines as we have now. I.e., a GM is someone who can average (after whatever toss-outs/exclusions we keep in the system) classifier scores higher than 95% of the shooting population's posted scores. That doesn't seem unreasonable to me. The GMs would be the top 5%, the M's would be people better than 85%, and so on. If you think that's too generous, pick different lines.

It doesn't need to be completely changed. But the HHF's didn't need to be tweaked. Most people agreed the system, as a whole, broadly put people into roughly appropriate stacks for purposes of finding a competitive peer group. We're clearly past doing only the things for which there is an absolute need. As long as we've moved into just improving things that can be improved, let's make a real improvement.

Absolutely. That's why I would like to see us move to a percentile system, which would continually update versus the shooting population, versus periodically and arbitrarily updated HHF's chosen when and how HQ feels like it. A percentile system would be automatic in tracking the gains (or losses) of overall population performance. It would also very quickly adjust to new divisions or major gear changes. Several other advantages, too. See the other thread.

No, we use percentages, not percentiles, in the match points system.

But they are screwing with it. Largely, it appears to me, because "too many" people have been "hooking up" on the peak performance/hero-or-zero runs. Since they're screwing with it, I would prefer to see them move to a better system that doesn't require judgment-based handicapping of HHF's. See for my suggestion of a percentile-based system.

Yep. This is one of many problems with the HHF approach. The organization is forced to try to set a HHF that will either appropriately measure peak-performance/high-risk runs, or will appropriately measure "match-mode" runs. With a single HHF, you really cannot do both. Because of the ratcheting mechanisms in the classification system, the overall output (i.e., shooter's total classification) generally works OK to exclude most of the match-mode runs so everyone is getting their peak/high-risk runs measured and used as the basis. (Except that doesn't work as well for people in B class, which is why B class is so notoriously hard to leave.) A percentile (not percentage) system, as I proposed in another thread, would mitigate this problem.

Cool. How did you pull the data? Could we get just the frequency of scores plotted? HF on horizontal axis, number/frequency of score on vertical axis? I'm very curious to see the shape of the curves. Are most of them bell shaped, but with a big upward jump at the 0? Or a power law type curve descending from the zero? Or are some double-humped things, with distributions centered around different points depending on how many penalties were taken? If you can tell me how you're pulling the information, maybe I can play with it myself.

I would very much like to see the score data for any given classifier plotted on a curve. And then the new and old HHF's marked on the curve.

Seems that way to me, too. OK, so I'm a 79% shooter in LTD. What does that mean? Am I 79% as good as Shane Coley? No, I don't think that's accurate. Am I better than 79% of the LTD shooters in USPSA? No, but that's what percentiles would tell us (or disprove!). I'm probably at least a little more skilled than someone who is sub-70, and probably less skilled than someone who is over 85. That's about all I can say from the current system.

This goes to what you think the purpose is. If you look at my first two stated/assumed purposes, they involve shooting skill, as distinct from running. Or, for that matter, prop manipulation, stage planning, focus over the course of a long day, or any of a number of other factors that do have some influence on match results. If you think the classification system needs to test, in a representative way, the whole gamut of skills and qualities needed to win matches, then simply looking at weighted match results (the ELO model) is worthwhile. I think there's some real value in parsing out the core shooting and gun-handling skills - which are, by far, the largest part of overall match performance - and measuring those. But that's a philosophical question. No right or wrong answers there. Instead, my post assumes that we want the classification system to basically do what it does today, but do it in a better way.

Reasonable people could disagree about whether that's necessary, but I'd be ok with that.

Why would that be better? That would likely be worse. In no small part because you'd end up with mis-set HHF's even more frequently. Your observation that some "obscure" classifiers "get shot barely" is apt. The question is why? Well, a few are obscure because they have stupid/rare/specialized props or bay dimensional requirements that are rare. HQ just killed most of those. But there are others that are rare specifically because shooters and MD's have learned that the HHFs are so high that few shooters will post scores that actually count in the current system (as being "below" - i.e., more than 5% lower than the lower boundary of their current classification). A percentile system fixes that, while a percentage system will always struggle with it.

Irrelevant, since the measured-against population is all scores on record.

No, not at all like that. Percentile versus percentage. As I wrote, if you don't know the difference, the rest of the post won't make any sense.