Jump to content
Brian Enos's Forums... Maku mozo!

New Classifier Percentages


B585

Recommended Posts

15 hours ago, Balakay said:

Here's another fun fact:

 

Solid GM, top 10 at a recent Area match...Only 1 out of the last 68 classifiers is greater than 95%.  1/68!!!  I would have better numbers batting against Justin Verlander.

 

This clearly begs the question, who is setting the high hit factors? The lack of transparency intrigues me to say the least.  

 

Just a thought, once someone makes GM they may not push as hard on classifiers. You can shoot a safe run with out taking any risk and still beat most people on a classifier stage. If I want to win the local club match, I'll take 80% runs on classifiers all day. I don't think I'm going to win the match on a 60 point stage, but you can loose it real quick.

Link to comment
Share on other sites

  • Replies 335
  • Created
  • Last Reply

Top Posters In This Topic

12 hours ago, malobukov said:

 

Here are Q-Q plots of the hit factors for the most popular classifiers in production. I only looked at a relatively small recent sample, but in absence of official data it's still better than guessing.

 

Cool.  How did you pull the data?  Could we get just the frequency of scores plotted?  HF on horizontal axis, number/frequency of score on vertical axis?  I'm very curious to see the shape of the curves.  Are most of them bell shaped, but with a big upward jump at the 0?  Or a power law type curve descending from the zero?  Or are some double-humped things, with distributions centered around different points depending on how many penalties were taken? 

 

If you can tell me how you're pulling the information, maybe I can play with it myself.   

Link to comment
Share on other sites

14 hours ago, Flyingpig said:

Actually heres a look at the best of the best and it tells a slighly different story. Way more consistent. This is a stretch in 2014. One  might be a major win but most are not. I counted 26 100% including major wins.

 

production shooter Local GM(and one of the best):

 

  97.3850 - 11/11/14 Legacy

 

  100.0000 - 10/08/14 Legacy

 

  96.1570 - 10/06/14 Legacy

 

  100.0000 - 10/13/14 Legacy

 

  75.8050 - 9/10/14 Legacy

 

  98.9668 - 9/10/14 Legacy

 

  91.2030 - 8/11/14 Legacy

 

  97.4359 - 8/12/14 Legacy

 

  100.0000 - 7/10/14 Legacy

 

  100.0000 - 7/09/14 Legacy

 

  100.0000 - 6/11/14 Legacy

 

  100.0000 - 6/05/14 Legacy

 

 

That's the kind of consistency I would expect from a member of the super squad.

Link to comment
Share on other sites

14 minutes ago, Racinready300ex said:

 

Just a thought, once someone makes GM they may not push as hard on classifiers. You can shoot a safe run with out taking any risk and still beat most people on a classifier stage. If I want to win the local club match, I'll take 80% runs on classifiers all day. I don't think I'm going to win the match on a 60 point stage, but you can loose it real quick.

Yep.  This is one of many problems with the HHF approach.  The organization is forced to try to set a HHF that will either appropriately measure peak-performance/high-risk runs, or will appropriately measure "match-mode" runs.  With a single HHF, you really cannot do both.  Because of the ratcheting mechanisms in the classification system, the overall output (i.e., shooter's total classification) generally works OK to exclude most of the match-mode runs so everyone is getting their peak/high-risk runs measured and used as the basis.  (Except that doesn't work as well for people in B class, which is why B class is so notoriously hard to leave.)  

 

A percentile (not percentage) system, as I proposed in another thread, would mitigate this problem.  

Link to comment
Share on other sites

44 minutes ago, ATLDave said:

The organization is forced to try to set a HHF that will either appropriately measure peak-performance/high-risk runs, or will appropriately measure "match-mode" runs. 

 

Really not. Find what works to get a fair spread of people in the classifications  according to skill level & then don't screw with it. Like handicapping is used in every other sport I can think of. 

Link to comment
Share on other sites

16 minutes ago, ATLDave said:

Yep.  This is one of many problems with the HHF approach.  The organization is forced to try to set a HHF that will either appropriately measure peak-performance/high-risk runs, or will appropriately measure "match-mode" runs.  With a single HHF, you really cannot do both.  Because of the ratcheting mechanisms in the classification system, the overall output (i.e., shooter's total classification) generally works OK to exclude most of the match-mode runs so everyone is getting their peak/high-risk runs measured and used as the basis.  (Except that doesn't work as well for people in B class, which is why B class is so notoriously hard to leave.)  

 

A percentile (not percentage) system, as I proposed in another thread, would mitigate this problem.  

 

I don't know, I think it's all in how you approach them. If you approach them as just a stage you should be fine. If you do the "hero or zero" that may lead you to being over classified. Most of the guys I watch try to hero it shoot a GM splits with bad hits and stay stuck in B class. It takes some pretty solid fundamentals to put it all together and get to the higher classifications.

 

The "I f-ed up a little so I'm going to put 4 in the NS so I zero it" thing people do is pretty lame. And skews the results.

 

I think being able to shoot 95%-100% at least was doable. Time will tell if these HF's are to high, they can always tweak them again in a few months if needed. Personally I've only shot two since the change, a 92 and a 96. The 92 was minor with a  HHF is based on Major. Both were about as good as I'd expect of myself in match conditions.  The one that was 92 was only 2.4 sec. with one Charlie. So even the tiniest mistake can add a few tenths and kill your run. Those to me are the hard ones.

Link to comment
Share on other sites

33 minutes ago, IHAVEGAS said:

 

Really not. Find what works to get a fair spread of people in the classifications  according to skill level & then don't screw with it. Like handicapping is used in every other sport I can think of. 

But they are screwing with it.  Largely, it appears to me, because "too many" people have been "hooking up" on the peak performance/hero-or-zero runs.  

 

Since they're screwing with it, I would prefer to see them move to a better system that doesn't require judgment-based handicapping of HHF's.  See 

for my suggestion of a percentile-based system.   

Edited by ATLDave
Link to comment
Share on other sites

26 minutes ago, ATLDave said:

But they are screwing with it.  Largely, it appears to me, because "too many" people have been "hooking up" on the peak performance/hero-or-zero runs.  

 

Since they're screwing with it, I would prefer to see them move to a better system that doesn't require judgment-based handicapping of HHF's.  See 

for my suggestion of a percentile-based system.   

 

The system has always been (or was supposed to be) based on the HHF of top level shooters. Our entire sport is based on percentages of the stage winner. To me it makes sense to keep the classification system based on a percentage of what top shooters can do on these stages.

 

Since it is supposed to be based on what the top shooters can do, it should be adjusted  from time to time to accurately reflect that. People can shoot El prez a hell of a lot faster now then they used to.

Link to comment
Share on other sites

15 minutes ago, Racinready300ex said:

Since it is supposed to be based on what the top shooters can do, it should be adjusted  from time to time to accurately reflect that. People can shoot El prez a hell of a lot faster now then they used to.

Absolutely.  That's why I would like to see us move to a percentile system, which would continually update versus the shooting population, versus periodically and arbitrarily updated HHF's chosen when and how HQ feels like it.  A percentile system would be automatic in tracking the gains (or losses) of overall population performance.  It would also very quickly adjust to new divisions or major gear changes.  Several other advantages, too.  See the other thread.

Link to comment
Share on other sites

It doesn't need to be completely changed.  But the HHF's didn't need to be tweaked.  Most people agreed the system, as a whole, broadly put people into roughly appropriate stacks for purposes of finding a competitive peer group.  

 

We're clearly past doing only the things for which there is an absolute need.  As long as we've moved into just improving things that can be improved, let's make a real improvement.  

Link to comment
Share on other sites

41 minutes ago, Racinready300ex said:

 

The system has always been (or was supposed to be) based on the HHF of top level shooters. Our entire sport is based on percentages of the stage winner. To me it makes sense to keep the classification system based on a percentage of what top shooters can do on these stages.

 

Since it is supposed to be based on what the top shooters can do, it should be adjusted  from time to time to accurately reflect that. People can shoot El prez a hell of a lot faster now then they used to.

 

you're talking apples and oranges now. Percentage of stage winner is based on what the stage winner shot that day in a match he was trying to win. Classifier HHF's *appear* to be based more on hero/zero tomfoolery.

Link to comment
Share on other sites

Just now, ATLDave said:

It doesn't need to be completely changed.  But the HHF's didn't need to be tweaked.  Most people agreed the system, as a whole, broadly put people into roughly appropriate stacks for purposes of finding a competitive peer group.  

 

We're clearly past doing only the things for which there is an absolute need.  As long as we've moved into just improving things that can be improved, let's make a real improvement.  

I agree

Link to comment
Share on other sites

2 minutes ago, ATLDave said:

It doesn't need to be completely changed.  But the HHF's didn't need to be tweaked.  Most people agreed the system, as a whole, broadly put people into roughly appropriate stacks for purposes of finding a competitive peer group.  

 

We're clearly past doing only the things for which there is an absolute need.  As long as we've moved into just improving things that can be improved, let's make a real improvement.  

 

Some of the HHF did NEED to be adjusted. I don't know if they all did.

 

I shot one the other day and dropped my mag on the reload and still hit like a solid Master time. That probably needed a higher HF.

 

El prez was easy. I saw that at a match the other day and thought "easy 100" then the match got rained out, then they adjusted the HF lol

Link to comment
Share on other sites

4 minutes ago, motosapiens said:

 

you're talking apples and oranges now. Percentage of stage winner is based on what the stage winner shot that day in a match he was trying to win. Classifier HHF's *appear* to be based more on hero/zero tomfoolery.

 

"Appear to be hero/zero" But the intent is to have a repeatable stage that can be shot anywhere in the country at any time and present the same challenge. Very similar to a stage at a match and with the HHF being the stage winner. The biggest mistake they made was not explaining how and what they did. I think that led to all the speculation on the Internets.  Really we probably should ask top level GM's if these new HHF's are reasonable. But, most of them are training not arguing about the classification system on the internets.

Link to comment
Share on other sites

5 minutes ago, Racinready300ex said:

 

Some of the HHF did NEED to be adjusted. I don't know if they all did.

I get what you're saying, but what was the dire consequence of not adjusting?  I think there has been broad agreement that people are generally appropriately classed/grouped, with exceptions primarily for new divisions or for shameless re-shooters/grandbaggers.  

Link to comment
Share on other sites

14 minutes ago, Racinready300ex said:

  Really we probably should ask top level GM's if these new HHF's are reasonable. But, most of them are training not arguing about the classification system on the internets.

Agreed.  The shooters who are truly competing for national titles spend about as much time worrying about the classification system as PGA tour golfers spend worrying about handicaps - they're all playing off scratch in the big games, so WGAF?  

 

The classification system (like the handicapping system in golf) is about the non-world-class players, helping them identify a competitive peer group, and generally helping them track progress (or lack thereof) in basic shooting and gun-handling skills.  

Link to comment
Share on other sites

30 minutes ago, ATLDave said:

I get what you're saying, but what was the dire consequence of not adjusting?  I think there has been broad agreement that people are generally appropriately classed/grouped, with exceptions primarily for new divisions or for shameless re-shooters/grandbaggers.  

 

Simple on some classifiers people were getting higher % than they should, over classified

 

15 minutes ago, ATLDave said:

Agreed.  The shooters who are truly competing for national titles spend about as much time worrying about the classification system as PGA tour golfers spend worrying about handicaps - they're all playing off scratch in the big games, so WGAF?  

 

The classification system (like the handicapping system in golf) is about the non-world-class players, helping them identify a competitive peer group, and generally helping them track progress (or lack thereof) in basic shooting and gun-handling skills.  

 

I understand who the system is for, someone like Stoeger who has talked about "the 100 project" I think he called it, has really dug into these skill tests. He probably has a good idea of where the HHF should be at least for his division. I don't think the current Admin is going to reach out to him for input. And I don't think he follows the Enos drama close enough to find this thread.

 

That's the point I was making, I wasn't saying WGAF.

Link to comment
Share on other sites

1 hour ago, Racinready300ex said:

I don't think the system needs to be completely changed.

I think current system works fine. It can be gamed, but if everyone is gaming then it’s still a fair comparison. In most cases a GM still beats a B class in a match.

 

Some HHFs may be too high or too low, but averaging six results evens it out so classification percent is a good representation of current shooting ability.

 

I tried to come up with a better system, but so far none of my attempts were quantifiably better.

 

If you don’t like hero runs specifically, this is easy to fix. Use last 8 classifier results to calculate classification percentage, without excluding anything. To prevent sandbagging, don’t lower classification. Chess seems to be doing fine. Your Elo rating can drop after a loss, but once a GM you’re still a GM.

Link to comment
Share on other sites

23 minutes ago, Racinready300ex said:

 

Simple on some classifiers people were getting higher % than they should, over classified

 

And on some they were getting lower.  I've seen GM's and M's shoot certain classifiers very well, and in a way that they were pleased with - and then seen those show up as some ~80% in the system.  You probably don't see that as often, because pretty soon MD's hear from their shooters that those classifiers are a waste of time, and then they don't get shot very often.  It's too bad, because there are a couple of those that are kind of fun as stages.  We bumbled into one of those as a match I MD'ed; the old HHF was jacked up (no idea what it is now, or if it even survived last weeks' purge), and basically nobody's score counted in the system - it was all excluded as "below" except for a few unfortunate souls near the top of a classification range who had it count to bring down their average.  

 

But, despite those classifiers, and the few where the HHF was "too easy," the system overall worked pretty well.  

 

Stoeger is definitely a smart and analytical guy.  He's addressed the classification system a few times in the past on his podcast/youtube channel.  IIRC, he identifies some problems with it, but mostly shrugs and says "it seems to work OK in the end."  

 

Incidentally, a percentile system could actually make the classification system interesting to those top level guys again, too.  Whereas the current system "clips" at 100, a percentile system wouldn't... and could help identify the real differences between the top 1% and the top .01%.  

Edited by ATLDave
Link to comment
Share on other sites

Sure, but the differences identified in big matches can be stage planning, movement, prop manipulation, endurance/focus over multiple days of shooting... all the stuff the classifier system doesn't look at.  Or it can be pure shooting.  Or both. 

 

The classifier system just looks at shooting and gun-handling skills.  A percentile-based system would allow the classification system to better identify the differences between top shooters in just the shooting stuff.  

 

Which may not matter much to them.  See my earlier analogy to PGA tour pros not caring that much about handicaps... they're all playing heads-up on Thursday-Sunday.  Same with the "real GM's."  

Link to comment
Share on other sites

Percentile based classification would not be good because as your user base grows, most people start out at the bottom and then they work up. So as the sport grows you would be doing more people injustice by giving them higher classification just due to more people joining the sport and the lie getting bigger. The hero/zero will work itself out, I’m sure the super squad scores are looked at compared to the outliers and the bar is set appropriately. The current prez addressed the fact that HHF have not been adjusted in a long time and was going to do something about it and they did. The only thing I look for the classification system to do is give me a base to track my progress. I’m glad they adjusted HHF because I did feel some were too hard and some were too low. After the adjustments, I would have moved up but I’m not worried about it. I know if I shoot for half a season or more it will all work itself out. 

Link to comment
Share on other sites

23 minutes ago, HoMiE said:

Percentile based classification would not be good because as your user base grows, most people start out at the bottom and then they work up. So as the sport grows you would be doing more people injustice by giving them higher classification just due to more people joining the sport and the lie getting bigger. 

Uh, no.  Scores will continue to be added at all levels.  

 

If you want to exclude scores from those people who have posted fewer than 6 or 8 scores, or for the first 6 months after issuance of a first USPSA number, you could do that.  That might be sensible... that way, you'd be measuring against the population of people who are at least somewhat committed, not the mere tourists who shot 3 matches then quit.  

 

HQ, of course, should have the data on what fraction of scores are posted by tourists and noobs.  That would be good data to know/see before deciding what kind of gating is needed on the front end before letting data into the measured-against population.

Link to comment
Share on other sites

3 hours ago, ATLDave said:

But they are screwing with it.  Largely, it appears to me, because "too many" people have been "hooking up" on the peak performance/hero-or-zero runs.  

 

Since they're screwing with it, I would prefer to see them move to a better system that doesn't require judgment-based handicapping of HHF's.  See 

for my suggestion of a percentile-based system.   

 

No debate. Not sure that the hero runs hurt anybody except the person that chooses to over classify.

 

If somebody has A class field course skills but wants to compete against folks with M and GM class skills (for example) at matches I do not see the harm to anyone else. 

 

Now if there was a cure for sandbagging, and I only know 1 person that deliberately abuses the system that way, I would think it would be good for the sport as a whole. 

 

(And I haven't read the other thread yet) 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...