[Limited optics half a year later]

LO is extremely popular, it's already 2nd most popular division after CO, it's definitely here to stay

Since this popularity is based on number of classifier scores submitted, I expect it to go down. We'll see.

 

One interesting fact though, with LO coming in hot and moving other divisions aside, it only took a small percentage our of CO's popularity, which can mean that it's mostly OTHER divisions shooters who are trying LO, and not CO shooters switching.

 

 

Personally I don't see our local match heat (who are mostly CO) shooters switch to LO, unless only for classification with existing CO gear. 

[Any hope for Ironsight divisions?]

It turned out to be a very unpopular opinion on reddit, but all irons divisions seem to be dying.

You can see it on hit factor.info Stats -> Divisions page, which uses number of classifier scores submitted to the HQ in order to determine division's popularity

 

[Limited is The Most Broken Division]

On 5/5/2024 at 10:11 PM, LuckyDucky said:

How are you deciding what the recommended HHF should be?

 

 

By looking at the distribution of scores and trying to find a best fitting percentile, based on one of the target ones:

 

Top 1% score == 95% classification GM
Top 5% score == 85% classification M
Top 15% score == 75% classification A

 

These are more or less "perfect" percentiles for GM/M/A in CO, which is most popular and calibrated division. Other brackets should be around 40th percentile for B, and 80th for C, but these aren't used for targeting.

 

Actual algorithm is:

1. Take all scores for division for classifier, ignore zeros

2. Sort, that gives you a place, divide by number of total non-zero scores - that gives you percentile

3. Default to 1th/95% targeting unless there's an override (without enough scores you might want to dip to 5th/85% to be closer to the graph, some classifiers aren't shot by good shooters at all)

4. Sort scores by being closest to target, e.g. 1th (0.97th is closer than 1.05th for example)

5. Extrapolate hit-factor to what it would've been if score is actually perfect target percentile 

6. Divide that hit-factor by target percent, so if our 0.97th was 7.03HF and extrapolated to 1.00th it became 7.01, we divide it by 0.95 and get 7.3789
7. That's your new HHF

With every score uploaded HHF can change, although for established classifiers (with more than 1k scores) it doesn't really change much.

All that is open source, you can see the actual logic here: https://github.com/CodeHowlerMonkey/hitfactor.info/blob/5d7ba87ba31986fea868d5465eede953f1a67a66/api/src/db/recHHF.js#L19-L36

[Limited is The Most Broken Division]

On 3/27/2024 at 4:52 PM, shred said:

50 % M in Revo because of one classifier match long ago.

On classifications stats page you just need to switch from By Class to By Percent to make yourself count as C class in Revo. 

 

CurHHF and RecHHF/Recommended are always reclassifications and by percent, so they will never even look at your official letter

 

 

[Limited is The Most Broken Division]

Hey it's me again, with analytics. Some updates landed, one of them being Shooters Distribution Graph. Here's what it looks like using new Recommended Classification for Carry-Optics:

 

 

This is just to serve as an illustration what a good distribution looks like. CO itself is pretty OK calibrated, even with HQ algorithms and current HHFs. The distribution above uses recommended High Hit Factors and "Best 6 out of Last 10" algorithm (with only best duplicate counting, basically USPSA minus B/C flags, that are nothing but hero-or-zero protection anyway). The colors are Orange for D-class, Green-C, Blue-B, Purple-A, Brown-M, Black-GM. Yes, it's BJJ color scheme adapted for D/C

 

Now, here's what Limited looks like, using HQ colors and HQ position:

 

 

"So it's not a smooth, what's a big deal?" you might ask.

 

Well, things change, if you keep HQ position (X = official classification and Y= percentile of shooter using that official classification), but use "what if all these people were classified against current High Hit Factors with original USPSA algorithm":

 

 

Yes, that orange means that there are a lot of C- and even B- and A-shooters who whould be classed as D-class!!! Green (would be C-class with curHHF) goes into Masters and Grandmasters!

 

See for yourself here:

https://violent-beverley-howlermonkeys.koyeb.app

 

Go to Classifications -> Distribution -> Limited and play with different Colors and Positions to see what the screenshots show.

Go to Shooters -> Limited and try sorting by CurHHF % and take a look at triple Class Tag next to shooter's name (Recommended, CurHHF, Official) to see the same thing in person by person basis. You can also hover your mouse over the graph and it will show a USPSA number and their classifications for each dot.

 

P.S.

Oh yeah I also added inconsistencies tab in the Classifications. Allows you to hunt Paper-GMs and Sandbaggers. Only for Ltd and CO, need to choose Recommended HHF calibration for other divisions...

 

 

[USPSA Updated Production Classifiers]

7 hours ago, Racinready300ex said:

top B class guys move to A where they can't win

I don't care about non-overall wins. 

 

1 hour ago, Fishbreath said:

Classification is the analog to a martial arts belt system

yep, which is made for what? Motivation

 

3 hours ago, Racinready300ex said:

a system that basically encourages shooters to swing for the fence is probably flawed

at any given local match with a single classifier, unrealistically high HHF is what makes people to swing for the fences. They often look up the HHF using calculator and try to get to that next class.

 

1 hour ago, akarhi said:

isnt a fair way for a classification. Since you can literally practice for it.

if you practice you get better lol, and not just at classifiers 

 

3 hours ago, Racinready300ex said:

that goal is move people up

no, the goal is to classify people where they belong. Your unlucky B class who should be M? You go up. You're a paper GM in CO, who got in time when CO used prod HHFs and now perform at A-class level? You go back to A.

 

Now we can't take a title from people, but what I can is introduced a recommended classification on my website. And that's what I'm working on. There will be GMs that have recommended classification of A/M. And there will be B/A shooters, who will be recommended as M/GM. 

 

3 hours ago, Racinready300ex said:

Some people would go down, others would probably go up, chips will fall. 

 

1 hour ago, BritinUSA said:

drop the bit about classification only going up

recommended classification on my website will allow going down. At first algorithm will be USPSA-like, with B and C flags which severely limit classification dropping, but I'll see what can be done about that and later switch recommended classification to something along 4 best out of 10 most recent, no exclusion flags.

 

[USPSA Updated Production Classifiers]

10 hours ago, rowdyb said:

if the number of gm's went from 1% of the membership to 3%

It actually would go from 720 to 336 total if we simply reclassify everyone using new HQ HHFs and same classification logic, but from scratch. That’s what new By Cur. HHF Percent mode does in Classification Stats tab

 

4 hours ago, Racinready300ex said:

So more fair in this case means move people up?

Not necessarily. Some will go up, some will go down. It’ll just measure things right. Fair. 

 

3 hours ago, shred said:

Let's just fix the screwed up HHFs first and then think about changing the whole system.  Crawl-Walk-Run

Sir, yes, sir!

[USPSA Updated Production Classifiers]

7 minutes ago, Racinready300ex said:

So what is the end game? We probably need to define that, then figure a system that can pull it off. 

There’s a GitHub repo with all that and feature tracking. There are many goals. HHFs-wise it’s about transparency and fair classification. Fixing just HHFs won’t take anyone’s title away. But it will help unlucky B and A shooters who should be classified higher. 
 

just more fair system, that’s it

[USPSA Updated Production Classifiers]

9 minutes ago, Fishbreath said:

are you pulling percentages out of the USPSA website

No practiscore. 100% official HQ data. Unless it’s a private USPSA account (like most cheaters do) - it’s there. 
 

[USPSA Updated Production Classifiers]

2 minutes ago, motosapiens said:

have you found that some classifiers have a tighter distribution?

Shapes look slightly different in hoser vs standards, the lower end “tail” gets flattened in high risk classifiers with people zeroing them more often. And fixed time ones look funny because they have whole numbers HFs. 
 

but overall distribution itself looks pretty normal with enough data. 
 

The scale just shifts around it depending on HHFs. I put dots on intersections on 1/5/15 percentile and corresponding % and you can tell if it’s easy or hard depending where relative to the dot scores fall 

 

 

 

[USPSA Updated Production Classifiers]

52 minutes ago, motosapiens said:

the hhf’s are only a small part of the problem.

Yes, bad HHFs is only one of the problems, but it has to be solved before everything else. For example, fluctuating classification will be completely broken if we don't fix HHFs/percentile first.

 

56 minutes ago, motosapiens said:

include all scores, and compute based on the best 6 of the most recent 10

planned. First I need to implement classification calculation to mimic USPSA.

Then we can start playing with algorithms / strategies.

 

58 minutes ago, motosapiens said:

use a percentile approach, which would also be self adjusting over time. 

this is what recommended HHFs are using right now. Although, honestly, it looks like without some major breakthroughs they won't be changing much by self-adjusting. The graphs already look like classic normal distributions. Just click around the app and look at the data. 

Speaking of which, I've just deployed an update with all public USPSA numbers, production HHF updates and 23-series classifiers: https://violent-beverley-howlermonkeys.koyeb.app Check it out.

[USPSA Updated Production Classifiers]

3 hours ago, motosapiens said:

because the percentages change when enough people hero/zero and hq decides to screw the hhf up.

 

So far I'm not seeing a correlation between top scores and HHF.
Some classifiers literally don't have historical GMs or Hundos, much less current adjusted GMs or Hundos.

Personal hero or zero by local M/GMs that I witnessed did result in pretty high placement and even some current records, but weren't higher than 105%.

There are some statistical anomalies, with people putting 110 and even 120%, but they're insignificant and don't affect the any of the recommended HHF.
 

I believe that a lot of issues that classification system has right can and will be solved.

[USPSA Updated Production Classifiers]

Working on re-importing things better for my analytics website, noticed that a lot of HHFs were changed, all for Production division.

Here's a quick rundown:

 

 

 

Nice round numbers.

[How Well Calibrated Are USPSA Classifiers?]

Added first iteration of recommended HHFs, as well as some additional insights like legacy scores and classification ages.

Recommended HHFs have yet to be used to calculate anything. Still, they show three different methods of HHF based on existing data from which some imaginary Classifier Committee can choose to fix HHFs.

 

Highlight of the release: 06-10 Steely Speed VII hates Limited shooters. 
This classifier has no reload component, no paper target, only steel. So there's absolutely no advantage in Minor vs Major. Yet it has a noticeably higher Official HQ HHF than the CO division. 

 

Full changes: https://github.com/CodeHowlerMonkey/hitfactorlol/pull/20

[How Well Calibrated Are USPSA Classifiers?]

3 hours ago, Stepan said:

it’s really frustrating to see talented and hard working shooters stay in A/M class simply because they weren’t lucky. 

That’s one of the reasons this project was started. We don’t want to create more or less GMs with it. Just simply calibrate classifiers and membership stats against each other, making classification process smoother and closer to “real percentage”. 
 

One of the ideas that can achieve that (that was originally posted by competition.shooting.analytics on IG IIRC) is to use percentile of real A-class shooters to make a calibration point AND count the lower bound of that percentile in each classifier as 75% for the division. 
 

this approach of course requires more testing, but so far it looks really promising. It’s already moving hard classifiers into easier HHF and easier ones into higher HHFs. 
 

quick demo on hard and easy classifier IN CO (red vertical lines — current HHF, yellow - proposed A-class calibration, green/blue - similar calibrations against M and G class):

 

 

 

if something like this is implemented — it should NOT change things much, 

more just smooth them over and make so called sandbaggers classify where they should be. 
 

It could also be used for fully dynamic classification range, that can go both up and down, but that’s separate topic, how to calculate classification based on classifier results and their average curPercent/highPercent. 

 

[How Well Calibrated Are USPSA Classifiers?]

14 minutes ago, waktasz said:

Just from a quick glance that data is obviously wrong because I have hundos on at least one classifiers they have listed as having 0% GM scores. 

Let’s review it. What is your USPSA number?

 

There are few issues that might be at play here. Are these hundos listed as Legacy scores on your USPSA profile? These don’t have HF in the records so I removed them from output for now. 
 

also you might have something with 100 percent historical, but curPercent much lower due to HHF updates, even if it’s not a legacy score
 

 

[classifier records?]

3 hours ago, shred said:

If there's enough data it would be interesting to see the new-HHF pie charts vs old.  I suspect a lot of the PCC and Open GMs are left over from when it was easier to get them.

 

There is already a way to see it. Within the classifier tables there are percent, curPercent, difference between the two and historical hhf columns. You can sort by them too. 
 

for pie chart I would need to implement actual classification calculation. Which will probably happen after uploads. But if you can create an issue on GitHub (link in top right corner) - it will help

[classifier records?]

There is now: https://violent-beverley-howlermonkeys.koyeb.app/

[18-01 percentage oddity]

Yep. Looks broken. 
 

Nobody could even score as A-class on it. 
 

[The Future Of The Limited Optics Division in USPSA]

Just merge it and allow 2011s in CO. 

 

Right now locally nobody good switched to LO from CO. And CO masters and A-class sandbaggers are still kicking everyone’s ass. 
 

also if you look at classifier data from my other post — you’ll see that there’s virtually no difference. 

[How Well Calibrated Are USPSA Classifiers?]

Well looks like now you can find out yourself:

 

https://violent-beverley-howlermonkeys.koyeb.app/classifiers/opn/
 

some look pretty bad. Not a single GM run on file.