Jump to content
Brian Enos's Forums... Maku mozo!

A Better Classification System


Recommended Posts

Hey everyone, it's been a while, since I posted here last time.

 

Thought I'd let everyone know, that HitFactor.Info has achieved Full Parity with original USPSA Classification System:

  • Uploads
  • WhatIf / Calculator
  • Reclassification 
  • Search
  • Etc

 

But it's not just a clone of classification system. It's much better.

First we've fixed the High Hit Factors for all Classifiers and Divisions, using Statistical Analysis.

Coupled with Reverse Engineered Classification Algorithm it actually bumped most shooters up, so we dropped the B and C flags that are nothing more than Tanking Protection really.

Then we killed Grandbagging (reshooting classifier twice same day for a fee) by averaging out the same day dupes, resulting in a Much More Realistic Overall Classification. 

 

There's also a better UI, Unlimited Search by name, Graphs, Stats, Reports, True Multi-Score WhatIf, etc, etc.

It also moved to a better name: https://www.HitFactor.Info

Oh and it's Free.

 

I will do some videos explaining different features of the app soon, but right now you can check it yourself, Home Page has some quick explanations. 

 

Here's a Quick Pic of Our Devs Celebrating this Release:

 

image.jpeg.3efe2689668eabda9a4be875079d0c4c.jpeg

Edited by CutePibble
Link to comment
Share on other sites

Show me which classifier stages where NOBODY has actually been able to shoot the 100%…. high hit factor.

 

Conversely, I want to be able to click on a classifier stage, then click on another “button” which takes me to a video of some dude actually shooting the highest hit factor… not necessarily the 100% HHF …. For that stage.

Link to comment
Share on other sites

38 minutes ago, MHicks said:

Is this going to be used by USPSA? 

 

With current leadership very unlikely, as that would be them admitting their incompetence. 

 

USPSA still has a message for web developers on their website that reads "Looking for data? Contact it@uspsa.org for details about our API!", not only they didn't respond to ANYONE who asked for it, they have deliberately blocked my access to the USPSA data, and half-broke their own website and mobile app in the process (classification lookup is extremely rate limited now), so I have to use PS data instead.

39 minutes ago, THS said:

that link won't open for me......

try typing it out fully, with HTTPS and www? it should open

 

Link to comment
Share on other sites

34 minutes ago, Chills1994 said:

NOBODY has actually been able to shoot the 100%

 

Go to Classifiers -> Division and sort by HQ Minus Rec.HHF Descending, you'll find a few, here's a ridiculous example, where record holder GM didn't even get an A-class run:
 

https://www.hitfactor.info/classifiers/lo/18-01

 

Video attachments is one of the ideas I'm entertaining, but it would require some other functionality to be added first (registration, auth, user profiles, etc)

Link to comment
Share on other sites

37 minutes ago, CutePibble said:

 

Go to Classifiers -> Division and sort by HQ Minus Rec.HHF Descending, you'll find a few, here's a ridiculous example, where record holder GM didn't even get an A-class run:
 

https://www.hitfactor.info/classifiers/lo/18-01

 

Video attachments is one of the ideas I'm entertaining, but it would require some other functionality to be added first (registration, auth, user profiles, etc)

 I believe they mis entered 18-02 for 18-01 and despite repeated requests never fixed it. 

Link to comment
Share on other sites

1 hour ago, CutePibble said:

 

Go to Classifiers -> Division and sort by HQ Minus Rec.HHF Descending, you'll find a few, here's a ridiculous example, where record holder GM didn't even get an A-class run:
 

https://www.hitfactor.info/classifiers/lo/18-01

 

Video attachments is one of the ideas I'm entertaining, but it would require some other functionality to be added first (registration, auth, user profiles, etc)

 Thank you sir!

 

I was able  to click on the El Prez and see the best hit factors shot (and the shooters’s names) rank ordered to the lowest hit factors.

 

That’s pretty slick.

Link to comment
Share on other sites

How about running the ELO algo over the all the practiscore data at match level.

(ie) Match results only.

 

I like the division stats.

 

Aggregate match data with ELO over say the last running 18 months would be primo

 

Like http://www.ipscrating.com

Link to comment
Share on other sites

Yes, would be nice to remove all classifier data, re-adjust match performance without, then use match data itself to calculate classification. With many using hero/zero on classifiers (which then adversely affects match standing), just curious. 

Link to comment
Share on other sites

On 6/18/2024 at 5:43 AM, vluc said:

Yes, would be nice to remove all classifier data, re-adjust match performance without, then use match data itself to calculate classification. With many using hero/zero on classifiers (which then adversely affects match standing), just curious. 

 

The problem is that clubs don't run the same quality of stages across the board, so you may be a GM as your local club with basic stages but if you come to my home club there is a chance you might make A. The local heat and the quality of stages and the relative challenge make all the difference.  

Link to comment
Share on other sites

well when I was shooting USPSA the classifiers had 2 things to do with matches..  One of them was Jack.
Classifiers were just about all stand and shoot, week hand strong hand, a mandatory standing reload...  shooting intensive challenges. 
Matches were all about who could run the fastest with easy shooting..  I have heard things have changed.. Id try it out again if there was a local club... I also wouldnt try that hard or worry about classifiers like I used to ..
Personally I think the entire classification system should be scrapped, and only use match performance at level II's and up.
At locals class doesnt matter, as mentioned above classifiers arnt uniform, and at least formerly didnt represent match skills.

Link to comment
Share on other sites

There has never been any method to ensure that classifiers are set up correctly. A small angle of difference can have an enormous impact on target presentation.

 

People want to know where they stand but if they are just shooting for fun and only competing at L1 matches then what’s the point?

 

In the UK we had our local matches and the next level up were Graded matches. Usually a two-day competition but each person completed it in one day. Those MATCH scores were used to determine classification.


Multi-state, Nationals and International matches could easily be used as a classification system.

Link to comment
Share on other sites

1 hour ago, BritinUSA said:

There has never been any method to ensure that classifiers are set up correctly. A small angle of difference can have an enormous impact on target presentation.

 

People want to know where they stand but if they are just shooting for fun and only competing at L1 matches then what’s the point?

 

In the UK we had our local matches and the next level up were Graded matches. Usually a two-day competition but each person completed it in one day. Those MATCH scores were used to determine classification.


Multi-state, Nationals and International matches could easily be used as a classification system.

well other than a probably huge percent of people that use the class system as a match within a match.. Basically them against the class system to get that next letter as feedback on their skills improving..
I mean yeh as we play the game longer we realize the class doesnt matter all that much.. 
Match performance would be a much more accurate system, but from a finance point ? Youd probably lose half your paid members. SO I retract the scrapping it part.. 

Link to comment
Share on other sites

Way back in the day when classification "books" were mailed out quarterly with every shooter's new classification in them, 'club classes' based on how you did at locals were a thing, because updates could take a while.  Club classes rarely reflected reality outside the club, but at least you knew about where you stood.

 

Funny enough, even though most classifiers were stand-and-shoot, reload, shoot, the same dudes that were M's and GMs shooting those were the ones at the top of the run-and-gun match standings too.

 

 

 

Link to comment
Share on other sites

  • 3 weeks later...
Posted (edited)

@CutePibble

 

Also, have you found that the data (once you have enough) is normally distributed? I'd assume once a classifier has been out a long time, it starts to skews left. How do you handle that when grouping competitors into their categories?

 

And what about just recognizing a basic level of skill to reach a certain point regardless of how many people are at that skill level? If USPSA is flooded with new members, or alternatively, if new memberships stagnate and existing members grow older/more practiced, that will change the composition of the data. Shouldn't the recognized skill level be somewhat independent of who is shooting them?

 

Or are there enough entries that it all averages out anyway?

 

 

 

Edited by LuckyDucky
Link to comment
Share on other sites

@CutePibble

 

A couple questions / observations:

 

When casually looking through, it looks like anyone with their USPSA profile set to private has their data excluded from the list. Is that correct?

 

Also, when goofing around with Practiscore I noticed that even my practice sessions get automatically listed as matches. Could those numbers inadvertently get pulled into such a database and pollute the match data?

Link to comment
Share on other sites

I like this version better lol.

 

I go from 95 to 99 in SS, I drop to M in Limited but that makes sense since I never had 95 in limited to begin with, it was a match bump to G. LO I'd jump up to 94.7 lol. I pretty much go up in everything but open because all that stuff is super old. 

Link to comment
Share on other sites

I don't understand why the classes of people needs to be distributed in a particular way (e.g. normal distribution) across all participants. Really, the only thing that matters is the HHF, which should be a measure of peak human performance (whether actual or theoretical).

 

The USPSA classifier system is not an ELO relative skill rating system or something used for matching participants for competitions (like in Valorant, CSGO, or Halo).

 

In all the howler monkey discourse, I may have missed a discussion on the purpose of the USPSA classification system and what it was intended to measure.

 

Edit1: The USPSA class system is a measure of absolute skill. And although it is an average, due to throwing out low scores, it is really only a top-line measure, not a general measure of how you will perform on any given day. 

 

Elo on the other hand is a relative skill measurement valid only in the rating pool in which it was calculated.  It makes me think of how SAT and ACT scores or even IQ test scores are normalized for given populations. 

 

Certainly, an interesting topic, maybe I'll read some more about it.

 

Edit2: Maybe USPSA should start maintaining an ELO system in addition to the class system. That way people that go to local matches can still be classed, but people that want to truly rise in the ranks can go to Level 2+ matches and face the ELO system.

 

Edited by LuckyDucky
Link to comment
Share on other sites

It appears someone has developed an elo system for USPSA already (see link below). I haven't delved too deeply into how his rating works, but it seems to show that the USPSA Classification System actually does classify skill level accurately. However, there is significant overlap between each class, that is, precision is not the best.

 

I think an elo system in addition to the class system would be good for USPSA for multiple reasons. First, many of the complaints with the classification system are that it doesn't replicate the real world. This complaint often comes from higher level shooters after they plateaued in class but improved in match performance. Second, relative skill ratings (elo) could be used to rate local competitors based off of other local competitors that shoot majors, and inform the non-major shooters where they stand. Third, elo rating could be used to track leader rankings nationally and generate more interest in the sport. Right now, I feel most of the top shooters are recognized as just, "oh another top shooter" but you don't know who is the best of the best, who is in the running to win a major match, what an upset would be, etc. 

However, the classification system is still good in that it gives new shooters something to aim for and is kind of like achieving a particular belt in martial arts. Achieving a class demonstrates a certain level of skill, but that achievement is not necessarily a precise relative rating to the current pool of competitors.

 

Shooting Sports Analyst | Elo ratings and more for the shooting sports | Patreon

image.png.c24df27da6869109466f5350de2f8bf4.png

Edited by LuckyDucky
Link to comment
Share on other sites

58 minutes ago, LuckyDucky said:

It appears someone has developed an elo system for USPSA already (see link below). I haven't delved too deeply into how his rating works, but it seems to show that the USPSA Classification System actually does classify skill level accurately. However, there is significant overlap between each class, that is, precision is not the best.

 

I think an elo system in addition to the class system would be good for USPSA for multiple reasons. First, many of the complaints with the classification system are that it doesn't replicate the real world. This complaint often comes from higher level shooters after they plateaued in class but improved in match performance. Second, relative skill ratings (elo) could be used to rate local competitors based off of other local competitors that shoot majors, and inform the non-major shooters where they stand. Third, elo rating could be used to track leader rankings nationally and generate more interest in the sport. Right now, I feel most of the top shooters are just, "oh another top shooter" but you don't know who is the best, who is in the running to win a major match, what an upset would be, etc. 

However, the classification system is still good in that it gives new shooters something to aim for and is kind of like achieving a particular belt in martial arts. Achieving a class demonstrates a certain level of skill, but that achievement is not necessarily a precise relative rating to the current pool of competitors.

 

Shooting Sports Analyst | Elo ratings and more for the shooting sports | Patreon

image.png.c24df27da6869109466f5350de2f8bf4.png

A ELO system has several short falls. I think what we have basically works fine. Maybe we tweak a little how we calculate the HHF.

 

I do kind of like this idea of killing the B and C flags like this and lowering the HHF. This would force shooters to be more consistent from classifier to classifier and match to match and probably give a better representation of their on demand skill which will effect how we place at majors. 

 

But, again the system as is does what we need it too. 

Link to comment
Share on other sites

9 minutes ago, Racinready300ex said:

A ELO system has several short falls. I think what we have basically works fine. Maybe we tweak a little how we calculate the HHF.

 

I do kind of like this idea of killing the B and C flags like this and lowering the HHF. This would force shooters to be more consistent from classifier to classifier and match to match and probably give a better representation of their on demand skill which will effect how we place at majors. 

 

But, again the system as is does what we need it too. 

 

I think HHF should be calculated based on peak human performance. Whether that is actually measured such as at nationals (or by paying Eric Grauffel to shoot a classifier 10 times and averaging his results 😆) or by some sort of theoretical measure of human performance (e.g. this classifier requires a transitions, b splits, c yards movement, which together should take X total time).

 

There could be a DGM (Distinguished GM) class for people that have GM class and have placed at nationals within a certain % of the winner or that have achieved a certain ELO rating.

 

ELO can do a pretty good job of telling where people will place at a match. Of course, it's not perfect either. "Any given Sunday."

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...