New Classifier Percentages

[MikeBurgess]

8 minutes ago, Balakay said:

Well, I think you might be underestimating this. Significantly.

Let's take a look at one of our National Champions classifiers:

He is currently at 98.xx%.  Included are 3x Hundos (one of which is a match win), and 3 x 97s

 

Looking more closely, what were the other scores that were shot:

73

67

69

83

86

46 (followed by 100 on a reshoot)

73 (followed by 97 on a reshoot)

59 (followed by 100 on a reshoot)

100

41 (followed by 87 on a reshoot)

91

65

52

60 (following a zero)

92

96

46

57 (followed by 92 on reshoot)

36 (followed by 93 on reshoot)

94

58 (followed by 92 on reshoot)

 

So clearly, even the best of the best still have issues. The statement that the top shooters have little difference between their typical performance and the high hit factor is laughable.  The HHF, in my opinion based on a sample size of 1, is unlikely to represent the typical performance of the pinnacle at our sport. I suspect if you went through the classifiers of the super squad, it would looked strikingly similar to the above.

 

I stand correted

 

[Paulie]

1 hour ago, Balakay said:

Well, I think you might be underestimating this. Significantly.

Let's take a look at one of our National Champions classifiers:

He is currently at 98.xx%.  Included are 3x Hundos (one of which is a match win), and 3 x 97s

 

Looking more closely, what were the other scores that were shot:

73

67

69

83

86

46 (followed by 100 on a reshoot)

73 (followed by 97 on a reshoot)

59 (followed by 100 on a reshoot)

100

41 (followed by 87 on a reshoot)

91

65

52

60 (following a zero)

92

96

46

57 (followed by 92 on reshoot)

36 (followed by 93 on reshoot)

94

58 (followed by 92 on reshoot)

 

So clearly, even the best of the best still have issues. The statement that the top shooters have little difference between their typical performance and the high hit factor is laughable.  The HHF, in my opinion based on a sample size of 1, is unlikely to represent the typical performance of the pinnacle at our sport. I suspect if you went through the classifiers of the super squad, it would looked strikingly similar to the above.

 

 

Really interesting. Thx for doing that research.

 

I had no idea reshoots were so prevalent. Also feel a little better about some of my botched classifier runs...

[HoMiE]

Are those 100% scores reshoots or just scores shot at a later date?

[OdinIII]

Balakay’s data looks exactly like I would expect from the GM’s that I’ve seen shoot at local matches. There is no penalty for failures in the hero or zero scenario so they go for it. Unfortunately, it seems our new new standards are based off of “burn it down” runs.

[Balakay]

51 minutes ago, HoMiE said:

Are those 100% scores reshoots or just scores shot at a later date?

same club, same day

[HoMiE]

11 minutes ago, Balakay said:

same club, same day

There should be something in place like drag racing where If you set a top speed, you have to back it up for it to count. I think it could help the Hero or Zero aspect to the HHF. You shot a 100%+ on a classifier, you should have to shoot a consecutive 95%+ on your next one for it to count. 

Edited July 9, 2018 by HoMiE
Speeling

[Balakay]

2 hours ago, OdinIII said:

Balakay’s data looks exactly like I would expect from the GM’s that I’ve seen shoot at local matches. There is no penalty for failures in the hero or zero scenario so they go for it. Unfortunately, it seems our new new standards are based off of “burn it down” runs.

Here's another fun fact:

 

Solid GM, top 10 at a recent Area match...Only 1 out of the last 68 classifiers is greater than 95%.  1/68!!!  I would have better numbers batting against Justin Verlander.

 

This clearly begs the question, who is setting the high hit factors? The lack of transparency intrigues me to say the least.  

[stick]

1 minute ago, Balakay said:

Here's another fun fact:

 

Solid GM, top 10 at a recent Area match...Only 1 out of the last 68 classifiers is greater than 95%.  1/68!!!  I would have better numbers batting against Justin Verlander.

 

This clearly begs the question, who is setting the high hit factors? The lack of transparency intrigues me to say the least.  

What you are looking at is clearly only a small sample of the thousands of classifier scores that get uploaded to USPSA weekly.  It's hard to use one GM as an example.  I do believe you would do better batting against Justin Verlander.

[teros135]

3 hours ago, Paulie said:

 

Really interesting. Thx for doing that research.

 

I had no idea reshoots were so prevalent. Also feel a little better about some of my botched classifier runs...

 

Not exactly "reshoots".  Two of the clubs allow for multiple runs of the complete match, including of course the classifier.  And the two scores published on uspsa.org are always listed with the higher score on top, although he didn't always shoot them in that order.  The 52 and the 65 above it were shot in the same match (a double run), so they should be listed on the same line.

 

I'm guessing that at the top ranks one is motivated to keep one's standing in the "Top 20", so pushing hard at classifiers is pretty much necessary, and you have to push to be at 100%, which means mistakes.  Which is probably why there are so many "low" scores in the mix.  Most of us probably have that kind of a mix as well, if we're pushing our skill level. 

 

I don't think calculating classification percentages with scores more than 5% below the bottom of one's classification is going to change the calculated score much (if at all?) because they only use the highest 6 of the past 8 scores, and it keeps rolling forward in time and the low ones would fall off anyway. 

[Balakay]

1 minute ago, stick said:

What you are looking at is clearly only a small sample of the thousands of classifier scores that get uploaded to USPSA weekly.  It's hard to use one GM as an example.  I do believe you would do better batting against Justin Verlander.

Clearly, I understand the sample size issues. Who cares if some C class shooter finally nails the draw and gets an 83% on El Prez.

 The point I am trying to make is that when you look at the fact that some of the top shooters in our sport are not consistently, or even occasionally achieving 95% classifier scores despite their GM classification. This does not bode well for majority of us. 

[stick]

Just now, Balakay said:

Clearly, I understand the sample size issues. Who cares if some C class shooter finally nails the draw and gets an 83% on El Prez.

 The point I am trying to make is that when you look at the fact that some of the top shooters in our sport are not consistently, or even occasionally achieving 95% classifier scores despite their GM classification. This does not bode well for majority of us. 

I totally agree with you.

[Balakay]

4 minutes ago, teros135 said:

 

Not exactly "reshoots".  Two of the clubs allow for multiple runs of the complete match, including of course the classifier.  And the two scores published on uspsa.org are always listed with the higher score on top, although he didn't always shoot them in that order.  The 52 and the 65 above it were shot in the same match (a double run), so they should be listed on the same line.

 

Fair enough, I was unaware. Still indicates that even a total beast will have many runs well below their class threshold

[teros135]

Just now, Balakay said:

Clearly, I understand the sample size issues. Who cares if some C class shooter finally nails the draw and gets an 83% on El Prez.

 The point I am trying to make is that when you look at the fact that some of the top shooters in our sport are not consistently, or even occasionally achieving 95% classifier scores despite their GM classification. This does not bode well for majority of us. 

 

If this is a typical pattern for top shooters, why is it a problem?  It seems to show us the difficulty of what we're trying to do, if even the top folks can't shoot all 95%+ classifiers all the time. 

[Balakay]

Just now, teros135 said:

 

If this is a typical pattern for top shooters, why is it a problem?  It seems to show us the difficulty of what we're trying to do, if even the top folks can't shoot all 95%+ classifiers all the time. 

Bingo!!!

This is why so many on this topic have questioned why so many classifiers have increased the HHF. Not saying it is unjustified but what is the methodology. Did someone really analyze the raw data?

[stick]

1 minute ago, Balakay said:

Bingo!!!

This is why so many on this topic have questioned why so many classifiers have increased the HHF. Not saying it is unjustified but what is the methodology. Did someone really analyze the raw data?

What if?

What is USPSA actually gave the statistics along with their reasoning for the update.  

By that I mean:

• Sample Size
• Scores taken from what Area's
• Actual stats for classifiers (how many times each classifier was shot)
• Best times/scores for each classifier

 

[Balakay]

2 minutes ago, stick said:

What if?

What is USPSA actually gave the statistics along with their reasoning for the update.  

By that I mean:

• Sample Size
• Scores taken from what Area's
• Actual stats for classifiers (how many times each classifier was shot)
• Best times/scores for each classifier

 

This is what should have already occurred

[stick]

2 minutes ago, Balakay said:

This is what should have already occurred

But did it?

This is what we are all wondering

[broadus123]

Keep in mind , if  they did this once, they can do it whenever they feel like it.

[ATLDave]

30 minutes ago, stick said:

But did it?

This is what we are all wondering

I would very much like to see the score data for any given classifier plotted on a curve.  And then the new and old HHF's marked on the curve.  

[teros135]

Just now, ATLDave said:

I would very much like to see the score data for any given classifier plotted on a curve.  And then the new and old HHF's marked on the curve.  

 

Good idea, Transparency (what a concept!)

[Flyingpig]

Actually heres a look at the best of the best and it tells a slighly different story. Way more consistent. This is a stretch in 2014. One  might be a major win but most are not. I counted 26 100% including major wins.

 

production shooter Local GM(and one of the best):

 

97.3850

-

11/11/14

Legacy

 

100.0000

-

10/08/14

Legacy

 

96.1570

-

10/06/14

Legacy

 

100.0000

-

10/13/14

Legacy

 

75.8050

-

9/10/14

Legacy

 

98.9668

-

9/10/14

Legacy

 

91.2030

-

8/11/14

Legacy

 

97.4359

-

8/12/14

Legacy

 

100.0000

-

7/10/14

Legacy

 

100.0000

-

7/09/14

Legacy

 

100.0000

-

6/11/14

Legacy

 

100.0000

-

6/05/14

Legacy

 

 

 

 

 

 

 

Edited July 9, 2018 by Flyingpig

[MikeBurgess]

Another top shooters current list. dropped decimals off because I'm lazy. only 1 is a Major match placement

 

100

83

81

100

94

87

100

91

96

100

100

 

Oddly of the 6 that count 2 would move down 1 of the 100s goes to 94.8 and the 96 goes to 94.6

 

 

 

[malobukov]

1 hour ago, ATLDave said:

I would very much like to see the score data for any given classifier plotted on a curve.  And then the new and old HHF's marked on the curve.  

 

Here are Q-Q plots of the hit factors for the most popular classifiers in production. I only looked at a relatively small recent sample, but in absence of official data it's still better than guessing.

 

Old HHF is blue horizontal dashed line, new HHF is green horizontal dashed line. You can see that using normal approximation, HHF is about three standard deviations above mean. You can also see how bent the left tail is because of zero runs.

[Balakay]

1 hour ago, MikeBurgess said:

Another top shooters current list. dropped decimals off because I'm lazy. only 1 is a Major match placement

 

100

83

81

100

94

87

100

91

96

100

100

 

Oddly of the 6 that count 2 would move down 1 of the 100s goes to 94.8 and the 96 goes to 94.6

 

 

 

Cue the dueling banjo music, another top 5 National shooter from a different division:

 

82

63

96

77

100

75

81

94

95

82

99

 

Let's not lose sight of the fact of why I posted this data originally. The HHF clearly is not what the best shooters in our sport can expect to do on a routine basis.

I think that it makes you question the true meaning of what a 100% classifier means. Sometimes they get 100%, more often they do not.

Is the HHF the best HF ever recorded?

Is it the average of X number of the best runs?

Is it based on retrospective analysis of all reported classifiers and adjusted accordingly?

Is it an arbitrary threshold that will continue to evolve over time?

 

Going a step further, as someone alluded to earlier in the thread, what exactly does classification even signify? M = 85% of what exactly

[IHAVEGAS]

9 hours ago, Balakay said:

 

Going a step further, as someone alluded to earlier in the thread, what exactly does classification even signify?

 

I'd peel off one more layer. What is the purpose of a classification?

 

In the rest of the world it is to make competition the most fun for the greatest majority of participants by giving people a chance for friendly competition against other people of similar abilities.