MikeBurgess Posted July 9, 2018 Share Posted July 9, 2018 8 minutes ago, Balakay said: Well, I think you might be underestimating this. Significantly. Let's take a look at one of our National Champions classifiers: He is currently at 98.xx%. Included are 3x Hundos (one of which is a match win), and 3 x 97s Looking more closely, what were the other scores that were shot: 73 67 69 83 86 46 (followed by 100 on a reshoot) 73 (followed by 97 on a reshoot) 59 (followed by 100 on a reshoot) 100 41 (followed by 87 on a reshoot) 91 65 52 60 (following a zero) 92 96 46 57 (followed by 92 on reshoot) 36 (followed by 93 on reshoot) 94 58 (followed by 92 on reshoot) So clearly, even the best of the best still have issues. The statement that the top shooters have little difference between their typical performance and the high hit factor is laughable. The HHF, in my opinion based on a sample size of 1, is unlikely to represent the typical performance of the pinnacle at our sport. I suspect if you went through the classifiers of the super squad, it would looked strikingly similar to the above. I stand correted Link to comment Share on other sites More sharing options...
Paulie Posted July 9, 2018 Share Posted July 9, 2018 1 hour ago, Balakay said: Well, I think you might be underestimating this. Significantly. Let's take a look at one of our National Champions classifiers: He is currently at 98.xx%. Included are 3x Hundos (one of which is a match win), and 3 x 97s Looking more closely, what were the other scores that were shot: 73 67 69 83 86 46 (followed by 100 on a reshoot) 73 (followed by 97 on a reshoot) 59 (followed by 100 on a reshoot) 100 41 (followed by 87 on a reshoot) 91 65 52 60 (following a zero) 92 96 46 57 (followed by 92 on reshoot) 36 (followed by 93 on reshoot) 94 58 (followed by 92 on reshoot) So clearly, even the best of the best still have issues. The statement that the top shooters have little difference between their typical performance and the high hit factor is laughable. The HHF, in my opinion based on a sample size of 1, is unlikely to represent the typical performance of the pinnacle at our sport. I suspect if you went through the classifiers of the super squad, it would looked strikingly similar to the above. Really interesting. Thx for doing that research. I had no idea reshoots were so prevalent. Also feel a little better about some of my botched classifier runs... Link to comment Share on other sites More sharing options...
HoMiE Posted July 9, 2018 Share Posted July 9, 2018 Are those 100% scores reshoots or just scores shot at a later date? Link to comment Share on other sites More sharing options...
OdinIII Posted July 9, 2018 Share Posted July 9, 2018 Balakay’s data looks exactly like I would expect from the GM’s that I’ve seen shoot at local matches. There is no penalty for failures in the hero or zero scenario so they go for it. Unfortunately, it seems our new new standards are based off of “burn it down” runs. Link to comment Share on other sites More sharing options...
Balakay Posted July 9, 2018 Share Posted July 9, 2018 51 minutes ago, HoMiE said: Are those 100% scores reshoots or just scores shot at a later date? same club, same day Link to comment Share on other sites More sharing options...
HoMiE Posted July 9, 2018 Share Posted July 9, 2018 (edited) 11 minutes ago, Balakay said: same club, same day There should be something in place like drag racing where If you set a top speed, you have to back it up for it to count. I think it could help the Hero or Zero aspect to the HHF. You shot a 100%+ on a classifier, you should have to shoot a consecutive 95%+ on your next one for it to count. Edited July 9, 2018 by HoMiE Speeling Link to comment Share on other sites More sharing options...
Balakay Posted July 9, 2018 Share Posted July 9, 2018 2 hours ago, OdinIII said: Balakay’s data looks exactly like I would expect from the GM’s that I’ve seen shoot at local matches. There is no penalty for failures in the hero or zero scenario so they go for it. Unfortunately, it seems our new new standards are based off of “burn it down” runs. Here's another fun fact: Solid GM, top 10 at a recent Area match...Only 1 out of the last 68 classifiers is greater than 95%. 1/68!!! I would have better numbers batting against Justin Verlander. This clearly begs the question, who is setting the high hit factors? The lack of transparency intrigues me to say the least. Link to comment Share on other sites More sharing options...
stick Posted July 9, 2018 Share Posted July 9, 2018 1 minute ago, Balakay said: Here's another fun fact: Solid GM, top 10 at a recent Area match...Only 1 out of the last 68 classifiers is greater than 95%. 1/68!!! I would have better numbers batting against Justin Verlander. This clearly begs the question, who is setting the high hit factors? The lack of transparency intrigues me to say the least. What you are looking at is clearly only a small sample of the thousands of classifier scores that get uploaded to USPSA weekly. It's hard to use one GM as an example. I do believe you would do better batting against Justin Verlander. Link to comment Share on other sites More sharing options...
teros135 Posted July 9, 2018 Share Posted July 9, 2018 3 hours ago, Paulie said: Really interesting. Thx for doing that research. I had no idea reshoots were so prevalent. Also feel a little better about some of my botched classifier runs... Not exactly "reshoots". Two of the clubs allow for multiple runs of the complete match, including of course the classifier. And the two scores published on uspsa.org are always listed with the higher score on top, although he didn't always shoot them in that order. The 52 and the 65 above it were shot in the same match (a double run), so they should be listed on the same line. I'm guessing that at the top ranks one is motivated to keep one's standing in the "Top 20", so pushing hard at classifiers is pretty much necessary, and you have to push to be at 100%, which means mistakes. Which is probably why there are so many "low" scores in the mix. Most of us probably have that kind of a mix as well, if we're pushing our skill level. I don't think calculating classification percentages with scores more than 5% below the bottom of one's classification is going to change the calculated score much (if at all?) because they only use the highest 6 of the past 8 scores, and it keeps rolling forward in time and the low ones would fall off anyway. Link to comment Share on other sites More sharing options...
Balakay Posted July 9, 2018 Share Posted July 9, 2018 1 minute ago, stick said: What you are looking at is clearly only a small sample of the thousands of classifier scores that get uploaded to USPSA weekly. It's hard to use one GM as an example. I do believe you would do better batting against Justin Verlander. Clearly, I understand the sample size issues. Who cares if some C class shooter finally nails the draw and gets an 83% on El Prez. The point I am trying to make is that when you look at the fact that some of the top shooters in our sport are not consistently, or even occasionally achieving 95% classifier scores despite their GM classification. This does not bode well for majority of us. Link to comment Share on other sites More sharing options...
stick Posted July 9, 2018 Share Posted July 9, 2018 Just now, Balakay said: Clearly, I understand the sample size issues. Who cares if some C class shooter finally nails the draw and gets an 83% on El Prez. The point I am trying to make is that when you look at the fact that some of the top shooters in our sport are not consistently, or even occasionally achieving 95% classifier scores despite their GM classification. This does not bode well for majority of us. I totally agree with you. Link to comment Share on other sites More sharing options...
Balakay Posted July 9, 2018 Share Posted July 9, 2018 4 minutes ago, teros135 said: Not exactly "reshoots". Two of the clubs allow for multiple runs of the complete match, including of course the classifier. And the two scores published on uspsa.org are always listed with the higher score on top, although he didn't always shoot them in that order. The 52 and the 65 above it were shot in the same match (a double run), so they should be listed on the same line. Fair enough, I was unaware. Still indicates that even a total beast will have many runs well below their class threshold Link to comment Share on other sites More sharing options...
teros135 Posted July 9, 2018 Share Posted July 9, 2018 Just now, Balakay said: Clearly, I understand the sample size issues. Who cares if some C class shooter finally nails the draw and gets an 83% on El Prez. The point I am trying to make is that when you look at the fact that some of the top shooters in our sport are not consistently, or even occasionally achieving 95% classifier scores despite their GM classification. This does not bode well for majority of us. If this is a typical pattern for top shooters, why is it a problem? It seems to show us the difficulty of what we're trying to do, if even the top folks can't shoot all 95%+ classifiers all the time. Link to comment Share on other sites More sharing options...
Balakay Posted July 9, 2018 Share Posted July 9, 2018 Just now, teros135 said: If this is a typical pattern for top shooters, why is it a problem? It seems to show us the difficulty of what we're trying to do, if even the top folks can't shoot all 95%+ classifiers all the time. Bingo!!! This is why so many on this topic have questioned why so many classifiers have increased the HHF. Not saying it is unjustified but what is the methodology. Did someone really analyze the raw data? Link to comment Share on other sites More sharing options...
stick Posted July 9, 2018 Share Posted July 9, 2018 1 minute ago, Balakay said: Bingo!!! This is why so many on this topic have questioned why so many classifiers have increased the HHF. Not saying it is unjustified but what is the methodology. Did someone really analyze the raw data? What if? What is USPSA actually gave the statistics along with their reasoning for the update. By that I mean: Sample Size Scores taken from what Area's Actual stats for classifiers (how many times each classifier was shot) Best times/scores for each classifier Link to comment Share on other sites More sharing options...
Balakay Posted July 9, 2018 Share Posted July 9, 2018 2 minutes ago, stick said: What if? What is USPSA actually gave the statistics along with their reasoning for the update. By that I mean: Sample Size Scores taken from what Area's Actual stats for classifiers (how many times each classifier was shot) Best times/scores for each classifier This is what should have already occurred Link to comment Share on other sites More sharing options...
stick Posted July 9, 2018 Share Posted July 9, 2018 2 minutes ago, Balakay said: This is what should have already occurred But did it? This is what we are all wondering Link to comment Share on other sites More sharing options...
broadus123 Posted July 9, 2018 Share Posted July 9, 2018 Keep in mind , if they did this once, they can do it whenever they feel like it. Link to comment Share on other sites More sharing options...
ATLDave Posted July 9, 2018 Share Posted July 9, 2018 30 minutes ago, stick said: But did it? This is what we are all wondering I would very much like to see the score data for any given classifier plotted on a curve. And then the new and old HHF's marked on the curve. Link to comment Share on other sites More sharing options...
teros135 Posted July 9, 2018 Share Posted July 9, 2018 Just now, ATLDave said: I would very much like to see the score data for any given classifier plotted on a curve. And then the new and old HHF's marked on the curve. Good idea, Transparency (what a concept!) Link to comment Share on other sites More sharing options...
Flyingpig Posted July 9, 2018 Share Posted July 9, 2018 (edited) Actually heres a look at the best of the best and it tells a slighly different story. Way more consistent. This is a stretch in 2014. One might be a major win but most are not. I counted 26 100% including major wins. production shooter Local GM(and one of the best): 97.3850 - 11/11/14 Legacy 100.0000 - 10/08/14 Legacy 96.1570 - 10/06/14 Legacy 100.0000 - 10/13/14 Legacy 75.8050 - 9/10/14 Legacy 98.9668 - 9/10/14 Legacy 91.2030 - 8/11/14 Legacy 97.4359 - 8/12/14 Legacy 100.0000 - 7/10/14 Legacy 100.0000 - 7/09/14 Legacy 100.0000 - 6/11/14 Legacy 100.0000 - 6/05/14 Legacy Edited July 9, 2018 by Flyingpig Link to comment Share on other sites More sharing options...
MikeBurgess Posted July 9, 2018 Share Posted July 9, 2018 Another top shooters current list. dropped decimals off because I'm lazy. only 1 is a Major match placement 100 83 81 100 94 87 100 91 96 100 100 Oddly of the 6 that count 2 would move down 1 of the 100s goes to 94.8 and the 96 goes to 94.6 Link to comment Share on other sites More sharing options...
malobukov Posted July 10, 2018 Share Posted July 10, 2018 1 hour ago, ATLDave said: I would very much like to see the score data for any given classifier plotted on a curve. And then the new and old HHF's marked on the curve. Here are Q-Q plots of the hit factors for the most popular classifiers in production. I only looked at a relatively small recent sample, but in absence of official data it's still better than guessing. Old HHF is blue horizontal dashed line, new HHF is green horizontal dashed line. You can see that using normal approximation, HHF is about three standard deviations above mean. You can also see how bent the left tail is because of zero runs. Link to comment Share on other sites More sharing options...
Balakay Posted July 10, 2018 Share Posted July 10, 2018 1 hour ago, MikeBurgess said: Another top shooters current list. dropped decimals off because I'm lazy. only 1 is a Major match placement 100 83 81 100 94 87 100 91 96 100 100 Oddly of the 6 that count 2 would move down 1 of the 100s goes to 94.8 and the 96 goes to 94.6 Cue the dueling banjo music, another top 5 National shooter from a different division: 82 63 96 77 100 75 81 94 95 82 99 Let's not lose sight of the fact of why I posted this data originally. The HHF clearly is not what the best shooters in our sport can expect to do on a routine basis. I think that it makes you question the true meaning of what a 100% classifier means. Sometimes they get 100%, more often they do not. Is the HHF the best HF ever recorded? Is it the average of X number of the best runs? Is it based on retrospective analysis of all reported classifiers and adjusted accordingly? Is it an arbitrary threshold that will continue to evolve over time? Going a step further, as someone alluded to earlier in the thread, what exactly does classification even signify? M = 85% of what exactly Link to comment Share on other sites More sharing options...
IHAVEGAS Posted July 10, 2018 Share Posted July 10, 2018 9 hours ago, Balakay said: Going a step further, as someone alluded to earlier in the thread, what exactly does classification even signify? I'd peel off one more layer. What is the purpose of a classification? In the rest of the world it is to make competition the most fun for the greatest majority of participants by giving people a chance for friendly competition against other people of similar abilities. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now