PocketPistol Posted October 4, 2018 Share Posted October 4, 2018 I am a PCC shooter B class shooter and decided I wanted to do some dry fire practice and compare my dry fire performance with hit factor cut offs. Not so easy as it turns out. Unfortunately http://www.classifiercalc.com/ and https://azshooters.org/classifier does not have PCC data and it is unclear if data represents the recent changes in high hit factors. There is a calculator on the USPSA website but if you try to query more than 15 stages in tells you that you have queried too many, and shuts you down for an hour. Very obnoxious. Fortunately, you can calculate the high hit factors yourself if you have a pair of percentages and hit factors. Well, I wrote a python script to scrape the USPSA member classifications levels and results from A100000 (mid 2016)-A105000 (mid 2017) (1912 users) users and all the life users L135-L4537 (3889 users). I ignored expired users whose data shows up with an 'x'. This is only about 20% of the USPSA membership, but I think it gives me some interesting none the less. I could scrape more, but haven' taken the time to figure out where the start and stopping numbers are for each of these membership categories (A, B, CA, CAL, CL, F, FL, FY, FYF, HCL, L, RD, S, TY, TYF, etc.). If someone wants to do that, I am happy to scrape more data... The first thing I tried to answer is which division does the average shooter do the best in? Is PCC really easier than limited? Again this is only based on 20% of the membership. Open Limited Limited10 Production Revolver Single_Stack Carry_Optics PCC Max 93.2122 98.1042 91.1341 95.0249 76.6005 85.8647 91.4708 99.9841 Average 52.5649 46.2836 41.955412 47.383663 48.41808 42.88808503 48.51231235 57.4404 The results seem to show that the average PCC shooter is scoring higher percentages than all the other classes compared to their own high hit factors. Since PCC is introduced so recently, maybe results are being skewed by experienced people moving into PCC vs. other divisions that have new people starting there? Well, here is the data just looking at users who joined mid 2016 through mid 2017... Open Limited Limited10 Production Revolver Single_Stack Carry_Optics PCC Max 88.4506 90.2799 78.0854 95.0249 76.6005 77.9958 91.4708 99.9841 Average 51.1567 45.8634 40.186086 47.2301201 49.17671 42.48816879 48.5311964 57.5547 So not only are PCC shooters shooting 12% higher than limited shooters from the same time period, not a single annual member shooter who signed up between mid-2016 to mid-2017 has attained GM besides in PCC and Production. This is probably why the PCC hit factors were changed, right? Well I don't think so... I also scraped all the recent results for the above member accounts (only kept the results that were not "legacy" as legacy results don't have HF's). This got me about 52,000 scores. So for instance, in my data I got 74 scores of Carry Optics shooting 03-02 which fall into 2 groups, those hit factors before 6/28 (7.15), and those hit factors after (7.07). By doing some simple excel magic, I was able to pull out all the older HHF's, and newer HHF's for the 81 stages * 8 divisions minus a couple particular scores that seems like have not been shot (e.g. could not find a single revolver shooter shooting 06-07 among these ~6000 shooters). Anyway, after doing this, I compared the new HHF's with the old HHF's, then calculated the percent change for each class. The average percent changes are below... Open Limited Limited10 Production Revolver Single_Stack Carry_Optics PCC 3.1% 3.1% 3.1% 2.3% -3.1% -3.5% -4.8% -3.8% So Open, Limited, Limited 10, Production got net harder, but Revolver, SS, CO, and PCC got net easier, on average. Now, it is true there are stages that got much harder -- e.g. 06-05 got 27% harder for PCC shooters, but where 09-13 got 23% easier for PCC shooters... That is a big change... So what is next? Ideas - My thought is gather more data, and do some data analytics on the data to see what the distributions look like for each stage? For this I could use some help with what ranges of member numbers I need to pull in (e.g. what is the first A number, last A number, is the first member number formatted as A1, or A0001, etc.). The USPSA member sticky was useful, but outdated. I'd really like to understand the algorithm about how the HHF is determined specifically (e.g. is it the average of the 10 highest shooters, etc.)? This may become evident if I had all the data. For everyone who wonders what the point of this is, well, I got to teach myself python and learn more about USPSA HHF's than I really wanted to know... Anyway, something to chew on... Link to comment Share on other sites More sharing options...
selecw Posted October 4, 2018 Share Posted October 4, 2018 Excellent. Link to comment Share on other sites More sharing options...
Flatland Shooter Posted October 4, 2018 Share Posted October 4, 2018 One heck of a first post. Also very interesting. Link to comment Share on other sites More sharing options...
robchavous Posted October 4, 2018 Share Posted October 4, 2018 InterestingSent from my SM-G930V using Tapatalk Link to comment Share on other sites More sharing options...
stick Posted October 4, 2018 Share Posted October 4, 2018 Interesting data and analysis. I would like to see the data from some of the more seasoned shooters that have been around for a few years. Link to comment Share on other sites More sharing options...
Balakay Posted October 4, 2018 Share Posted October 4, 2018 Nice!! I get the sense that you have already put more effort into this than the "statisticians" at the USPSA Link to comment Share on other sites More sharing options...
PocketPistol Posted October 4, 2018 Author Share Posted October 4, 2018 1 hour ago, stick said: Interesting data and analysis. I would like to see the data from some of the more seasoned shooters that have been around for a few years. Easy answer is to look at the life shooters whose data I already have. There is probably a better metric for what defines a seasoned shooters than life, but it is all the data I have at the moment. Open Limited Limited10 Production Revolver Single_Stack Carry_Optics PCC Max 99.9833 100 97.9607 100 99.8677 99.8923 97.3264 100 Average 62.39926 58.57738 55.30444 56.35329 49.5888 53.98289 50.93657 61.95755 I have been running my analysis on "Pct" field, not the "High Pct" field for the max/average comparisons. Am I correct that Pct is your average of your best 6 of your last 8 and high percent is your best 6 of any consecutive 8 in history? If so, my data is really a latest snapshot in time for all shooters whose data I have. Might be interesting to compare the difference between Pct and High Pct? Not sure what that would tell you. The larger this number, maybe the more you have to spray and pray? Don't know. I wonder if the lower average scores reflect when someone reaches a certain point in a division they stop and move to a different division? E.g., I got my A in revolver, so now I am moving to limited. I reached the point in limited I want, so I am moving to open? Link to comment Share on other sites More sharing options...
stick Posted October 4, 2018 Share Posted October 4, 2018 5 minutes ago, PocketPistol said: Easy answer is to look at the life shooters whose data I already have. There is probably a better metric for what defines a seasoned shooters than life, but it is all the data I have at the moment. Open Limited Limited10 Production Revolver Single_Stack Carry_Optics PCC Max 99.9833 100 97.9607 100 99.8677 99.8923 97.3264 100 Average 62.39926 58.57738 55.30444 56.35329 49.5888 53.98289 50.93657 61.95755 I have been running my analysis on "Pct" field, not the "High Pct" field for the max/average comparisons. Am I correct that Pct is your average of your best 6 of your last 8 and high percent is your best 6 of any consecutive 8 in history? If so, my data is really a latest snapshot in time for all shooters whose data I have. Might be interesting to compare the difference between Pct and High Pct? Not sure what that would tell you. The larger this number, maybe the more you have to spray and pray? Don't know. I wonder if the lower average scores reflect when someone reaches a certain point in a division they stop and move to a different division? E.g., I got my A in revolver, so now I am moving to limited. I reached the point in limited I want, so I am moving to open? Great analysis. Interesting question you pose about different divisions and shooters ability. An interesting analysis would be seeing if you could see the differences of percentages from shooters with multiple classifications. Link to comment Share on other sites More sharing options...
motosapiens Posted October 4, 2018 Share Posted October 4, 2018 11 hours ago, PocketPistol said: I'd really like to understand the algorithm about how the HHF is determined specifically (e.g. is it the average of the 10 highest shooters, etc.)? This may become evident if I had all the data. i lol'd at that. over time there appear to be different methods used at random, most recently it appeared to be a person with little/no statistical background looking at numbers and making a wild-ass guess. Link to comment Share on other sites More sharing options...
PocketPistol Posted October 5, 2018 Author Share Posted October 5, 2018 13 hours ago, stick said: Great analysis. Interesting question you pose about different divisions and shooters ability. An interesting analysis would be seeing if you could see the differences of percentages from shooters with multiple classifications. Here is the data for all shooters I have scraped so far (again, only 20% of members) that are classified in 8 divisions.. Link to comment Share on other sites More sharing options...
stick Posted October 5, 2018 Share Posted October 5, 2018 6 hours ago, PocketPistol said: Here is the data for all shooters I have scraped so far (again, only 20% of members) that are classified in 8 divisions.. Interesting numbers and great work. I'm not sure that shooters who shoot 8 divisions are necessarily the norm. Can you scrape shooters with 2 or 3 divisions? I'm feeling that this would be a better representation of the general population of USPSA shooters who should bring the numbers more inline with your initial findings. Link to comment Share on other sites More sharing options...
Steve RA Posted October 5, 2018 Share Posted October 5, 2018 L1740 seems to be doing very well ! Link to comment Share on other sites More sharing options...
PocketPistol Posted October 5, 2018 Author Share Posted October 5, 2018 3 hours ago, stick said: Interesting numbers and great work. I'm not sure that shooters who shoot 8 divisions are necessarily the norm. Can you scrape shooters with 2 or 3 divisions? I'm feeling that this would be a better representation of the general population of USPSA shooters who should bring the numbers more inline with your initial findings. I have that data, but the problem is there are 56 combinations of ways people can shoot 3 of 8 divisions, and 28 ways people can shoot 2 of 8 divisions... I am not really sure the best way to present that data in a meaningful way? Plus, by the time I take that data, and divide it up into 84 bins, I am not sure if it is statistically significant? Link to comment Share on other sites More sharing options...
ATLDave Posted October 5, 2018 Share Posted October 5, 2018 Dear lord... actual data! PocketPistol, if you happen to find yourself in Atlanta, a beer or coffee is on me. Thanks for your effort and contribution! Link to comment Share on other sites More sharing options...
Bkreutz Posted October 5, 2018 Share Posted October 5, 2018 OOOOh, all those numbers make my head hurt. Link to comment Share on other sites More sharing options...
motosapiens Posted October 5, 2018 Share Posted October 5, 2018 2 hours ago, PocketPistol said: I have that data, but the problem is there are 56 combinations of ways people can shoot 3 of 8 divisions, and 28 ways people can shoot 2 of 8 divisions... I am not really sure the best way to present that data in a meaningful way? Plus, by the time I take that data, and divide it up into 84 bins, I am not sure if it is statistically significant? i would think for that data to be meaningful it would need to take into account when the classifiers were shot, and how many in a division. My L10 and CO percentages look pretty low compared to Lim and SS, but I haven't shot L10 since the 2nd year after started, and I've hardly shot any CO classifiers, so no real conclusions could be drawn from that. Link to comment Share on other sites More sharing options...
nate89 Posted October 5, 2018 Share Posted October 5, 2018 Awesome post! Very interesting data and I look forward to seeing what else shows up. Link to comment Share on other sites More sharing options...
Flatland Shooter Posted October 5, 2018 Share Posted October 5, 2018 6 hours ago, Steve RA said: L1740 seems to be doing very well ! Based on the dates of his recent classifiers, he has also swung over to the darkside. Link to comment Share on other sites More sharing options...
PocketPistol Posted October 9, 2018 Author Share Posted October 9, 2018 Per forum guidance, I won't be posting HHF's even though I have them and they are now published on the cell phone app. However, here is the percent change from the HHF update...Missing cells I don't have data for post HHF update so the cell is undefined. Green indicates the stage got easier, red means it got harder. The last row has the average change. Link to comment Share on other sites More sharing options...
PocketPistol Posted October 9, 2018 Author Share Posted October 9, 2018 Here is a chart of popularity of stages and divisions (green popular, red unpopular) over roughly the last 3 years... I just have a sampling of data but trends should hold... Link to comment Share on other sites More sharing options...
PocketPistol Posted October 31, 2018 Author Share Posted October 31, 2018 I had a request for distribution of shooter classifications. I have data for about 15,000 members (I hear there are about 30,000 members). Not currently too concerned about getting all the data, I think this is enough to draw some conclusions. I am also plotting "percent" and not "high percent" so someone who was a master but is currently shooting A will show up in my data as an A. Link to comment Share on other sites More sharing options...
Smithcity Posted October 31, 2018 Share Posted October 31, 2018 12 minutes ago, PocketPistol said: I had a request for distribution of shooter classifications. I have data for about 15,000 members (I hear there are about 30,000 members). Not currently too concerned about getting all the data, I think this is enough to draw some conclusions. I am also plotting "percent" and not "high percent" so someone who was a master but is currently shooting A will show up in my data as an A. Very cool. Based on your data it appears carry optics hasnt pulled away much from production when it comes to HHF and is ripe for more GMs. Link to comment Share on other sites More sharing options...
Derrick-508 Posted December 13, 2018 Share Posted December 13, 2018 thank your for doing this! It is all very interesting Link to comment Share on other sites More sharing options...
Seanziegler Posted December 17, 2018 Share Posted December 17, 2018 (edited) This is super cool. One thing that confuses me when trying to interpret the data: why is it that PCC has the highest average classifier yet Open has a higher percentage of A/B classified shooters Edit: Where did you scrape this from exactly? Edited December 17, 2018 by Seanziegler Link to comment Share on other sites More sharing options...
Bruno45 Posted December 17, 2018 Share Posted December 17, 2018 This is interesting Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now