Steel Challenge PST Formula Needed

[JM_]

Hey Everyone, I'm looking for assistance and recommendations on a formula we can use to accurately define what PST times need to be.

 

A couple rules that I would like to see followed as best as possible: 

Attempt to only use WSSC data (maybe take average over multiple WSSC's?), but if we need to reach further and use Area matches, we could.

Must be able to be used year over year without drastically changing times (unlike like the current formula)

Still make the divisions competitive. I believe a goal of 2.5-3% GM rate per divison is competitive. RFPO is currently in that range, so ideally that peak time hardly changes or stages are adjusted slower and faster but still fall within the current overall peak time.

 

This post is not intended as a "please do the work for me (Jesse)" becuase I would like a lot of discussion and ideas thrown out so we can make this robust as possible.

@Hoops @GKB @Renno have some fun with it

 

Once we get something good, we can pitch it to the SCSA comittee!

 

Edited July 25 by JM_
typos

[JM_]

Also to add: Lets try not to use the current PST in the equation as they are flawed as it is. Starting fresh would be ideal.

[Hoops]

@JM_ as you think about this….or all of us…..perhaps some consideration should be given to how classifications can be earned.  Many ranges can’t shoot Outer Limits or Speed Option.  So many classifications from C thru GM can be earned shooting only 4 thru 6 stages.  Just a thought.

[JM_]

Just now, Hoops said:

@JM_ as you think about this….or all of us…..perhaps some consideration should be given to how classifications can be earned.  Many ranges can’t shoot Outer Limits or Speed Option.  So many classifications from C thru GM can be earned shooting only 4 thru 6 stages.  Just a thought.

You are correct. So ideally all stages would be equally as difficult to hit GM as each other. However I do believe thats just the nature of the beast that we will have to accept. There are people who could theoretically game it, but those people aren't the norm.

[shred]

Random thoughts-- maybe take the top-10 (in competitive divisions) total match scores and use those to generate the "100%". 

 

A person that shoots Top-10 at the WSSC should be a GM, right? 

 

If Top-Ten isn't the right number, use Top 8 or Top 16 or whatever works (this also implies miniscule-participation divisions should be folded into others or at least use their 100%). 

 

I don't shoot enough major SC anymore to know, but back in the Piru days, 8-10 was about the number of people in the hunt to win it all. 

 

Then sort out how to parcel that total time out among the stages and issue classifications.  

 

Doing the top-score on a per-stage basis gives extra weight to hero-or-zero runs versus across-the-match consistency.

 

[JM_]

Heres what I've got so far.

 

Taking the top 20% of RFRO/RFPO stage times, averaging them together, and then totaling it gives me roughly the same PST for RFPO and a 2.7% GM rate for RFRO. PCCO would be 3.9% GM, but if that many people are good at PCCO then I suppose it is what it is. The tricky part are the iron divisons. I would almost say we ignore WSSC of RFRI/PCCI/RFPI and instead base it off the perceived difficulty compared to their optics counter part. Taking top 20% PCCI times of worlds nets me with a peak time of 65, which is faster than PCCO. Common sense says irons should always be slower than optics, so we could make RFRI/PCCI 5% slower per stage compared to the peak times of rfro/pcco and rfpi can be 15% slower (rfpi is much more difficult compared to rfpo than when comparing rfro/rfri)

[Hoops]

I think there is some anticipation of more shooters requesting reclassification in early 2025.  Maybe by those that shoot majors?  But the vast majority probably won’t.  For some, there is a strong position of I earned back when….and I’m keeping it.

[ZackJones]

I'm looking forward to seeing your input on this one. It's good to see some of you interested in proposing improvements instead of just complaining. Some advice:

 

1-Keep this about PST's How classifications are earned (4 versus 6), etc. should be a separate topic. I'm inclined to think 6 would be better than 4 but that's just me. 

2 - Each Stage and Division is unique. Meaning don't look at the time for RFRI versus RFRO on a given stage.

3 - I've calculated every stage / division from using 95-110, 115, 120, 125, 130, 135, 140, 145 and 150 to see where the numbers fall. Those numbers are pretty interesting. 

4 - We currently use a minimum of 5 GM times before we consider adjustment. We are discussing bumping it to 10 in the future. That may be worth looking into. 

5 - Averages for top 20-50 times shot at WSSC matches results in dramatic changes. I would avoid that path as a possibility. 

6 - PST's only go down, we have never adjusted up. Should we? 

7 - There is no target percentage of classifications within any division. Right now B/C class has more in it than the other classes across all divisions. 

8 - Should we cap the amount of change per year? What's the pain point? Is it 4 seconds? Anything 4.00 and below is okay but 4.01 or higher makes the membership grab their torches and pitchforks? 

9 - Whatever you come up with try it over multiple results and see how the numbers look. 

10 - Not advice but I had so many I wanted to make it ten - Thanks for doing this. No classification system is perfect. 

[JM_]

2 minutes ago, ZackJones said:

I'm looking forward to seeing your input on this one. It's good to see some of you interested in proposing improvements instead of just complaining. Some advice:

 

1-Keep this about PST's How classifications are earned (4 versus 6), etc. should be a separate topic. I'm inclined to think 6 would be better than 4 but that's just me. 

2 - Each Stage and Division is unique. Meaning don't look at the time for RFRI versus RFRO on a given stage.

3 - I've calculated every stage / division from using 95-110, 115, 120, 125, 130, 135, 140, 145 and 150 to see where the numbers fall. Those numbers are pretty interesting. 

4 - We currently use a minimum of 5 GM times before we consider adjustment. We are discussing bumping it to 10 in the future. That may be worth looking into. 

5 - Averages for top 20-50 times shot at WSSC matches results in dramatic changes. I would avoid that path as a possibility. 

6 - PST's only go down, we have never adjusted up. Should we? 

7 - There is no target percentage of classifications within any division. Right now B/C class has more in it than the other classes across all divisions. 

8 - Should we cap the amount of change per year? What's the pain point? Is it 4 seconds? Anything 4.00 and below is okay but 4.01 or higher makes the membership grab their torches and pitchforks? 

9 - Whatever you come up with try it over multiple results and see how the numbers look. 

10 - Not advice but I had so many I wanted to make it ten - Thanks for doing this. No classification system is perfect. 

Thanks Zack!!

 

1, I think 6 is better, would promote ranges to shoot more stages. But I don't think its urgent that this is changed.

 

2, I strongly disagree, I shoot rfri the exact same as rfro, but its marginally slower becuase its an irons gun. I shoot rfpi the same as rfpo but slower because its irons. Lets use common sense here, irons will always be slower than optics, but the way they are shot will be the same.

 

3, Agreed, I think roughly using average of top 20% shooters of worlds in a division gives us a good challenge but nothing too hard. However we lack data for most irons in WSSC. If we use 20% rfri the peak times will be faster than RFRO - that doesn't make sense.

 

4, I would have to think about this after coming up with my hard copy of a peak time equation

 

5, Taking top 20% of a division with enough entries gives good data (RFRO,RFPO,PCCO). top 20% of small divisions gives very bad data (PCCI, RFRI, RPIF). Thats where percieved difficulty comes into play. PCCI will always be slower than PCCO, common sense. I on average shoot PCCI 5% slower than PCCO (as do many others). That should be relfected in our peak times. I can use math to back this up by comparing the time difference between the optics and irons for the same exact person. 5% is a rough estimate, could be 4% could be 6%.

 

6, If we want to keep center fire divisons alive, making them easier to GM will give people a reason to shoot them. Yes the great centerfire shooters of long ago shot faster, but they are gone and we need to adapt to what we have now. If RFRO gets marginally slower one year, peak times should reflect that.

 

7, I agree, but we should use percentage of GM's as a guide to make sure we aren't going off track. 

 

8, No need to cap since the equation I'm coming up with will prevent mass changes.

 

9, I'll graph it out over multiple WSSC's so we can see how times change year over year.

 

10, 100% agree it will never be perfect, but we can atleast try!

 

[Hoops]

@ZackJones.  Thanks for joining in.  Appreciate your comments.

 

Your item 7 caught my eye.  If B/C right now are stacked ( @GKB charts show this), what was objective for 2025 PST’s to address this.  

The predictive assumptions are that Jan 2025 these classes probably will get larger and the bell curve will be much steeper on both ends.  

 

This is a good conversation.  
 

thanks

[Hoops]

@JM_  it’s July.  So are we discussing 2025 or 2026?

 

 

[JM_]

5 minutes ago, Hoops said:

@JM_  it’s July.  So are we discussing 2025 or 2026?

 

 

I would like to see the current proposed peak times to be tossed and create a new formula for 2025 (using 2024 data).

Edited July 25 by JM_

[Hoops]

29 minutes ago, JM_ said:

I would like to see the current proposed peak times to be tossed and create a new formula for 2025 (using 2024 data).

Good.

 

Just a thought—— For combined scores and peak percentages, Steelrankings.com is an excellent reference.  2023 consolidates all state, area and 2023 WSSC matches.  While it does not include stages it is a fast reference to show how many people actually shoot class times regardless of classification.

 

I’m looking forward to a robust discussion.

 

 

[JM_]

Just now, Hoops said:

Good.

 

Just a thought—— For combined scores and peak percentages, Steelrankings.com is an excellent reference.  2023 consolidates all state, area and 2023 WSSC matches.  While it does not include stages it is a fast reference to show how many people actually shoot class times regardless of classification.

 

I’m looking forward to a robust discussion.

 

 

I've already been using it to guide myself!

[Hoops]

@jrdoran did a fantastic job building Steelrankings.com.  It started from a discussion about the top 20 rankings.  Jim deserves huge credit for his ability and the hours he spent on it.  Thanks Jim.

[jrdoran]

All good and thanks.  It is a pleasure seeing people like you guys use it.  

 

Really like the discussion about a repeatable formula year to year that would support a given BoD strategy around goals of classification system.    Reverse engineering a formula to achieve distrubtion  by stage is doable, but I still think we don't yet know what the end state goal of classification distribution is.  

 

I have the ability to simulate and publish a proposed change in PSTs and recalc ALL of 2024 shooters to see what the distribution would be.  I don't expose the stage by stage, but I would be willing to add that if useful.   While the PSTs are at a stage level, the end classification is at a combined shooter level.  You can be a 'A' shooter on one stage and a 100% on the next stage and still achieve GM.  I know you all know this, but they are different levers to pull.  

 

@JM_. The topic of match scores expiration should also be on the table.  The (current year  -2 )  approach drives hero/zero behavior.  The TOPSHOT2024 approach I added has gotten alot of feedback from people that they like the current year performance point of view.   

Edited July 25 by jrdoran

[JM_]

19 minutes ago, jrdoran said:

All good and thanks.  It is a pleasure seeing people like you guys use it.  

 

Really like the discussion about a repeatable formula year to year that would support a given BoD strategy around goals of classification system.    Reverse engineering a formula to achieve distrubtion  by stage is doable, but I still think we don't yet know what the end state goal of classification distribution is.  

 

I have the ability to simulate and publish a proposed change in PSTs and recalc ALL of 2024 shooters to see what the distribution would be.  I don't expose the stage by stage, but I would be willing to add that if useful.   While the PSTs are at a stage level, the end classification is at a combined shooter level.  You can be a 'A' shooter on one stage and a 100% on the next stage and still achieve GM.  I know you all know this, but they are different levers to pull.  

 

@JM_. The topic of match scores expiration should also be on the table.  The (current year  -2 )  approach drives hero/zero behavior.  The TOPSHOT2024 approach I added has gotten alot of feedback from people that they like the current year performance point of view.   

Scores expiring is a hard subject becuase of the poeople who zero or hero it. Again those that game the system like that are always going to game the system. May just need to accept that it happens and ignore it. Top Shot is great though and it would be cool to integrate it with actual ranks somehow.

[shred]

Should say Marvin the Martian dropping in to shoot a division 2x faster than the next-fastest competitor cause the PST to drop dramatically?

 

Where are the Median and Average shooters vs the PST?

[JM_]

4 minutes ago, shred said:

Should say Marvin the Martian dropping in to shoot a division 2x faster than the next-fastest competitor cause the PST to drop dramatically?

 

Where are the Median and Average shooters vs the PST?

I'm currently drafting up my equation ruleset, but honestly if someone is able to shoot twice as fast, that time would get averaged in. Would it make a crazy difference? No. If I took top 20 overall scores, and made one 8 stage score score a 1 second run, it would drop the average by only 2 seconds.

[JM_]

6 minutes ago, JM_ said:

I'm currently drafting up my equation ruleset, but honestly if someone is able to shoot twice as fast, that time would get averaged in. Would it make a crazy difference? No. If I took top 20 overall scores, and made one 8 stage score score a 1 second run, it would drop the average by only 2 seconds.

Adding onto this: I'm not saying I'm going to use top 20 for the peak times. Was just using top 20 as an example. 

[GKB]

I don't know if this helps or is just noise, but since I already had pulled it...Here are the #1 classified shooter in each division compared to the 2024 WSSC winning time.

 

The impressive part on the WSSC times is that these are a single match where the #1s are the time from their best stage from any match they've shot from 2022 to now.

 

[Hoops]

@JM_ @ZackJones 

 

At last Saturday's match, there was a lot of discussion regarding the proposed 2025 PST's.  We have a good cross section of classes and categories.  The biggest questions I had was; (1) Why?  (2) What is the point of trying to reach another classification when "they" [SCSA] will just change the peak times each year?  (3) What is SCSA trying to achieve through the constant changes in peak times?  Some form of strategy?

 

When I told them that the peak times are developed from the WSSC's, the responses are why are 100 or less people dictating times for the entire membership? 

 

The questions that I read in Enos from the four different topics so far, is why did the methodology/formula change this year?  Zack's number 3: "I've calculated every stage / division from using 95-110, 115, 120, 125, 130, 135, 140, 145 and 150 to see where the numbers fall. Those numbers are pretty interesting."   The impression is there is no set basis but instead a search for some pre-determined outcome even though Zack also stated in number 7: "There is no target percentage of classifications within any division. Right now B/C class has more in it than the other classes across all divisions."

 

Zack's number 8 was interesting: "Should we cap the amount of change per year? What's the pain point? Is it 4 seconds? Anything 4.00 and below is okay but 4.01 or higher makes the membership grab their torches and pitchforks?"

 

Breaking down number 8, I had several thoughts:

 

4.01 seconds.  The classification system is regressive.  4.01 seconds reduction for a GM breakdown for other classes:

M=4.22 to 4.72

A=4.71 to 5.36

B=5.35 to 6.68

C=6.69 to 10.03

 

@GKB has shown several tables that support that B/C class has more in it than the other classes across all divisions.  I get the argument that with practice, etc, the B and C shooters should be able to work up to A, and so on.  I don't have the data from 2019 or 2020 to compare to see if B/C has always been the largest groups.  

 

If there is no cap on classifications, then I would suggest that whatevere formula is used for PST', it should be tested against all the classes and not changed for subsequent years.  Adjusting PST's from 110% up to world record levels arbitrarily may not be the best approach.....and it certainly has caused a lot of comments here in Enos and at the local level matches/membership as the new peak times are distristributed.

 

I would suggest that using the WSSC scores as the exclusive basis may not be representative of the membership.  At a minimum perhaps the evaluation of PST's should include Area and State matches?  

 

@JM_ I understand you believe the 2025 times should be reviewed before they are made formal, but usually when times are posted on the SCSA website, there is usually an article written in the USPSA magazine.  I do not know how @ZackJones is proceeding with this, but perhaps this may be a question.  The final publication of the 2025 PST's should be paused pending further discussion?

 

 

[shred]

The new PSTs are a done-deal unless a lot of people make a lot of noise with the USPSA BOD.  Very few of them shoot any SCSA, let alone seriously, so they're just going to defer to whatever the "steel guy" says.

 

 

[JM_]

8 minutes ago, shred said:

The new PSTs are a done-deal unless a lot of people make a lot of noise with the USPSA BOD.  Very few of them shoot any SCSA, let alone seriously, so they're just going to defer to whatever the "steel guy" says.

 

 

Agreed, But Zack seems open to the idea, and I've already had personal talks with some BOD personnel. 

[GKB]

I think everybody needs to stop and think what the PSTs and Classification represent.

 

It's an acknowledgement of a person hitting a skill level ("A", etc.) and allows recognition of performance within that subset of similar skills.  I think SCSA would die without it if each match was just a ranking by time of fastest to slowest.  The classification system lets people have something to work to that is independent of the individual match results.

 

It's not a Top 20/Top 3% or something club.

 

So why change the PSTs?  Hitting a skill mark (say, moving up to "B") doesn't change that skill no matter how well somebody shoots at WSSC.  (Exception, of course, for new Divisions where the PST is still being felt out.)

 

In theory, everybody in Steel Challenge could put in the time and hit 95%/GM.  That doesn't mean the times are too slow, just that people are putting in the effort to develop the skill.

 

Pick a mark that fairly represents the skill progressions and leave it alone.  This will also prevent the demoralization/discouragement of having the goal posts move on you every year.