Where were the sandbaggers?

[grandbagger]

Match A and Match B are simple samples of the population. As a A class shooter, I can score 95% of the points in a match. That does not make me a GM. I can also score 30% in a match, again that does not make me a D shooter

very true, i resemble this remark.

Edited December 1, 2009 by grandbagger

[bbbean]

bbean,

With all due respect, the distributions of sample and population are key. You are using simple percentages which I guess prove your point to yourself. But reality is this is a statistical analysis problem.

I'm glad we're showing each other all due respect!

I'm familiar with "statistical analysis problems", having spent a substantial portion of my academic career in research design and data analysis. In this particular case, I believe looking at the top finishers in each class at the national championships is a valid means of either supporting or challenging the hypothesis that the USPSA class system is a good predictor of major match performance.

Match A and Match B are simple samples of the population. As a B class shooter, I can score 95% of the points in a match. That does not make me a GM. I can also score 30% in a match, again that does not make me a D shooter. Hence, I have proven the null hypothesis to be true which means your hypothesis is not true.

Note first and foremost note that the null hypothesis is never proven. It can only be "not rejected". Also note that matches A & B were simply q&d illustrations to show that distribution is not the primary phenomenon I'm interested in. I'm not sure what point you're arguing.

If you are going to compare a sample (a match) to the population (classifier system) the data has to be normalized.

At this point I concede the point.

One could argue that the classifier system is normalization, although that's not how I choose to use it here. At any rate, the question I'm interested in is; "Does the USPSA class system accurately predict match performance (given an adequate number of GMs)?" A secondary question is "Is there a problem with sandbaggers manipulating the class system to enhance their finish in major matches?"

While a complete study wouldn't be that hard to do, I'm pretty satisfied that this preliminary study would indicate the answers to be (in order) yes and no. The data is there if someone wanted to look at Nats scores and other Major match scores going back to the beginning, but I'll leave that for others.

[Loves2Shoot]

While a complete study wouldn't be that hard to do, I'm pretty satisfied that this preliminary study would indicate the answers to be (in order) yes and no. The data is there if someone wanted to look at Nats scores and other Major match scores going back to the beginning, but I'll leave that for others.

As you say you are someone with a history in data analysis I can not see how you would draw that conclusion, even if you chose to ignore the 15-20% gap of the top 20 GM's and sandbagging M's

Going to the major matches you will see that the shooters in the top of A, B, M are finish above the majority of the shooters in the class above them, that fact alone disproves your hypothesis in my small brain.

Analysis from the match you took your data from:

M class winner beat more GM's than beat them. The same is true of the winner of A class as they beat more M's than M's beat them, B class winner beat more A class shooters then beat them (by a ton), and C class winner beat more B class shooters than beat them.

Would you not say that if you beat more shooters in the class above you than beat you that you should be in that class if the classification system was an accurate reflection of skill?

Fact is the classification system only works for those who work it. For the 10+ years I've shot the sport the top finishers in there class at the majors have always been under classified in relation to the other shooters in the class by the fact they beat more shooters in the class above them than beat them.

I believe the classification system is only good for folks who use it as a tool to track their own personal improvements, it just is not accurate measurement tool when you use it to give away prizes, and it measures (in large) only a static skill set (stand and shoot), where by in large the stages at real matches are dynamic (run and gun.)

I do think the hypothesis you stated is the reason some shooters (not Chris) are under classified. I know well one guy who had always been a class under his skill level, and when called out his comment was always he finished in his percentage at the majors. That is why I first did the analysis myself. After I showed him the numbers, he quickly classified up to his actually skill level.

[Nik Habicht]

Scott,

so you're suggesting that if a master beats a GM or a bunch of GMs at Nats, he's underclassified? Maybe the GMs are overclassified? Maybe the GMs just had a really bad match, or may be older and not up to the rigors of three or four days in a row anymore? Not picking on anyone specific --- just in my (albeit very limited) experience Nationals require a unique blend of skill, endurance, and the ability to let go of things that are out of your control.....

I think any shooter finishing a Nats in their classification percentage range against the top GMs is appropriately classified, regardless of who they beat....

[bbbean]

While a complete study wouldn't be that hard to do, I'm pretty satisfied that this preliminary study would indicate the answers to be (in order) yes and no. The data is there if someone wanted to look at Nats scores and other Major match scores going back to the beginning, but I'll leave that for others.

As you say you are someone with a history in data analysis I can not see how you would draw that conclusion, even if you chose to ignore the 15-20% gap of the top 20 GM's and sandbagging M's

Going to the major matches you will see that the shooters in the top of A, B, M are finish above the majority of the shooters in the class above them, that fact alone disproves your hypothesis in my small brain.

I'm only interested in the top of the class. I'm looking for sandbaggers. Grandbaggers, shooters having a bad day, shooters with equipment problems, etc., aren't knocking out shooters who should have taken home a prize. Like the classification system, we can throw out finishes below class as outliers.

Consider - MM zeroed a stage at Nats. I don't think any of us would argue that stage was a true reflection of his abilities as a shooter. I beat Doug Koenig on one stage at the Single Stack Nats (as did 195 other shooters), but no one in their right mind would argue that I'm a better shooter than he is - he just happened to tank a stage I did well on.

Would you not say that if you beat more shooters in the class above you than beat you that you should be in that class if the classification system was an accurate reflection of skill?

I think the best 6 of my most recent 8 classifiers are a far better representation of my skill level than the fact that several shooters at a major match had a bad day or equipment problems. I doubt many of the people who beat MM on stage 16 at the Open Nats would argue that their finish on that stage made them a GM.

[gmantwo]

While a complete study wouldn't be that hard to do, I'm pretty satisfied that this preliminary study would indicate the answers to be (in order) yes and no. The data is there if someone wanted to look at Nats scores and other Major match scores going back to the beginning, but I'll leave that for others.

As you say you are someone with a history in data analysis I can not see how you would draw that conclusion, even if you chose to ignore the 15-20% gap of the top 20 GM's and sandbagging M's

Going to the major matches you will see that the shooters in the top of A, B, M are finish above the majority of the shooters in the class above them, that fact alone disproves your hypothesis in my small brain.

Analysis from the match you took your data from:

M class winner beat more GM's than beat them. The same is true of the winner of A class as they beat more M's than M's beat them, B class winner beat more A class shooters then beat them (by a ton), and C class winner beat more B class shooters than beat them.

Would you not say that if you beat more shooters in the class above you than beat you that you should be in that class if the classification system was an accurate reflection of skill?

Fact is the classification system only works for those who work it. For the 10+ years I've shot the sport the top finishers in there class at the majors have always been under classified in relation to the other shooters in the class by the fact they beat more shooters in the class above them than beat them.

I believe the classification system is only good for folks who use it as a tool to track their own personal improvements, it just is not accurate measurement tool when you use it to give away prizes, and it measures (in large) only a static skill set (stand and shoot), where by in large the stages at real matches are dynamic (run and gun.)

I do think the hypothesis you stated is the reason some shooters (not Chris) are under classified. I know well one guy who had always been a class under his skill level, and when called out his comment was always he finished in his percentage at the majors. That is why I first did the analysis myself. After I showed him the numbers, he quickly classified up to his actually skill level.

This post goes along with my own thinking nearly 100%. On a stage by stage basis, I can very occasionally put together a GM class score on 1 stage, maybe a M class score on one or two more. But in a 10-12 stage match my overall percentage always ends me up where I am in Class. I especially like this "it measures (in large) only a static skill set (stand and shoot), where by in large the stages at real matches are dynamic (run and gun.)". That is the ONE thing IMO that needs changed in the classification system. We usually include ONE classifier in a major match, and yet those are what determine who we shoot against in these matches. So only 8-10% of the stages we will see in a match are used to put us in class!? I would like to see all matches of level II or above be used as your classification. Shoot 2 matches and you're classified. Only in their first match would a new shooter have to be "unclassified" because they would class you upon the finish of your second. I know new shooters don't usually shoot level II matches, but might be more inclined to if they didn't have to shoot "unclassified". I can remember not attending some matches because I wasn't yet classified. It's a crap shoot, you might be shooting against a bunch of newbies, or you might be shooting against a couple of GM open shooters who are getting into Limited or some other division that's new to them.

You could keep the current system for "club level" classification. I would say probably 75% of people who shoot our local club matches are NOT USPSA members and never will be. They don't need, nor do they care about a class, they only want to see where they finish in our club. All of this, of course is IMNTBHO, lol.

You should never be H about your opinions, they are one of the few complete freedoms we have. Just don't be pushy!

[gmantwo]

Oh, yeah. Grandbaggers?? People who shoot below their class?? I thought that's what sandbaggers did? At least most of the time. What is the point of shooting below your average? Man, if I could shoot with a whole bunch of grandbaggers, I could maybe win a match!!

I just had an apifony, ipiphony, ipifony(??), whatever, maybe a reason why someone would grandbag. He has a "grand" scheme to be moved down in class?? Seems like a lot of work to me.

[Cy Soto]

I just had an apifony, ipiphony, ipifony(??), whatever...

Epiphany (\i-ˈpi-fə-nē\) - a (1) : a usually sudden manifestation or perception of the essential nature or meaning of something (2) : an intuitive grasp of reality through something (as an event) usually simple and striking (3) : an illuminating discovery, realization, or disclosure b : a revealing scene or moment.

[Loves2Shoot]

I'm only interested in the top of the class. I'm looking for sandbaggers.

They are winning their class at the nats, so they are easy to find. To win in your class at nats you MUST shoot a score that is above the performance of more than half of the shooters the next class up from you, that what the statistics say.

When the top of the M (or A, B, C the next class up) class finishes in the upper percentage of GM class (or the next class up), methinks they are not classified properly, as the idea of classification is group the like skilled shooters together, and the classification doesn't match the performance. Classification 100%'s take an aggregate of the top scores, not the best score ever recorded. If you took the top scores and aggregated them as they do for classification purposes, then you see the people shooting scores above their classification.

[CHA-LEE]

All of the Sand/Grand bagging does not matter if access to the prize table is done by finishing order per Division regardless of Classification. If you win your Class then you get a little trophy, ribbon, Pat on the back, kick in the nuts or whatever. But prize table access should only be done by finishing order. I don't see any better motivator on getting better than that.

The American Competition Mentality of “If I can’t win, I don’t want to play” kills me. This is why we have six different divisions (Open, Limited, Limited-10, Production, Single Stack, and Revolver) and then seven classifications within each division (GM, M, A, B, C, D, and U). Not to mention the sub classes within each one of those Divisions (Law, Lady, Senior, Super Senior, Junior, etc). All of these divisions, classes and categories are nothing more than something to appease the “If I can’t win, I don’t want to play” factor.

Just go shoot and have fun while doing it. Does it really have to be more complicated than that?

[SA Friday]

Oh, yeah. Grandbaggers?? People who shoot below their class?? I thought that's what sandbaggers did? At least most of the time. What is the point of shooting below your average? Man, if I could shoot with a whole bunch of grandbaggers, I could maybe win a match!!

I just had an apifony, ipiphony, ipifony(??), whatever, maybe a reason why someone would grandbag. He has a "grand" scheme to be moved down in class?? Seems like a lot of work to me.

I've always thought a grandbagger was a shooter that intentionally did things to improve their classification to above their actual shooting capabilities. You know the GM that can't ever beat the M's and half the A's at a local match...

[bbbean]

I'm only interested in the top of the class. I'm looking for sandbaggers.

They are winning their class at the nats, so they are easy to find.

With the exception of one D class shooter, the top finishers in each class finished exactly where you'd expect them to. If sandbagging were an issue, we'd expect to see each class won by someone shooting more than a few percentage points above the top of the class.

[gmantwo]

Thanks CY, I couldn't find my damn dictionary, lol.

[Loves2Shoot]

I'm only interested in the top of the class. I'm looking for sandbaggers.

They are winning their class at the nats, so they are easy to find.

With the exception of one D class shooter, the top finishers in each class finished exactly where you'd expect them to. If sandbagging were an issue, we'd expect to see each class won by someone shooting more than a few percentage points above the top of the class.

You are going off the best score not the average of the top scores of the top GM's to get the 100% to start with (like the classification system does.) If the classification system went off the best ever score, guys like Max would set the bar at least 10-20% higher than it is currently, lowering everyone's classifier scores by 10-20%, and that is what you see happen at the Nationals (if you look) or any other match with the top shooters.

You continue refuse to look at the whole set of data and the relationship to the other shooters in the other classes or the acknowledge the percentages you are using are calculated in different manners, and that is where your analysis fails in my opinion. You had around 35 masters in the data set you were looking at, only 5 shot scores above 85 percent, 30 shot less than 85%, and the 5 shooters shoot as high a score or better than half the GM's.

You can say the numbers "say this", but they aren't the same numbers and they are not calculated the same way, so to use them to draw a conclusion makes no sense.

All you have to do to see the classifier percentages don't work at a match with the top shooters is to look at the stage scores, only a handful of shooters will shoot above 95% (GM) score with over 30 GM's in attendance, and if you average the top 10% of the scores you will see the pattern continue, that same 10-20% upward skew.

When very few shooters in the class can shoot a score inside their classification percentage, what does that mean to you?

Is it the vast majority of folks who are over classified or a handful that should be in the next class up and who might just be "sandbagging" (or that the classification system doesn't catch up with)?

Numbers don't mean much if you do not understand what they are trying to measure or use the same standards for setting the base line measurement.

[el pres]

I dont know about all of this sand grand bull. As I stated in an earlier post that no one responded to,

I'm one of the guys of that list. I'm the 3rd B-class shooter, I have first hand expirience on this subject.

Here's the kicker, I made A-class a match or two right after the Nationals, why ?? Here's what I can tell you.

I only had one goal at the Nationals and that was to win b-class, nothing else, no prize table,nothing. Since

I believe b-class is about speed without all the alphas. I practiced and practiced for the Nats focusing on

alphas. The result was that I made A-class with all the classifiers leading up to the Nats as a side effect of

....get ready for this .... "practice" see me :TY59168

My percentage was right on with most of the classifiers i was shooting at the time. Wierd but even with a couple

of really stupid Mikes that cost me first place, I ended up at the right percentage. Leading up to the Nats I studied

the results of last years B-class winners to get a feel of where I needed to be and found that they were all A-class

shortly after the Nats as well ??? Huh ??

[Nik Habicht]

You are going off the best score not the average of the top scores of the top GM's to get the 100% to start with (like the classification system does.) If the classification system went off the best ever score, guys like Max would set the bar at least 10-20% higher than it is currently, lowering everyone's classifier scores by 10-20%, and that is what you see happen at the Nationals (if you look) or any other match with the top shooters.

Scott,

that's a lovely concept --- but we're now seeing classifiers that have originated in area matches and Nationals. On quite a few of those the HHF is equivalent to the HHF shot at the match of origination --- not some blend of various shooters high scores averaged together....

[Loves2Shoot]

You are going off the best score not the average of the top scores of the top GM's to get the 100% to start with (like the classification system does.) If the classification system went off the best ever score, guys like Max would set the bar at least 10-20% higher than it is currently, lowering everyone's classifier scores by 10-20%, and that is what you see happen at the Nationals (if you look) or any other match with the top shooters.

Scott,

that's a lovely concept --- but we're now seeing classifiers that have originated in area matches and Nationals. On quite a few of those the HHF is equivalent to the HHF shot at the match of origination --- not some blend of various shooters high scores averaged together....

See P Pres post, he did his research and saw from previous years where he had to finish to win, in A Class. It has been pretty consistent since I began shooting Nats is consistent even if they are pulling the stages from other places these days.

[Nik Habicht]

All you have to do to see the classifier percentages don't work at a match with the top shooters is to look at the stage scores, only a handful of shooters will shoot above 95% (GM) score with over 30 GM's in attendance, and if you average the top 10% of the scores you will see the pattern continue, that same 10-20% upward skew.

When very few shooters in the class can shoot a score inside their classification percentage, what does that mean to you?

Is it the vast majority of folks who are over classified or a handful that should be in the next class up and who might just be "sandbagging" (or that the classification system doesn't catch up with)?

I'd say --- based on personal experience and looking at a few sets of Nationals results --- that the classification system favors moving people up. It actually appears to be an impediment to sandbagging.....

Based on my Nationals (and hey, Area/Sectional/Other major match) performance against shooters like Dave Sevigny and Rob Leatham, I'm overclassified. I've got a B card, and my high finish was at the 06 Production Nats with 53.11%.....

But then --- strictly comparing the classification system with Nationals finishes is also an apples to oranges comparison. In the one we average the best six scores of the last eight, while disregarding any scores that are duplicates or that would pull us down a class. In the other we're comparing match performance over multiple days --- multiple alternating mornings and afternoons in the last few years. Two very different animals.....

All that said --- the classification works as well as it can as a predictor of match performance, given that it needs to work for all members of USPSA, not just those that shoot big matches....

[Nik Habicht]

You are going off the best score not the average of the top scores of the top GM's to get the 100% to start with (like the classification system does.) If the classification system went off the best ever score, guys like Max would set the bar at least 10-20% higher than it is currently, lowering everyone's classifier scores by 10-20%, and that is what you see happen at the Nationals (if you look) or any other match with the top shooters.

Scott,

that's a lovely concept --- but we're now seeing classifiers that have originated in area matches and Nationals. On quite a few of those the HHF is equivalent to the HHF shot at the match of origination --- not some blend of various shooters high scores averaged together....

See P Pres post, he did his research and saw from previous years where he had to finish to win, in A Class. It has been pretty consistent since I began shooting Nats is consistent even if they are pulling the stages from other places these days.

Scott,

you're saying the system's broken, because someone who wins B-class at Nats gets his A-card shortly thereafter? Someone who just finished the match with a percentage that's right up at the top of the B-range? What else would you expect to happen? Those shooters are on the cusp.....

[Loves2Shoot]

The result was that I made A-class with all the classifiers leading up to the Nats as a side effect of

....get ready for this .... "practice" see me :TY59168

That is the best effect of caring about classification

[Loves2Shoot]

But then --- strictly comparing the classification system with Nationals finishes is also an apples to oranges comparison. In the one we average the best six scores of the last eight, while disregarding any scores that are duplicates or that would pull us down a class. In the other we're comparing match performance over multiple days --- multiple alternating mornings and afternoons in the last few years. Two very different animals.....

Correct.

I'm not saying the system is broke, but as you said big match percentage are not an apples to apples comparison to classifier percentages.

I should clarify that by sandbaggers, I only mean people who the classification does not represent their skill, why is a case by case situational. My first Nats I was C Class practiced hard and was beat hugely my a dude named Mike Seeklander (he might have even been in the top 16.) I quickly realized all the winners below GM were a class or two under-classified.

[JThompson]

But then --- strictly comparing the classification system with Nationals finishes is also an apples to oranges comparison. In the one we average the best six scores of the last eight, while disregarding any scores that are duplicates or that would pull us down a class. In the other we're comparing match performance over multiple days --- multiple alternating mornings and afternoons in the last few years. Two very different animals.....

Correct.

I'm not saying the system is broke, but as you said big match percentage are not an apples to apples comparison to classifier percentages.

I should clarify that by sandbaggers, I only mean people who the classification does not represent their skill, why is a case by case situational. My first Nats I was C Class practiced hard and was beat hugely my a dude named Mike Seeklander (he might have even been in the top 16.) I quickly realized all the winners below GM were a class or two under-classified.

"Say it ain't so Joe" err Scott.

[bbbean]

Numbers don't mean much if you do not understand what they are trying to measure or use the same standards for setting the base line measurement.

I'm sorry you disagree with my conclusion. I'm fairly sure I understand what the numbers mean, and that I've made a valid observation. I'm certainly open to seeing your data and your conclusions.

[JThompson]

I used to get worked up about this stuff, but anymore I just figure it's beyond my control and no amount of bitching, or self torment, is going to change it. Therefore, I say fkit.

Not meaning to dis anyone who has posted either for or against, just been trying to reduce the stuff i fret over to stuff that really matters, like paying the house payment, will my car last another year etc.

Edited December 3, 2009 by JThompson

[Loves2Shoot]

Numbers don't mean much if you do not understand what they are trying to measure or use the same standards for setting the base line measurement.

I'm certainly open to seeing your data and your conclusions.

data here one stage from limited nats. You can use most any stage, I just clicked one at random.

repeat All you have to do to see the classifier percentages don't work at a match with the top shooters is to look at the stage scores, only a handful of shooters will shoot above 95% (GM) score with over 30 GM's in attendance, and if you average the top 10% of the scores you will see the pattern continue, that same 10-20% upward skew.

When very few shooters in the class can shoot a score inside their classification percentage, what does that mean to you?

I've cited my conclusion several times already. They are not comparable percentages.