Classifier HHF Determination

[Chills1994]

Average of the highest, eh?

So that means at a minimum two scores were added together and divided by two. As far as we know it could have been two scores shot by the same person.

Or was it 5, or 10, or 15 people?

I know there are a few folks out there who have the keys to the database, and can easily search for these things. And they are eye searingly absent from this thread. IIRC, they've been absent from other threads like this one too.

If it really was an average of the highest, then that doesn't explain why at first Single Stack and L-10 HHF's were the same.

http://www.brianenos.com/forums/index.php?showtopic=75558&hl=+same&page=1

[JakeMartens]

Val Reule
Club/Section Support & Affiliation, Activity Credits, Address Changes, Classification Coordinator
Ext. 103
val@uspsa.org

Val holds the keys

[Chills1994]

go to a major match, heck go to several major matches, where you place in your division, that's your "rank"

and for me and most of the folks I've traveled with, that rank has been very close to our current classification. That tells me that regardless of how the HHF is derived, it's at least approximately accurate.

Sure, there are some people that are already shooting at the next higher classification, but haven't had enough classifiers to get bumped. Sure, there are people who are no longer training and practicing as seriously as the year they got their A or M or GM card. But for most active competitors, the system seems to work fine.

I guess you missed my other post.

I'll rephrase it this way:

How many GM's are there per each division?

How many GM's show up and compete in any division at Nat's?

What percentage of overall GM's in that division do they represent?

If you know you're a grandbagger, why blow $200 on match fees, $300 plus on a hotel room, meals, and gasoline only to have your butt handed to you?

In short, any major match results over-sample the sandbaggers.

Edited November 19, 2013 by Chills1994

[Chills1994]

Val Reule[/size]Club/Section Support & Affiliation, Activity Credits, Address Changes, Classification Coordinator[/size]Ext. 103[/size]val@uspsa.org[/size]

Val holds the keys[/size]

Nope, not even close. There are other people on this forum who have all sorts of access to the database. One night when I was away from home and away from any of my issues of Front Sight magazine, I was able to PM somebody here and they were able to PM me back with my PIN number so I could log into USPSA.org .

[JakeMartens]

Val Reule[/size]Club/Section Support & Affiliation, Activity Credits, Address Changes, Classification Coordinator[/size]Ext. 103[/size]val@uspsa.org[/size]

Val holds the keys[/size]

Nope, not even close. There are other people on this forum who have all sorts of access to the database. One night when I was away from home and away from any of my issues of Front Sight magazine, I was able to PM somebody here and they were able to PM me back with my PIN number so I could log into USPSA.org .

Then cut and paste your email to your AD with their response on how it works

Edited November 19, 2013 by JakeMartens

[JakeMartens]

Why are they secret? And how are they deterred? Too much to ask? I believe they are applied as fair as possible....but where do they come from?

Great questions.

AD5 as already posted

[motosapiens]

I guess you missed my other post.

I'll rephrase it this way:

How many GM's are there per each division?

How many GM's show up and compete in any division at Nat's?

What percentage of overall GM's in that division do they represent?

If you know you're a grandbagger, why blow $200 on match fees, $300 plus on a hotel room, meals, and gasoline only to have your butt handed to you?

In short, any major match results over-sample the sandbaggers.

I saw your post, but I guess I didn't understand your point. Why would I care if some GM's get beat? Maybe they just enjoy shooting. Maybe the people you call 'grandbaggers' just aren't as serious about the sport as they were when they got their gm card, but they still like to go to local matches.

Does it bother you that some people have a classification that doesn't always match up with their match performance? Does it create some kind of problem for the sport? If so, can you explain what that problem is, and how to fix it? I don't reall see it as a problem, nor do I see what bearing it has on the huge controversy over HHF's (lol).

Edited November 19, 2013 by motosapiens

[Chills1994]

You're basically paying a per capita tax to the headquarters of an organization which then ranks you against some invisible metric that they are not forthcoming about how they actually derived or "divined" that metric???

You don't see a problem with that?

Here, let's go down to the football field and I'll get you to try to kick some field goals for me. You pay me a $1.50 per kick, and when the ball almost makes it over the crossbar I'll move the field goal further back, when you're not looking. Wanna try again? Okay, that'll be another buck fitty.

Even as a long jumper in high school I knew that Bob Beamon set the record at the 1968 Olympics with a jump of 29 feet two and a half inches. And that was way before the internet.

[Chills1994]

Val Reule[/size]Club/Section Support & Affiliation, Activity Credits, Address Changes, Classification Coordinator[/size]Ext. 103[/size]val@uspsa.org[/size]

Val holds the keys[/size]

Nope, not even close. There are other people on this forum who have all sorts of access to the database. One night when I was away from home and away from any of my issues of Front Sight magazine, I was able to PM somebody here and they were able to PM me back with my PIN number so I could log into USPSA.org .

Then cut and paste your email to your AD with their response on how it works

Val Reule[/size]Club/Section Support & Affiliation, Activity Credits, Address Changes, Classification Coordinator[/size]Ext. 103[/size]val@uspsa.org[/size]

Val holds the keys[/size]

Nope, not even close. There are other people on this forum who have all sorts of access to the database. One night when I was away from home and away from any of my issues of Front Sight magazine, I was able to PM somebody here and they were able to PM me back with my PIN number so I could log into USPSA.org .

Then cut and paste your email to your AD with their response on how it works

I let my USPSA membership lapse for several reasons.

"My AD" has known about my beef with the classifier system for a while now. And the AD before him when I specifically asked for a classifier records page to be brought up to the next BoD meeting. My suggestion never did appear in the BoD meeting minutes.

[Chills1994]

Here is a thread from 2008 where I posted my suggestion for a classifier records page. I even put it in bold :

http://www.brianenos.com/forums/index.php?showtopic=74817&page=2&hl=%2Brecords#entry868649

[Flexmoney]

Val Reule[/size]Club/Section Support & Affiliation, Activity Credits, Address Changes, Classification Coordinator[/size]Ext. 103[/size]val@uspsa.org[/size]

Val holds the keys[/size]

Nope, not even close. There are other people on this forum who have all sorts of access to the database. One night when I was away from home and away from any of my issues of Front Sight magazine, I was able to PM somebody here and they were able to PM me back with my PIN number so I could log into USPSA.org .

Really? Who? (please send that to me in a PM)

[motosapiens]

You're basically paying a per capita tax to the headquarters of an organization which then ranks you against some invisible metric that they are not forthcoming about how they actually derived or "divined" that metric???

You don't see a problem with that?

Here, let's go down to the football field and I'll get you to try to kick some field goals for me. You pay me a $1.50 per kick, and when the ball almost makes it over the crossbar I'll move the field goal further back, when you're not looking. Wanna try again? Okay, that'll be another buck fitty.

Even as a long jumper in high school I knew that Bob Beamon set the record at the 1968 Olympics with a jump of 29 feet two and a half inches. And that was way before the internet.

No, I don't really see a problem with that (but if you can actually explain the problem, maybe I'll see) because the 'rank' doesn't really matter. You don't get a prize or money or chicks or guaranteed entry to an area match because of it. IMHO, the classification system serves 3 purposes (none of which are that big a deal compared to match results);

first, it allows people to sort of keep track of their improvement (insofar as the classification represents general shooting skill).

Second, it allows you to roughly gauge your performance against people you don't know. I don't really care about the class of the local guys I shoot against locally because I know how good they are, but if I travel somewhere else, seeing the shooter's classes in the results gives me a better idea of how I performed that day.

Last (and least important imho), it allows MD's to hand out self-esteem awards to people who would probably never get trophies otherwise, and some people are motivated by trophies and trinkets rather than intrinsic performance.

So while I see that there could be some miniscule benefits to tweaking the system, I don't really see why some folks are getting so wrapped around the axle about it instead of practicing shooting or designing stages or putting on matches or running for section coordinator or otherwise behaving more constructively.

The $3/match activity fee seems pretty cheap to me to have someone else organize an entire sport with rules, championships, classification, etc.... Seems like a better system than some other sports where you have to join an organization to even participate once a year. This way the people who shoot the most matches pay the most to support the organization they benefit from. Makes sense to me.

So help me understand, why should someone care how the HHF is determined as long as it appears to be roughly accurate? What competitive unfairness is occurring because of the current system? What bad consequences is it currrently having for the sport, or for individuals? Why is it important?

Edited November 19, 2013 by motosapiens

[beltjones]

I don't have a problem with multiple processes to determine a HHF. To me, it makes sense to use the HHF from Nationals for new classifiers (provided they were actually shot as-constructed at Nationals, which isn't the case with some of the 13 classifiers).

It also makes sense to periodically update the HHFs using relevant data obtained from actual shooter performance.

What bothers me is I don't think they're using statistically valid techniques. They really shouldn't be using the mean of the top performances - it should be the median. They also should be forthcoming about what the top performances have been. If they take the median of the top "n" performances on a given classifier, why not publish those top "n" performances and update them every few years?

The systems seems to work reasonably well, but there are some classifiers that, from a HHF perspective, are just garbage in some divisions. For example, if you show up with your production rig to your local match and see that "Can you Count" is set up, you just know that it's not going to help your overall classification percentage. That's a failing on the part of HQ, in my opinion.

[beltjones]

You're basically paying a per capita tax to the headquarters of an organization which then ranks you against some invisible metric that they are not forthcoming about how they actually derived or "divined" that metric???

You don't see a problem with that?

Here, let's go down to the football field and I'll get you to try to kick some field goals for me. You pay me a $1.50 per kick, and when the ball almost makes it over the crossbar I'll move the field goal further back, when you're not looking. Wanna try again? Okay, that'll be another buck fitty.

Even as a long jumper in high school I knew that Bob Beamon set the record at the 1968 Olympics with a jump of 29 feet two and a half inches. And that was way before the internet.

No, I don't really see a problem with that (but if you can actually explain the problem, maybe I'll see) because the 'rank' doesn't really matter. You don't get a prize or money or chicks or guaranteed entry to an area match because of it. IMHO, the classification system serves 3 purposes (none of which are that big a deal compared to match results);

first, it allows people to sort of keep track of their improvement (insofar as the classification represents general shooting skill).

Second, it allows you to roughly gauge your performance against people you don't know. I don't really care about the class of the local guys I shoot against locally because I know how good they are, but if I travel somewhere else, seeing the shooter's classes in the results gives me a better idea of how I performed that day.

Last (and least important imho), it allows MD's to hand out self-esteem awards to people who would probably never get trophies otherwise, and some people are motivated by trophies and trinkets rather than intrinsic performance.

So while I see that there could be some miniscule benefits to tweaking the system, I don't really see why some folks are getting so wrapped around the axle about it instead of practicing shooting or designing stages or putting on matches or running for section coordinator or otherwise behaving more constructively.

The $3/match activity fee seems pretty cheap to me to have someone else organize an entire sport with rules, championships, classification, etc.... Seems like a better system than some other sports where you have to join an organization to even participate once a year. This way the people who shoot the most matches pay the most to support the organization they benefit from. Makes sense to me.

So help me understand, why should someone care how the HHF is determined as long as it appears to be roughly accurate? What competitive unfairness is occurring because of the current system? What bad consequences is it currrently having for the sport, or for individuals? Why is it important?

You're begging the question when you say "it appears to be roughly accurate." In many cases it's not accurate at all, hence the request by some for more transparency. Why don't you get that?

[Flexmoney]

We have an in-person USPSA Board of Directors (BOD) meeting in December. This is on the agenda. Well, the timing of classification updates is on the agenda, but I'll be bringing this up as well...for sure.

------> So, if you have thoughts on this, contact your Area Directors. Do it directly. Don't count on them reading this thread. Realize the BOD does have lots to do, and little time to do it in at these in-person meetings. Any one BOD member can only tilt at so many windmills at once.

[motosapiens]

You're basically paying a per capita tax to the headquarters of an organization which then ranks you against some invisible metric that they are not forthcoming about how they actually derived or "divined" that metric???

You don't see a problem with that?

Here, let's go down to the football field and I'll get you to try to kick some field goals for me. You pay me a $1.50 per kick, and when the ball almost makes it over the crossbar I'll move the field goal further back, when you're not looking. Wanna try again? Okay, that'll be another buck fitty.

Even as a long jumper in high school I knew that Bob Beamon set the record at the 1968 Olympics with a jump of 29 feet two and a half inches. And that was way before the internet.

No, I don't really see a problem with that (but if you can actually explain the problem, maybe I'll see) because the 'rank' doesn't really matter. You don't get a prize or money or chicks or guaranteed entry to an area match because of it. IMHO, the classification system serves 3 purposes (none of which are that big a deal compared to match results);

first, it allows people to sort of keep track of their improvement (insofar as the classification represents general shooting skill).

Second, it allows you to roughly gauge your performance against people you don't know. I don't really care about the class of the local guys I shoot against locally because I know how good they are, but if I travel somewhere else, seeing the shooter's classes in the results gives me a better idea of how I performed that day.

Last (and least important imho), it allows MD's to hand out self-esteem awards to people who would probably never get trophies otherwise, and some people are motivated by trophies and trinkets rather than intrinsic performance.

So while I see that there could be some miniscule benefits to tweaking the system, I don't really see why some folks are getting so wrapped around the axle about it instead of practicing shooting or designing stages or putting on matches or running for section coordinator or otherwise behaving more constructively.

The $3/match activity fee seems pretty cheap to me to have someone else organize an entire sport with rules, championships, classification, etc.... Seems like a better system than some other sports where you have to join an organization to even participate once a year. This way the people who shoot the most matches pay the most to support the organization they benefit from. Makes sense to me.

So help me understand, why should someone care how the HHF is determined as long as it appears to be roughly accurate? What competitive unfairness is occurring because of the current system? What bad consequences is it currrently having for the sport, or for individuals? Why is it important?

You're begging the question when you say "it appears to be roughly accurate." In many cases it's not accurate at all, hence the request by some for more transparency. Why don't you get that?

Because except for 'can you count', I haven't seen many example of where it's not accurate (but I'm interested, if you'll post some). I personally think the problem with 'can you count' is that it's just not hard enough to provide reasonable separation between levels of shooters. It doesn't *seem* to me that it's the hhf causing the problem, but the lack of actual aiming required. (btw, if you're a B or C that practices draws and bill drills/trigger speed, that classifier will probably help your classification).

I think it would be reasonable to look at how representative classifiers are, and get rid of the ones that are not doing a good job. I think that has less to do with HHF than with the distribution of scores in generals, i.e., are most people shooting about what they're shooting on other classifiers and in matches most of the time?

So if there are classifiers that suck, lets focus on them, and not muddy water how HHF is calculated.

[beltjones]

You're basically paying a per capita tax to the headquarters of an organization which then ranks you against some invisible metric that they are not forthcoming about how they actually derived or "divined" that metric???

You don't see a problem with that?

Here, let's go down to the football field and I'll get you to try to kick some field goals for me. You pay me a $1.50 per kick, and when the ball almost makes it over the crossbar I'll move the field goal further back, when you're not looking. Wanna try again? Okay, that'll be another buck fitty.

Even as a long jumper in high school I knew that Bob Beamon set the record at the 1968 Olympics with a jump of 29 feet two and a half inches. And that was way before the internet.

No, I don't really see a problem with that (but if you can actually explain the problem, maybe I'll see) because the 'rank' doesn't really matter. You don't get a prize or money or chicks or guaranteed entry to an area match because of it. IMHO, the classification system serves 3 purposes (none of which are that big a deal compared to match results);

first, it allows people to sort of keep track of their improvement (insofar as the classification represents general shooting skill).

Second, it allows you to roughly gauge your performance against people you don't know. I don't really care about the class of the local guys I shoot against locally because I know how good they are, but if I travel somewhere else, seeing the shooter's classes in the results gives me a better idea of how I performed that day.

Last (and least important imho), it allows MD's to hand out self-esteem awards to people who would probably never get trophies otherwise, and some people are motivated by trophies and trinkets rather than intrinsic performance.

So while I see that there could be some miniscule benefits to tweaking the system, I don't really see why some folks are getting so wrapped around the axle about it instead of practicing shooting or designing stages or putting on matches or running for section coordinator or otherwise behaving more constructively.

The $3/match activity fee seems pretty cheap to me to have someone else organize an entire sport with rules, championships, classification, etc.... Seems like a better system than some other sports where you have to join an organization to even participate once a year. This way the people who shoot the most matches pay the most to support the organization they benefit from. Makes sense to me.

So help me understand, why should someone care how the HHF is determined as long as it appears to be roughly accurate? What competitive unfairness is occurring because of the current system? What bad consequences is it currrently having for the sport, or for individuals? Why is it important?

You're begging the question when you say "it appears to be roughly accurate." In many cases it's not accurate at all, hence the request by some for more transparency. Why don't you get that?

Because except for 'can you count', I haven't seen many example of where it's not accurate (but I'm interested, if you'll post some). I personally think the problem with 'can you count' is that it's just not hard enough to provide reasonable separation between levels of shooters. It doesn't *seem* to me that it's the hhf causing the problem, but the lack of actual aiming required. (btw, if you're a B or C that practices draws and bill drills/trigger speed, that classifier will probably help your classification).

I think it would be reasonable to look at how representative classifiers are, and get rid of the ones that are not doing a good job. I think that has less to do with HHF than with the distribution of scores in generals, i.e., are most people shooting about what they're shooting on other classifiers and in matches most of the time?

So if there are classifiers that suck, lets focus on them, and not muddy water how HHF is calculated.

That's fair. However, I would argue that the first step to determining which classifiers suck is to see what the true HHF is/should be, and then determining why it doesn't accurately gauge individual performance.

There is another issue related to the above, and that is, how do they determine what an "A" class performance is on a classifier? It's one thing to assign a HHF, it's another thing entirely to figure out how the other scores apply to each sub-class. All of the scores for each classifier should be a bell curve of a sort, but each classifier should have it's own bell curve, if the statistics are being used correctly.

The above may sound complicated, but it's a simple matter of data management. If USPSA has the data, then determining all of the above is about a 5 minute exercise per classifier, at most.

[motosapiens]

That's fair. However, I would argue that the first step to determining which classifiers suck is to see what the true HHF is/should be, and then determining why it doesn't accurately gauge individual performance.

There is another issue related to the above, and that is, how do they determine what an "A" class performance is on a classifier? It's one thing to assign a HHF, it's another thing entirely to figure out how the other scores apply to each sub-class. All of the scores for each classifier should be a bell curve of a sort, but each classifier should have it's own bell curve, if the statistics are being used correctly.

The above may sound complicated, but it's a simple matter of data management. If USPSA has the data, then determining all of the above is about a 5 minute exercise per classifier, at most.

The bolded is a very interesting and attractive idea. I think the HHF is by far the less important issue. Using a curve and more advanced statistical techniques (rather than the arbitrary levels in current use) would make the whole idea of an HHF unnecessary. Some classifiers may have a squishier curve and some may have a broader curve. I vote for beltjones to chair the classification system redesign committee.

[Chills1994]

That is called kurtosis. There are two technical terms for when a distribution is skewed left or skewed right, but they escape me now. That is why in some cases it is better to calculate a median instead of an average/mean.

You could make the split between A and B class the average or median HF out of all the classifier scores submitted for that stage.

You'd have the lower 3 classes, D, C, and B, to the left of the centerline of the bell curve. The A's, M's, and GM's would be to the right of the centerline. You could set it out so that the people falling between the mean and the first plus one standard deviation would then be A class. The folks between the first and second standard deviation would be Masters. The HF's that are farther than plus two SD's would be the GM's.

But, with the lowest hit factor submitted being a zero and the highest maybe being a 15, there might not be enough range/resolution/spread to adequately separate each class.

It would be a ton of work too.

[motosapiens]

It would be a ton of work too.

Work that could be done by a computer.

[Chills1994]

Yeah, once you have the code written correctly.

[whitedog]

I am fairly consistent shooting in my class at any match I shoot. My "average" on the result page is about 52%. My match "average" over a 3 year period is about 54%. (Yup,actually sat down with paper and pencil to do it). My point is I believe the system does gauge ability,to a point.

It's how they come up with it and why it's secret I don't get. Does any one have any real ideas or reason why it's so secret? The whole process and such?

[kneelingatlas]

There have been all kinds of workable ideas how the classifier system could work discussed in this thread, but the OP asked how it does work; a question which has been answered, but none of those answers seem consistent with the results which we can all see.

Others have commented that it doesn't matter, and that a big deal is being made of nothing, or that the means don't matter to them so long as they're happy with the ends, which is all well and good, but I believe some of us, myself included, are curious how the HHFs are determined?

The more distraction from the question and derision towards those asking, the more curious I become.

[Flexmoney]

It doesn't need to be complicated.

It is a High Hit Factor. The idea being to measure from the top, not to measure from the average.

- Of course, a classifier could be setup wrong. A shooter could jump the timer and not get called on it, etc... These things could skew the results if a single run (outlier) was used as the sole HHF. So, it would make sense to maintain the HHF by averaging a number of top runs.

- Different from maintaining the HHF, establishing the HHF could be done a few different ways. (but, Open runs shouldn't equal Production runs, as we saw when "Can You Count" came into being)

I don't see why the system can't be governed by math and ran with greater automation. But, I am not in the home office and I have not been familiarized with the functional/operational capabilities involved in the task.

I do know that the home office, under our new Executive Director, is looking at improving various aspects of the operation. Maybe this fits in? Not sure of it's priority status. Will ask.

[sperman]

Flex,

One thing to consider on this project and many others. The USPSA membership has expertise in many different areas. The home office doesn't have to do it all. Volunteers don't have to be limited to setting up stages and holding the timer. I bet if they asked for help they would be surprised at both the quantity and quality of volunteers.