Evidence based training for IPSC (concept)

[Mr Unnatural]

I want to contribute to the development of more effective ways of deciding what skill to develop for IPSC. There are already lots of great posts about this subject but I want to add my 2 cents.

I would like to combine some of the already great ideas of the people on the forum and others in our sports to develop a more evidence based approach of how to decide what skills need training to do better in matches (not standards).

My idea started with the 'skill assessment worksheet' as presented by saul kirsch and discussed on the forum http://www.brianenos.com/forums/index.php?showtopic=56824&hl=sheet

This worksheet lets you make the assumption of what is the importance of a certain skill and how to grade yourself. Since assumption is the mother of all ****ups, this is not a good thing. The prioritizing function of the worksheet is ok I think (I do not want to start a discussion on HOW to train certain skills).

In golf players are scored with a handicap rating. Since golf is a big sport (money and number of players) it's not surprising that they have collected lots of data on the game and what sets the good players apart from the average players. Dave Pelz is one of the people who has collected lots of data and discovered that a lot of assumptions were not correct (e.g. the discovery of the 'bra pattern', the shortage bible). By collecting data they were able to look for evidence for what sklls a player needs to reach a certain handicap level.

The most important statistics are the number of GIR (greens in regulation), FIR (fairway in regulation), sandsaves, driving distance, number of putts up and downs etcetera. these statistics are comparable to the skills used in the worksheet.

If we could start measuring a standard set of skill (tests/exercises) and brake them up in meaningful parts (not only the total time but also or only the relevant split times). If we could put the results of all shooters in a database we could analyze what skills are needed to win matches (winners that perform 'bad' on certain skills but still win matches are indicators for the 'real' importance of this skill). Not only could we determine what skill are important but also at what level they need to be. If we could collect enough data we could compare shooters within the same class and compare individual skill levels for tips on what skills need the most improvement.

The blog of Jake DiVita has tons of great small skill tests which could be used as does Sauls book on training. The drills on the blog of Jake are challenging and seem to be more comparable to what one could expect in a match situation.

I think we could use the concept used for golf for our sport. For a insightful explanation on the Suunto website you can download the manual of the golfwatch G6 with the appropriate tittle "how not to rely on luck when lowering your handicap". This will give you a better understanding of the concept.

http://suunto.com/en/Products/Outdoor_Sports_Instruments/Suunto-G61/Suunto-G6-Pro/#Manuals

This explains the concept and importance for golfstatistics very well. This concept is also used in the following apps, for golf there are two nice apps that put it all together golfshot en golfplan by shootzoom. This gives a good indication of how a feature app for us coiled look like and what kind of benefits are waiting.

I would like to get some comment to further develop this concept or comment that this is to complex/time consuming to research.

I hope to get some input to further develop the idea and keep it simple and realistic, so that it could be developed opens source for or by the community at low cost.

Rick

PS I hope this is the correct category for this posting.

[GrumpyOne]

All that sounds like a great idea! You do all the leg work, and we'll reap the benefits! (Just kidding 'bout the last part!). It does sound interesting, and possibly quite helpful.

[eric nielsen]

If this was facebook I would hit the "like" button [thumbs up].

[rlv65]

Pat Harrison posted a discussion in Shooting Forums > Training Techniques (6/13/09) regarding "what to work on" (can't figure out how to link).

This is a great discussion that might tie into what you are thinking about.

[Mr Unnatural]

Hi rlv65

I have found the post http://www.brianenos.com/forums/index.php?showtopic=86718 (this big picture 'implies' that matches are won between targets and not on the target).

Pat has a good point in his post, using his insight I have come up with the following list/approach.

At this point I see 3 options for determining (data collection) what is important.

1 measuring real match stages

2 measuring a 'postal match'

3 measuring predefined drills

This would be my preference since we measure what matters at a match, under match conditions.

Things to measure:

match level

shooter level

class

major/minor

score on each target

splits between shots

penalties

total time

stage layout

...

For my job I use software to determin what 'aspect' matter en what 'aspects' don't. I would be able to analyse large databases of match information to discover 'evidience' of the importance of certain skills. My profesional experience in analysing maintenance data teaches me that most assumptions on the effectiveness of maintenanc are wrong and that only with a proper quantitative analasys of data we can make a sound qualitatively analasys.

Maybe the data that is needed is allready collected some time ago for another purpose. Lack of data and lack of a sound plan are the main obstacles to test this concept.

GrumpyOne

is right, if data collection is needed there is a lot of 'legwork' needs to be done.

Rick

[rlv65]

Would the data from a "Nationals" level match be a starting point?

From your list of Things to measure:

match level - high, most performing near their peak

shooter level - (or class - GM, M, A, etc.?) all levels including world's best to beginners

class - (assuming division - Open, Production, Limited, etc.) due to current USPSA format, only 2 classes shoot at a time. If comparing only within a division (i.e - analyzing production vs production only), this may not be a problem.

major/minor - both

score on each target - may not get the exact score for T1, etc., but can get #A's, #B's, etc. per stage.

splits between shots - this is problematic. Transitions (which wasn't in your list) also presents the same problem. May have to use some type of average (total time / total shots).

penalties - available.

total time - available

stage layout - available.

Sounds like an interesting project. Even though I suck at golf, I found the link and discussion of the golf statistics interesting.

[Mr Unnatural]

Hi rlv65

I think this would be a great starting point. Either it will produce the insight we're looking for or it will give us the information needed to develop the project further.

The splits are necessary I think (I concluede that if we know all splits and the layout we would know the transitions as well), I (may not be correct but) think that transitions are crucial to determin what sets the masters apart from the mere mortals among us. It is important to know/compare how different shooters asses the necessary trade-off between speed, precision and 'safe scoring'/safety. I think that good shooters asses thes trade-offs differntly by experience, ability and confidence in their skill.

While analysing the data I would like to make a selection of experienced match schooters (someone who's shooting there first matches is greatly (hypothesis) influenced by match stress).

Rlv65 I assume you have acces to this data?

Cheap and easy way for collecting data is to try to find a university to help 'study' this subject and involve them in the research and data collection.

I'm not familiar with rules and guidelines of RO, maybe it's possible to equip every RO with to timers and alternate between them so that 'this student' notes the readouts 'splits' of the shooter after official scoring by the RO.

I'll work on a research mindmap (in the next weeks) and post try to get is posted here so everyone can contribute to the development of it.

Rick

[rlv65]

I agree with your comments on the need for the split and transition times.

I don't have access to the Nationals data other than what is posted on the USPSA website under Match Results (left side) / Major Match Results / 2009 / 2009 S W USPSA Ltd Ltd10 Rev Nationals & 2009 S W USPSA Open Prod Nationals. If you choose final results and click on a shooter number, it will give a breakdown by stage.

I don't know if USPSA would retain the data that generated these results or what their policy for getting it may be.

[Graham Smith]

Just to add a different perspective to this - in some ways, the skills training needs to be scaled to the level of the shooter. For example, for most D shooters, the most important thing may simply be don't miss the target. Mikes and No-Shoots will kill a score faster than anything so keeping the bullet on the target, anywhere on the target, is often the first goal. And the fact is that this involves a whole lot of basic skills ranging from trigger control to speed control which continue to be important throughout.

For that reason, skills development is often based on some kind of pyramid - a few basics to start with then at each new level, more skills are layered on. For example, you may start with feet firmly planted in a good stance shooting at at one target and end with shooting as you move from position to position engaging multiple, between those two things are a myriad of skills that you must gain proficiency in.

It's like building a house - you do it from the bottom up, one brick at a time. Skip a brick and you end up with a problem. The ultimate goal may be to build the house, but each brick becomes an intermediary goal that must also be accomplished.

FWIW, I know a trainer who has taken a set of basic shooting drills that all the advanced students go through and has been recording the results. This gives him a baseline so he can say to a class, "Your peers have shot this drill in x seconds, let's see how well you do in comparison." But while this gives a good overall comparison, it still comes down to evaluating the details of each student as they shoot to see what they need to work on with the goal of making them a better shooter - not better than the next guy but better than when they arrived.

[Mr Unnatural]

Graham

Intended or not but your comment underlines the benefit/importance of this knowledge. What you are referring to is the so called 'theory of constraints' (I recommend reading The Goal by E. Goldratt), this theory states that your performance/progression is hinderded by constraints. In other words when you are not able to hit a target at a normal pace, don't bother training e.g. speed shoots.

By knowing what skills at your level are considered constraints you what to focus on first to develop a foundation to progres on without hitting 'a plateau' become dissapointed and stop participating in the sport.

As a comparison I will ad this example from the golf.

Most people that start playing golf spend hours on the driving range hitting balls and trying desperately to hit it further then last week. Even if they succeed in hitting long distances their scores will not improve much. Being able to hit over 250 yards with a driver as a beginner will not improve your score by much.

This is because most starting golfers need about 120-130 strokes to compleet a course (72 strokes) 40 to 50 strokes at this level is putting. 80% of all golf shots are within 100 yards of the green. If you want to improve spend time on improving your shortgame skills and putting. Hitting it straight and consistent is the only goal when teeing off as a beginner (eg hitting the target in the a-zone).

Most beginners overrate their skills, when they hit a ball in the rough (spelling??) they still think they can hit is straight to the target, because a few months ago this miracle shot was a succes. What happens is they hit it even further in the rough costing several strokes over playing it save to the fairway or taking a penalty shot.

For beginners it is important to get a good sence on what skills are important to get a better result at matches and what level their skill rate is at this moment to train effectively.

Rick

[Mr Unnatural]

Finally I have found some time for continuing this post.

I've been busy finishing a book I'm writing (Implementing Performance Based Real Estate Mainanance Concepts, it's now being edited) and finishing a Master thesis.

I have allmost finished the research set-up. The bad news is this going to be time consuming project because of the data collection.

I will need detailled info on the stages, score per target and time (splits also). Only then I will be able to produce something that will make sence and is of value to the shooting community.

Are club will host a match in november and I plan to test my project on one stage (the match is a week after the a.t.a. (birth) of our second son, if all goes well I can be at the match). I will ask the RO for coorperation as I will need his help for the scorring. For time I plan to use a ced7000 in silent mode to record the shooters. I will have write down all splits in excel or something and match this with the shooters score. I plan to divide the stages into short sections eg: from last shot (T2) box b to first shot (T3) box c (transition time). I will be making more sections on each stage. I can also make section last shot box b to last shot (T5) this would give transition time and target acquisition time.....

For a larger scale data collection I want a timer that I can hook up to a computer. The more matches are analysed the more 'evidence based' the outcome will be. I don't know if it's easy to couple the time data to the scoring data, I have no recent experience with scoring software used at matches.

The results

the results will give you a 'hitfactor' for each stage section see above. These hitfactors will be coupled to the class the shooter is ranked. This will give an average hitfactor per class on a section. The more data there will be on different stages and shooters (let's go international) we will know the standard deviation on this hitfactor.

To give everybody an idea of the things we could find out:

1. we will know what technical skills are needed in matches, the more match stages are analysed the better we will know how often eg a prone position is needed and in range the distance to the targets will be. (Depending on the data this could be further specified to the level of the match eg level 3 versus level 2 matches)

2. You would be able after measuring your match results to compare them to the results of shooters in your class and find out your strong and weakpoints.

3. Because of the knowledge of the frequency of certain skills needed (1) we could establish a training priority for practice.

4. we will find out what skills set the GM's apart from us mere mortals.

5 this could be the basis for a objective ranking system because it's on actual match performance not on classifiers you can train yourself on.

6 better insight in the scoring game of IPSC, improved stage strategy

7 if there are any breakthrough skills that are 'needed' to reach a certain level of performance.

8 and a lot more.

I have a draft of the research set up in VUE. http://vue.tufts.edu/index.cfm this is:

The Visual Understanding Environment (VUE) is an Open Source project based at Tufts University. The VUE project is focused on creating flexible tools for managing and integrating digital resources in support of teaching, learning and research. VUE provides a flexible visual environment for structuring, presenting, and sharing digital information.

PM me for the draft in vue (i will clean it up first) and feel free to comment or ad your two cents.

I will attache a pdf version of it. It contains my visual map for understanding practical shooting. I did not think of all this I merely connected some dots and I have not credited all my sources because I made this for private use only.

Thinking out this idea has given me a greater understanding of what it takes to perform well in match conditions. I will complete the research set-up and will share it here but I do not plan to do all of the research (data collection).

kind regards,

Rick

“ It is hard to explore what happens when people behave with a purpose without becoming

curious, even concerned, about how well or how badly the outcome serves the purpose.”

T.C. Shelling, 1978:19

Practical shooting research.pdf

[Jake Di Vita]

I'm interested.

Let me know if I can help.

[Chills1994]

just one little tip...an FYI..go to the nationals results (limited, or open, or production)...compare the minimum round count for the entire match to the total raw stages's times for the top five competitors in that division...see what you get.

[Stefano]

Rick, your idea is great !

In my opinion it would be useful for you to contact Mike Seeklander at USSA:

Mike uses a tool called "skilltest" to gauge the ability and skill-level of a shooter, comparing the results (points & times) of a particular shooter with a database he has implemented with the infos coming from shooter of different classee and divisions, from the rookie d-class shooter, to the Top-GM experienced shooter.

The skilltest is based on a list of very simple and "easy-to-do" exercises (not standards): this is just to work on a objective-basis in collecting data ...

[Chills1994]

at first I was thinking that as far as statistical analysis goes, I was thinking that one could do an Analysis of Variation (ANOVA) between the classes at a major match on their cumulative time for the match.

but.... then I started thinking ....maybe do multiple correlations between the classes of say cumulative time vs. cumulative hit factor. plot CHF on the X axis for example and cumulative time on the Y axis. have a seperate graph for GM's, M's, A's, B's, C's, and D's .

I am just thinking out loud...rambling really...I am thinking that there isn"t going to be just one easily quantifible or easily define-able skill that makes or breaks a USPSA shooter.

[Mr Unnatural]

Hi

It's been a while since I posted in my tread here. I would like to thank you for your interest and suggestions in this thread.

The good news, I solved all of the problems, obstacles!

I have solved the problems I had with setting up the research and how to analyze and present the results and give feedback to the shooter about his performance.

The solution for the data collection is solved by the new CED timer ,as announced by Charles Hardy on personal Armament podcast (ability to download to a pc, to be released in 2011).

Note

imagine they include the spy mode of the CED 7000 and you can spy on your squad buddies or even the super squad and analyze and compare there results to yours.

As a source for the data one could organize a postal match on the forum. This way all forum members could contribute and receive an analysis. This would give the needed match performance. It could be easier but less relevant if training stages are used and shooters hand in there training results.

For relevant stage design one would have to analyze USPSA match design and note what all the elements. This could be used to determine how often certain positions are needed in a match and at what distance targets are to be expected. For example what distances are targets at from a prone position.

If you know how often prone positions occur in a match and you what the typical distances and targets are you can compare this ‘skill’ to other skill’s.

Since all matches are sanctioned by USPSA or IPSC the organization there has be some kind of library with stage designs. One could distinct between the level of sanctioned matches. It could be that level 3/4 matches ask different skills to be more developed then in level 1 matches.

With the information in the database you can compare you’re performance to your peers that are ranked similarly or with a GM. Only this way you can find out what training priority a certain skill has for you. It makes no sense to develop your skills unevenly or not according to the skills needed in matches.

I have made some test stages analysis and these work, one can clearly see where to focus on in training to get a better result in match scoring. You still have to figure out how to train certain skills for yourself off course.

The easiest way to start this would be to select a number of short stages (eg the ones on Jake DiVita’s website) make sure that the stages are similar to match stage design (just as hard).

Then start to write down your own score and times (including splits) and the ones for your training partners and start comparing them. PM for a simple excel example.

The bad news, I don't have the resources for this project to make this available for every one (time and money). So I will put the project on hold or if someone would like to make this his project feel free to send a PM and I will try to help you.

I will use it for personal purpose and compare and analyze my skill level with my training partners. This will be sufficient for me in the next two years.

Kind regards,

Rick

[owen]

This sounds like the sort of thing the AMU might be interested in funding...

[Steven Cline]

I haven't read all of the posts, but a local GM and I were talking about what he observed from scoring his local club that the really good shooters have lots of A hits and few Mikes or No-Shoots.

I think I read something along these lines recently and I know that Brian Enos himself has commented on the importance of accuracy and percentage of points being high.

So... like hitting greens and fairways in regulation, it seems as if hitting targets... uh, in regulation... is a pretty good indicator of shooting skill.

And not messing up reloads or missing steel...

OK, seriously, I got nothing at the moment.

Edited March 4, 2011 by Steven Cline

[bofe954]

The only thing I could see that could confound your data is that people that win, win by playing their game. Knowing how to maximize their score on a stage (or a golf course) using what they have.

Obviously someone with C skills will never beat a GM, but somehow out of people with similar skill sets certain people seem to win a lot more than others.

Somehow TGO wins at SS nats all the time, and there are GM's there that should be able to produce data a lot like his.

The data would be awesome to look at though.

[Joe4d]

I think you could end up with alot of data with no way to compare it. There are just two many variables, different match directors build different stages, no two stages are the same, no two stages require the same skills, no two shooters are the same, Different shooters will shoot a stage diffently to get the best score for them, For instance I can run for squat, but I shoot a 6" gun and am very accurate, I will score higher if I take the long shot than if I try to run up to the target. Another shooter may not. I have won my class at majors helped by a few stages where the match director liked long shots, but that doesnt mean the ability to run quickly isnt important at a match somewhere else.