Ugh, Timers

[shred]

TLDR: if you want to be fair, use the exact same timer for every shooter on a stage because they do differ one to the next, even units of the same make.

 

After some back and forth with @euxx about timers and splits, I rounded up a handful and did some testing.

Test process is:

 

Start them all one after another, then fire 2 shots with several (16 in this case) seconds in between them (for future testing, I'll try for splits under 10 seconds to keep the last decimal place displayed on the SG Go)

 

Compare the timers and also subtract the end-time from the start-time to see if the total matches the split (it did in all cases)

 

 

Ok, those two agree... so on to the next...

 

 

 

Uh oh, well, could be rounding got the better of it...

 

 

Yikes!  That's more than rounding error.   This unit was not consistently different than the others by that much, but it did show more variance.

 

I don't have pics handy, but I then compared both of the CEDs to each other and saw .01 difference between the two pretty consistently (#4 was faster than #2).   

 

I'll have to borrow an AMG or two next.

 

Now you may think .03 isn't that big a deal... but just one or two of those could change the winner of a Nationals.

 

[dannyd]

The error would definitely be there mainly because each manufacturer would calibrate their device a little differently.

[shred]

Calibration shouldn't be that much a thing although it likely is.  Even a cheap crystal oscillator should be good for +/- .002 seconds per minute, but even with same mfg devices, I saw variations, and with real-world changes in distance to the muzzle, different muzzle blasts, etc things will vary even more, so best-practice is still to use the exact same device for every shooter on any given stage so somebody doesn't get +.03'ed by accident.

 

What that means for fixed times like steel challenge stage records is a different thing.

 

Here's the 2 CED's disagreeing with each other (to be fair this could be rounding based on exactly when the clocks started-- FWIW, SG Go also said 0.90.)

 

 

 

[xrayfk05]

2 hours ago, shred said:

Now you may think .03 isn't that big a deal... but just one or two of those could change the winner of a Nationals.

 

I don't believe that for a second, certainly not on majors. The more stages in a match, the more it evens out. 

On nationals every shooter will have give or take 20 stages with +- 0.03 difference, in the end it's a wash.

 

If the USPSA site wasn't as slow as it is currently I'd check more results, but I have yet to find stages with < 0.06s between the top shooters. Let alone overall results.

[shred]

1 hour ago, xrayfk05 said:

I don't believe that for a second, certainly not on majors. The more stages in a match, the more it evens out. 

On nationals every shooter will have give or take 20 stages with +- 0.03 difference, in the end it's a wash.

 

If the USPSA site wasn't as slow as it is currently I'd check more results, but I have yet to find stages with < 0.06s between the top shooters. Let alone overall results.

2022 Limited Nationals.  Second place was 0.36 match points behind Nils.  Depends on the stage HF but that could be as little as .03 seconds.

 

3rd and 4th overall at the WSSC are separated by 0.05s this year.  Happens a lot more often in SC of course.

 

If it "averaged out", it would be marginally ok, but if say, the batteries die on one timer and the RO's swap to another that runs .02 faster on average after lunch, everyone in the afternoon squads gets free points.

 

[Farmer]

So you’re saying that the farther away changes the times too? Like when the guy running the timer can’t keep up with the shooter and the difference goes from 3-10 feet? I can see what you’re saying about the accumulation of time and also the differences between a fast and slow timer. 

Edited Friday at 05:07 PM by Farmer

[euxx]

7 hours ago, shred said:

Here's the 2 CED's disagreeing with each other (to be fair this could be rounding based on exactly when the clocks started-- FWIW, SG Go also said 0.90.)

 

The variance could be also attributed to how timer's mic was oriented relative to the gun, different sensitivity or echo. I.e. timers hear differently from where they are.

Different timers may have slightly different algorithms for the shot detection, echo compensation, etc. Also need to make sure you have the same settings on all timers.

Also, we generally not measuring time between shots for score. We measure time from the start signal to the last shot...

PS: the CE timers could also have different firmware. May not be updatable in non-BT models.

[xrayfk05]

If the RO is 30 feet behind the shooter that would add 0.027 to the shooters time since sound only travels at 1125fps. (depending on..)

Time for the shooter to wear a timer on his belt? 

[Joe4d]

4 hours ago, xrayfk05 said:

If the RO is 30 feet behind the shooter that would add 0.027 to the shooters time since sound only travels at 1125fps. (depending on..)

Time for the shooter to wear a timer on his belt? 

RO's generally arnt that far behind,  we also know where the last shot is most likely to be fired and have made it there.. I , and I would bet many others tend to push the timer forward towards the shooter to bring the screen into the field of view and near the gun to kinda verify that it picked up the last shot.
Thats why in USPSA at least,,, you cant scroll back on shots for any type of official information.. 

[shred]

13 hours ago, euxx said:

 

The variance could be also attributed to how timer's mic was oriented relative to the gun, different sensitivity or echo. I.e. timers hear differently from where they are.

Different timers may have slightly different algorithms for the shot detection, echo compensation, etc. Also need to make sure you have the same settings on all timers.

Also, we generally not measuring time between shots for score. We measure time from the start signal to the last shot...

PS: the CE timers could also have different firmware. May not be updatable in non-BT models.

Yeah, I did the math and end time minus start time equalled the split time, so then I used the split time as it's easily comparable and avoids the difficulty of starting timers all at exactly the same time.  These were all lying on a table in the open next to each other and shooting over the top of them to minimize differences in position.

 

As such, the main point remains-- for the most accurate results, use the exact same timer for every shooter on a stage so you don't have to worry about algorithms and firmware versions and echo compensation and whatnot coming into play. 

 

With Practiscore some local clubs lock the tablet to a stage and sync a BT timer to it and the tablet and timer stay on the stage, which works well and does this automatically.  Others like to do squad buckets with timers and tablets travelling with the squads which is less ideal.

 

If a person was running a major match and wanted backup timers per stage, line some up and shoot a few rounds over them for calibration before the match.  Designate the ones that all read the same splits as the "ones-to-use on this stage" timers.

[euxx]

1 hour ago, shred said:

I used the split time as it's easily comparable and avoids the difficulty of starting timers all at exactly the same time

Having it easy to compare does not mean it is something representative.
Perhaps to some extent, but at matches - splits don't mean much, but start to last shot - does.

Yes, it is harder to compare. Yet you don't have to start all timers at the same time. Can instead record video/audio and look at the audio track in the editor to measure the actual time looking at the spectrogram.

 

1 hour ago, shred said:

These were all lying on a table in the open next to each other and shooting over the top of them to minimize differences in position.

Each timer has mic in a different place and it also depend on other factors. Was it a hard table surface, or anything around table to produce echo. How far gun was from the timers (also it is unlike that RO will have their timer at the same position as in the test you describing). Etc...

Though using the same timer for all shooters is just a common sense. Or at least the same timer model. And NOT only because of the differences how they register time. The start signal is also different.

 

Edited Friday at 04:26 PM by euxx

[shred]

10 hours ago, euxx said:

Having it easy to compare does not mean it is something representative.
Perhaps to some extent, but at matches - splits don't mean much, but start to last shot - does.

Yes, it is harder to compare. Yet you don't have to start all timers at the same time. Can instead record video/audio and look at the audio track in the editor to measure the actual time looking at the spectrogram.

 

I did the math.  Total-time (shot 2) minus start-time (Shot 1) was equal to the displayed split time in each case.  These split times are 100% representative of the total time unless there's some discrepancy between when timers start the beep and the clock.  That isn't what I was testing, only the relative accuracy of those time clocks over 15-ish seconds.

 

With audio tracks and spectrograms you'd have calibrate the sample rate of those too.   Video frame rates vary.  

 

But it doesn't matter since these are relative comparisons between timers located similarly to each other, used similarly to see how they vary rather than trying to figure out which one is the most-correct to the real-world.  Somebody want to spring for some fancy NIST gear? Or maybe the Oehler people across town have something I can borrow.

[euxx]

On 6/14/2024 at 10:31 PM, shred said:

I did the math.  Total-time (shot 2) minus start-time (Shot 1) was equal to the displayed split time in each case.  These split times are 100% representative of the total time unless there's some discrepancy between when timers start the beep and the clock.  That isn't what I was testing, only the relative accuracy of those time clocks over 15-ish seconds.

 

The math you did is off. The shot 2 minus shot 1 - is not a total time. It is only a split between two shots.

At matches these splits aren't used for scoring. Only the total time is used. And there are more things in play there.

While your test does have some merit to check raw time deviation (not the same as accuracy) relative to some other timers with another unknown/unmeasured accuracy. But it does not tell you if timer measures time correctly (even between these two shots).

 

On 6/14/2024 at 10:31 PM, shred said:

With audio tracks and spectrograms you'd have calibrate the sample rate of those too.   Video frame rates vary.  

 

You don't need video track for that. The 44.1 kHz or 48kHz is common sampling rate in audio track and it is more than enough to measure time at 0.01 precision.

 

On 6/14/2024 at 10:31 PM, shred said:

But it doesn't matter since these are relative comparisons between timers located similarly to each other, used similarly to see how they vary rather than trying to figure out which one is the most-correct to the real-world.  Somebody want to spring for some fancy NIST gear? Or maybe the Oehler people across town have something I can borrow.

All you need is a phone to record a video for each timer. Recording shooting while the time is measured from the beep to the last shot using a given timer. Then you can look at the audio track from each video using some basic video/audio editor.

You don't even have to take that video yourself. Can just grab them from Youtube. E.g. match runs of people timed with a known timer (often can see that in the video too) and stage scores available in the PractiScore.
 

[shred]

2 hours ago, euxx said:

 

While your test does have some merit to check raw time deviation (not the same as accuracy) relative to some other timers with another unknown/unmeasured accuracy. But it does not tell you if timer measures time correctly (even between these two shots).

 

 

That is exactly what I was checking for "does timer A differ from timer B in the same situation?"  and they do. 

 

So, if a MD doesn't want times differing too, they should use the same timer per stage.

 

Now where you're going is if you want to know which timer is "correct", then you can do video and calibrated time-bases.  Worth doing, but isn't important to the first point that different timers do show different times for the exact same shots.

 

[euxx]

1 hour ago, shred said:

...that different timers do show different times for the exact same shots.

Sigh. Even if they show the same time, it is just a common sense to use the same timer on a given stage through the whole match. E.g. each timer has different beep sound (volume, frequency, duration, etc).