Machine vision to sort by headstamp

[FingerBlaster]

I got the new camera, lens and the nvidia jetson nano.  I played around with everything for a couple hours. 

 

The new camera and lens combo is great, i just need to figure out a good mounting solution.  I can capture and decode an image in less than a tenth of a second and I think it's a better quality image.  So looks like a win/win.  The only downside i see so far is ribbon cable it uses is a little annoying vs USB.

 

The jetson nano, i'm not sure it was a good purchase.  Might return it.  it's a little more complicated to set up than the raspi, it's still runing ubuntu 18.04 which leads to some depdency issues. 

 

It's also slower to classify images right now, but i must be doing something wrong, i need more time to mess with it.  It SHOULD be much faster by all accounts.  

 

I also tested using the jetson to train the model, and i'm estimating that it will take about 6-8 hours to train.  Not bad considering it's not made to do that. However, when you can do it in the cloud in 20 minutes *shrugs*.  Might be worth it to some people though, esp if they don't have good internet to upload almost 1gb worth of images to the cloud. 

 

 

[assnolax]

The Sortinator seems to view the images in high contrast but the quality didn't seem to be crazy high.  Would that be just to speed things up? Not sure what impact it would have on accuracy. 

 

Had a thought while sorting brass.  You mentioned before junk stepped brass, I came across some berdan primed brass.  Should we hold onto that to train for definite rejects or will you be keeping the most common  sorted good headstamps for use with the last bucket of misc being the recycle pile? 

 

 

 

 

[FingerBlaster]

So I'll be honest, i'm not sure this paragraph is all speculation.  If i were to guess, the preprocessing a high contrast image may make it more accurate, the edges of the text seem to be more clearly defined.  But you lose the ability to seperate based on case material.  I'd like to be able to identify steel cased for example once i get to .223 by color, and his images look almost black and white, so i'm not sure how that kind of pre-processing would allow you to pick out case color.

 

I'm messing with the camera and lens mainly because the issue is the image transfer is very slow.  I also believe a better quality/sharper starting image will generally produce better results in general, i have nothing to base that on other than intuition.  Part of the issue is these cheap USB cameras are not meant for "macro" photography.  The standard/common lenses are made for cctv type stuff and of the dozens I've looked at they won't focus on objects closer than 200-500 mm, and they have wide fields of view.  So currently in order to get a 512x512 image of the headstamp because the field of view is so big - I'm capturing a huge high res 4000x3000 pixel images, with these cheaper cameras, they get a little noisy/grainy at high def as well, at least the one i'm using.  Then cropping down to around a 700x700 image and throwing  11.5 million pixels away resizing to a 512x512 image that I'm saving as is, then doing a little pre-processing and shrinking the image down to 256x256 to feed to to image classification model.  I'm saving the higher res photo for now, until i decide on the optimal size to send to the image classifier, right now 256x256 is having the best results of speed & performance.

 

The new camera and lens combo I have, is actually a low power microscope, it will focus from 200mm-50mm on a part and have a very narrow field of view, at the lowest magnification it captures a field of view of about 1.5" inches wide.  So now I can go down and capture a lower res image 1920x1080 and only have to throw about 1.5 million pixels away to get a 700x700 image and then do the same steps as above.  The raw capture size is about 1/10th of size with the other camera, so it should be about 1/10th the speed, and looks like it generates a less noisy image from my very limited testing.

 

Right now save as many oddball cases as you can, rather need them and not have them than the inverse.  My gut feeling is we'll need at minimum 100, and ideally closer to 1,000 of each case type to reliably identify, but I'm unsure how things change if i make a "junk brass" classification label and put a bunch of different heatstamps in there.

 

I'm starting to think about different calibers now too,  Once i mount the camera i'm trying to set everything up with that in mind.

[btp]

@FingerBlasterthink I recall you mentioning using tensorflow before, but what type of model are you using? Are you rolling your own or performing transfer training from an existing one? Contrary to intuition you are better off with lower resolution images as this will allow you faster processing time which is what is critical here for speed, but you need to make sure that the image resolution you are using matches what the model was designed/trained for.

[FingerBlaster]

 

Question to all: What calibers would you like to be able to sort with this? 

 

for me, 9mm, .45, .223, and .308.  I want to redesign the position of the new camera, and i want to do it with the longest cartridge in mind.  Looking for any likelyhood of interest in anything longer than 308?

Edited April 30, 2021 by FingerBlaster

[FingerBlaster]

1 minute ago, btp said:

@FingerBlasterthink I recall you mentioning using tensorflow before, but what type of model are you using? Are you rolling your own or performing transfer training from an existing one? Contrary to intuition you are better off with lower resolution images as this will allow you faster processing time which is what is critical here for speed, but you need to make sure that the image resolution you are using matches what the model was designed/trained for.

 

Using a totally custom trained model.  I've tried various image sizes and 256x256 seems to be the sweet spot, Larger, images and it's too slow, smaller images and i can't get good accuracy...  at least based on my CNN parameters. 

[btp]

And @FingerBlasterwhat type of pre-processing are you doing on your training data?

[btp]

Just now, FingerBlaster said:

 

Using a totally custom trained model.  I've tried various image sizes and 256x256 seems to be the sweet spot, Larger, images and it's too slow, smaller images and i can't get good accuracy...  at least based on my CNN parameters. 

 

If you're not getting the accuracy you need, try transfer training with a prebuilt model. The best image classification models (the ones that win the competitions) are (going from oldest to newest) are ResNet (multiple different versions depending on the layers), InceptionV3, and EfficientNet (again multiple different versions). These models are huge and take a long long time to train from scratch, but transfer learning allows you take a pre-trained model, remove the last output layer and replace with your own output layer to classify as you need. Then transfer training allows you to start with the prebuilt model so much quicker. 

 

Have a go at transfer training with those existing models. Each one of those models will need a specific size (e.g. resnets typically a 224 image size but can accept certain others), along with different versions of those models being deeper than others (e..g resnet-18 has 18 resnet blocks (layers) vs the much deeper resnet-152). The trick is to find the smallest prebuilt model that satisfies your needs - you don't need 100% accuracy. Tensorflow (and all other major frameworks) include these prebuilt models as well as examples for transfer training.

 

Are you using the full blown tensorflow? Or tensorrt? If not, try the latter, as includes optimizations for lower powered devices such as raspberry pis and Jetsons.

 

Regarding image pre-processing, this is where you can improve the accuracy more than you think. I typically make 100's of variations of a single image for training by rotating, skewing, changing the color, changing the light, applying perspective transformations, noise, etc. 

 

Hope that helps.

[FingerBlaster]

@btp with opencv, i'm resizing and using HoughCircles to apply a mask over the primer and measure the diameter of the primer (knowing the diameter is useless for 9mm, but will be handy for .45). The mask is to eliminate features of the primer I'm not concerned about.

 

when training the model I'm only applying a RandomRotation of 2pi, i didn't see the point in any other keras image preprocessing like random zoom, flip, contrast  since the casing will always be in the same position.

[FingerBlaster]

Very helpful!  Thank you.  I'm just learning about TF and Machine Learning.  I'm gonna send you a DM, sounds like you're way smarter with this stuff than me.  Would like to keep in touch.

 

I'll try resnet and transfer learning as you suggested.  

 

I'm using TFLite for image classification right now.  Just started looking into TensorRT last night, but haven't had any success yet.

 

 

[btp]

2 minutes ago, FingerBlaster said:

@btp with opencv, i'm resizing and using HoughCircles to apply a mask over the primer and measure the diameter of the primer (knowing the diameter is useless for 9mm, but will be handy for .45). The mask is to eliminate features of the primer I'm not concerned about.

 

when training the model I'm only applying a RandomRotation of 2pi, i didn't see the point in any other keras image preprocessing like random zoom, flip, contrast  since the casing will always be in the same position.

 

Yeah you don't need to  bother with the masking. Masking is important in the traditional computer vision sense (e.g. just using opencv to detect it), but when applying deep learning it's not needed as the model itself will learn whats important. Thats why the training image transformations as much as they don't make sense to a human, such as skewing it, rotating it, applying noise, and adjusting the color, to the point where you can't tell what it is by looking at with the human eye, is key. The model will be able to pick up on patterns in the pixels that are indistinguishable from the human eye.

[FingerBlaster]

16 minutes ago, btp said:

you don't need to  bother with the masking.

 

I have 2 data sets: My main set which i do an 80/20 split for training/validation and a testing dataset.  In my test set, masking saw an almost a 10% increase in accuracy.

 

Possible reasons:

I still don't have a good distribution of training images, they are heavily weighted to 3-4 different headstamps. 

I will try more data augmentation like zoom/skew/color/contrast as you suggested. 

Could just be my CNN layers not being ideal.

I will also try transfer learning. 

Edited April 30, 2021 by FingerBlaster

[assnolax]

This went well beyond my head this point.

 

  

Another possible issue while hand sorting.  I have brass I consider the same headstamp like the attached FC but the depth of stamping and the location of the dimples changes.  Will it be smart enough to ignore the extra details or will each FC headstamp need its own training set with the code to sorting both headstamps down the same chute?

 

I've also seen military brass with a year stamp that would possibly cause a similar issue.

[FingerBlaster]

@assnolax I'm not anticipating any issues as long we have enough sample images of different types.

[jmorris]

 

Quote

 

So I'll be honest, i'm not sure this paragraph is all speculation.  If i were to guess, the preprocessing a high contrast image may make it more accurate, the edges of the text seem to be more clearly defined.  But you lose the ability to seperate based on case material.  I'd like to be able to identify steel cased for example once i get to .223 by color, and his images look almost black and white, so i'm not sure how that kind of pre-processing would allow you to pick out case color.

Unfortunately case color isn’t a positive ID.  Like the brass plated steel S&B 9mm case below, clean and new they look the same as an S&B brass case, a magnet picks them out instantly though.  I have also came across nickel plated steel cases as well, my magnet at the case feeder caught it.  Aluminum cases, if not already removed during a previous sorting process would be pretty easy because the “N R” in the head stamp.

 

You are making great progress, keep up the good work.

[FingerBlaster]

2 hours ago, jmorris said:

Unfortunately case color isn’t a positive ID.  

Well ain't that annoying!  nickle and brass coated steel? Jeez!

 

Great idea with the magnet on the case feeder though, definitely a good enough solution for now. 

 

I wonder if I could add in a sensor to see if a casing is magnetic and kick it based on that.  No idea if this would work or how to implement it at this point, i'm vaguely remembering stuff from highschool physics.  Something along the lines of an inductive loop, when the bass drops through it if it's ferrous it should do some magnetic voodoo, something with fields, maybe currents and somehow detect it...  That's a problem for another day, but just a thought.

Edited May 1, 2021 by FingerBlaster

[jmorris]

I wouldn’t worry about them, steel is by far the easiest to sort out.

[Hansimania]

Hey again, 

 

Not been here a few days and everything seems to go along super fast!!! You are THE MAN!

To answer your question: I plan to use this mainly for 9mm and 223. Maybe later for 7.62x39 and 308.

To find any casings that are not brass, I can imagine that a inductive sensor could find them? Like the one I suggested to check if a case is in place for the next picture a few posts ago. However I can not find proof for this idea, I hope I am just too dumb to use google:)

 

I still have not much to contibute, but I have a question. Right now you are teaching the machine some case brands. If I was to build the same machine that you have right now, couldn't I start taking pictures and send them to you so you have a much bigger dataset? Or is it even possible to teach my machine some case brands, and then just let my machine tell your machine "what to look for"? I might be dreaming, but it would be super nice to have a place to share all the data that all the machines are collecting. So each machine will be able to recognize all the brands that everyone was sorting just by downloading the most recent dataset. (But I think this would be something to set up for a later stage)

 

By the way, If this project goes as planed, you might make the guys that sell the Sortinator sad. everyone will be able to just build their own sortinator, even a cooler version, for much less. I can even imagine that some commercial reloaders choose this instead of spending the money on a expensive machine just to have brass sortet by headstamp. 

 

Best regards and keep up the good work!

 

 

[FingerBlaster]

@Hansimania my expectation is that we should be able to share the trained model and it should as a starting point at the minimum, hopefully work darn good if it's sufficiently trained.  I'll end up putting my model and all my training data on a github repo along with my code once i'm there, which is starting to get close

 

@btp has actually helped me a lot with the machine learning side of things, he gave me a a very in-depth example of code to try along with detailed instructions that i'm just getting to.  Dude is clutch.

 

I made quite a few mechanical upgrades over the last week. 

 

1. The biggest is I should be able to support multiple calibers now, you only have to change out 1 part, and i made it an easy swap.  I designed it to work with 9, 40, 45, 223, 300blk, and 308.  I'm sure it will work for other calibers too, ultimately has to do with case length.  i briefly tested 223 just to see if it should work out of 9mm drop assembly and it looks like it should.   I have to print a new part to test 308 and just haven't gotten around to it, i expect i will have to make another tweak elsewhere though.
2. swapped out the bailing wire with servo pushrod wire.  It's a lot cleaner looking and the servos are struggling a lot less.
3. I have 4 output gate modules set up now, so i can support 8 buckets, plus a reject bucket. I can maybe add 1 or 2 more drop modules so another 2 or 4 buckets, but any more than that and you'll have to put the case feeder ridiculously high in the air, or cut that clear output tube that comes off the case feeder and make it shorter.  so the practical limit is probably around 10 to 12 buckets.  Realistically i imagine much more than most people need.
4. I changed the camera and lens, I think it's a big upgrade.  I can capture and crop an image in around 0.07 seconds, vs around 0.5 second with the old camera.  This should speed things up a bit and should be able to hit that case a second goal.  Also, the images are in better focus and less noise in the image.
5. I got a new light ring - I haven't installed it yet, but it's controllable by the raspi/jetson instead of just having a separate on/off switch.  have to design and print a housing/mount for it.  It's a few bucks more than the usb powered one i got on amazon, but i like being able to control it via software vs a separate switch.

 

I'll be monkeying with code a lot more, taking more images, working with what BTP helped me with, working on the interface.  I've been working on cleaning up the code to post on a github repo along with the cad files.  I also started building a shopping list.

 

 

The sortinator is a turn key product.  This is definitely more geared towards a hobbies/tinkerer/maker, some assembly required..  You're going to need some intermediate computer knowledge: SSH, command line Linux, maybe a bit of python.  A 3d printer of course.  Some soldering skills.  Probably around $350 in parts.  

 

Edited May 9, 2021 by FingerBlaster

[quiller]

Excellent news...$350 verses 5k is good in my books and time to tinker I have. 

[FingerBlaster]

Ran into an issue today.  It didn't come out of left field, it was done for simplicity of design.  I was concerned it might happen, but had a workaround I thought would work.  Spoiler, it didn't hold up.

 

The flapper gates that divert the brass to the flex tubing that goes to the buckets are powered by a short pulse from a $2 toy DC motor to open and close.

 

The issue with DC motors is when no power is applied they rotate freely with almost no resistance.  So if a piece of brass in free fall bumps into one of the gates there's nothing to keep it from bouncing open.  I have been applying 20% power to the motors, to hold them closed, hoping it wouldn't burn them out and it seemed to be working fine.  Until today when i got magic smoke.

 

So I'm going to have to do what I should have done from the start and redesign this with servos.  And possibly go through a few test print iterations.

 

Also using black hose I probably wouldn't have even noticed an issue if i didn't see the smoke. I heard some of the doors opening and closing and everything seemed to be working after i powered it back up.  I'm looking for a clear hose alternative.  The current black hose i'm using is a little over 1.25" ID and is like $0.50/foot and is very flexible, it's a cheap  sump discharge hose kit from home depot.   2.5" ID dust collection hose for shop dust collection systems looks like the right concept, but it's just too big and I can't find a smaller ID alternative that's flexible and not unreasonably priced. 

Edited May 11, 2021 by FingerBlaster

[FingerBlaster]

Had the morning off, worked on some design ideas, the changes are minimal and i don't think i'll have to reprint everything.  Tomorrow I should have a ten pack of cheap Chinese servos from amazon for $20.  Made a few partial test prints to work on tolerances, and conceptual ideas, with the few spare servos I already have and i think this will work, but the servos i have are more expensive.  Waiting on the cheap Chinese ones to take measurements before i print and test the entire design.

 

My hope is that the cheap servos will work.  Some of the cheap servos have issues where they "lose their place" and don't return to the same spot.  Hopefully these servos don't have that issue, but it's possible.  Even if it happens, my main concern is not the gate being close/open 98% vs 100%, but rather the gate trying to be closed 102%, constantly pushing the motor and the servo burning out

Otherwise i'll have to use the same mid range servos i used for the pins to drop the brass, which are like $10 each.  They should have no issue tracking zero.  The Issue is if you want 12 output bins, that's $120 in servos vs $24 with cheap chinese servos...

[Hansimania]

Why not design the doors so that they will close automatically, with a weight on one end. So if no power is aplied, the will just close themselves and the motor just needs power to open them? Something like this, but the other way around

If this is no option, make the doors so that they can be swung open more than 100%? It wouldnt hurt if they have a gap on one side while they are open would it? 

[muncie21]

@fingers- have you considered using a vacuum star wheel to control distribution of the cases?  More complicated to setup than a servo, but very robust, repeatable design. 

[FingerBlaster]

@Hansimania Good idea, i didn't think of that, but ultimately i don't think workable.  These dirt cheap little DC motors are all speed and zero torque.  Any sort of weight and they don't move, a flea would keep them from moving.  There are work arounds:  Gear reduction, stronger motors, heck a worm gear would solve the issue of keeping the door in place with the motor unpowered... Either way they all require some redesign and servos are just the easier solution for me.

 

@muncie21 I have a few other ideas, in fact something similar to a star wheel is one of them.  However my main goal is to keep this as simple as possible.  I feel that's a level of complexity I'm not ready for yet.  I do think that kind of design has the ability to operate faster.

 

Funny thing is, i actually didn't burn out the motors as i thought.  Turns out I burnt out one of the chips on the motor control board.  The failed chip was running 2 output gates, for the FCNT and FC headstamps, so it was the one being taxed the most since those 2 headstamps account for 75% of the brass I've run through it so far.  The magic smoke came while the motors were stalling, my guess is just too much current.  The chip on the the bottom controller motors M1 and M2, you can see the burn marks in the attached pic

 

I got a couple different servos. Some cheap $2 servos with all plastic gear sets, and some slightly upgraded $6 dollar servos with metal gears.  The $2 servos need m2.2 which where a little hard to source in the size i needed, i found #2 screws which work since they just cut their own threads into the plastic output shaft, the $6 servos use m2.5 which were easier to find.  Also with the metal gears I had a much better feel for how much torque to apply without stripping the output shaft.

 

I bench tested the "zero" repeatability on both of these servos and i'm pretty impressed that they both tracked fine.  I just ran a script to put them through various motion cycles and i checked on them at the end of each cycle.  my biggest iteration was 1000 cycles and they were both fine.

 

I tested both of them running different output gates.  The programming is much easier, i got rid of some sleeps that I needed for the motors and gained a little speed.  Overall I thnk going to servos was the right call.  The $20 servo control board I was using can handle 16 servos, 4 are to control the flow of brass, so 12 can be used for output bins.  The $20 Motor control board can only run 4 motors per board.  So for 12 output bins you need no extra control boards with servos, for motors you need 3, which is another $60.  $5 per motor offsets the cost of $6 servos with this revision.

Now that I think i have this issue tackled it's back to working the programming.

 

Edited May 17, 2021 by FingerBlaster