Ok, so I think I found a solution to the issue. Time will tell. I only post it here, since I'd like to keep all the great advice on this thread in one place.
I recently bought a new set of drums for the auto-drum. Since I'm only trying to drop 4.5 grains, I opted to use the small drum, as opposed to the large drum with insert.
After cleaning the small drum with Dawn and letting air dry, I installed it. I put about 1/2 lb through to "condition" it, and started seeing 4.4-4.6 grain drops! I only did about 25 drops, but its progress.
As an experiment, I tried the same thing, but with the large drum and insert. With that setup, the cavity for the powder is very shallow, as opposed to the small drum. And the results were back to inconsistent drops.
Seems that the lesson is that you gotta use the small drum for the smaller grain drops. Even thought the large drum will "hold" those smaller grain loads, I'm guessing that the wiper shears off the top layer, and causes inconsistent drops, especially with the cavity being so shallow.
Hope that helps someone out there.