So I haven't had time to take it apart again, but I did do some dry firing. I'm thinking you guys are on to something about the trigger bar and spring. The trigger pull is nice and light at first, but after about 5 or 6 pulls, the weight starts to go up. If I cycle the slide, it basically resets and goes to a nice light pull again.
Currently has a 11lb mainspring, Average of 10 DA pulls is 4lb 9oz. Fairly consistent if I rack slide between pulls. If I don't rack between pulls, the weight consistently increases and goes all the way to 9lb.
So cycling the slide is doing something to reset it.