Thanks Butlersrangers.
I had swapped side plates with just a little difference going both ways. Part of the problem was I was experimenting with out the action fully assembled. Leaving the magazine cutoff out causes major problems as well as causing damage to the rims.
After figuring that out I did some minor polishing on the upper region of the side plate, as well as enlarging the ID radius leading into the chamber. Not perfect yet but I have a plan.
I made up some dummy 308 cartridges of what was used before the conversion and they did not feed that well either, I had just not noticed.