The reason the loaded round kicks out better is the concentrated weight at the front end "carrying" it, inertia-wise, so to speak. By contrast, the empty case has its' weight concentrated at the back, right at the pivot point, so does not have the same incentive to jump. That being said, you still may have a sprung extractor, or a worn hook on same.

And, let's be honest - ejection is not the Krag's strong point - never has been- never will be. The soldiers were even taught to roll the rifle slightly to the right to "help" the case fall free. Some rifles will snap 'em out, but for most it's a rather labored mechanical process.