Hi, Marty. Sounds like an annoying problem but one that can be remedied in many ways. I'm sure you'll get a lot of different suggestions here.

Pictures of the rifle and stock where the band is poorly fitting would help a lot.

You can try to do something to build the stock up a little or work with the band itself. Working with the band might be easiest. Can you apply some duct tape to the inside of the band to allow for a tighter fit to the stock? Some people have carefully squished the band to decrease the inside diameter to allow a tighter fit.

You could put a thin shim of wood in between the two to make it fit. Or even put some duct tape around the stock where the band slides on.

Is the screw tightened completely when it's on and it's still loose?

See if any of these suggestions help. I always try simple options first that do no damage to the rifle that can permanently alter the rifle and its value. If it's bad enough, you might consider a replacement stock which can be costly but a nice option. Many Krag stocks have been altered and damaged that require fixing or a different stock to correct.

Good luck. We'd love to see pictures and keep us informed. There may be something else going on. I had to rebuild the front of one of my stocks that had been damaged when someone tried to put the incorrect front band on it. It was cracked and damaged bad but I made an acceptable repair.