Not having seen any pictures, I have to assume (since you said there was a plate on the side) that you have the earlier stock which is 30" long. Is that correct?

The proper rear sight would be the 1896C. They are very rare, and usually run around $600 or so. They have also been widely faked, so I do not recommend purchase by a novice. A much better choice would be the common 1896R (rifle) sight which looks identical - only the graduations are different. A special barrel band, with hump, is required. All of those parts are available, though the handguard will have to be a repro.

Without more knowledge (you say the buttplate has been changed) that may be the biggest issue. If anything at all has been done to the stock which would prevent reinstallation of a proper buttplate, it will have to be replaced, and that will not be cheap.

I think the first thing you should do is get Poyer's book on the Krag, for $20. It will answer a lot of your questions. Google "North Cape Publications". All of the parts you need are available - Google "Krag parts", or check out gunbroker.com - and they are not all expensive. The barrel band, sling swivel, rifle sight, and buttplate are common. Assuming your stock is OK at the butt, I'd think you should be able to make the gun LOOK right for around $200-$300.