Which rear sight you need depends on which model of handguard you have.  There's a handguard fits the 1892 and 1896 model sights, one that fits the 1898 and 1902 sight, and one that fits the model 1901 sight.  Pictures here:  (You need to Login to view media files and links)

'Kenserres - Welcome to the KCA Forum.

If I understand correctly, you recently purchased a U.S. model 1898 Krag rifle and its rear sight is not completely original.

If you post some pictures of your rifle's hand-guard and rear-sight base, KCA members can tell you which parts or model sight you need to complete your model 1898 rifle.

Most likely your Krag had a model 1901 or model 1902 rear sight. The model 1896 sight is also a remote possibility. (The model 1898 rear sight is unlikely, because these were removed when the higher velocity 2,200 fps ammo was abandoned).

As others have stated, the rear-sights have specific wood hand-guards. The different models of sights require different openings in the hand-guard.

It is possible the OP meant a correct sight for his model 1898 Krag, rather than specifically the model 1898 rear-sight.

Only a small percentage of model 1898 Krags ever wore the three sighting notch Dickson sight, a.k.a. model 1898 sight. Its use was confined to model 1898 rifles and model 1899 carbines made during a specific date and serial number range.

'Rmasters3' - Some brief explanation:

The various U.S. Krag rifle and carbine models are basically referred to as model 1892, model 1896, model 1898, and model 1899. (BTW - This is not dependent on the year they were actually manufactured).

Coincidentally, the various models of Krag rear-sights, (rifle and carbine), are also referred to by model 1892, model 1896, model 1898, model 1901, and model 1902. (The "sight model" does not change the model of rifle or carbine the sight is found on).

Many Krag rifles and carbines received 'sight updates'.

For example, it is not unusual to find a model 1896 Krag rifle with a model 1902 rear-sight (and appropriate hand-guard), on it. Such a rifle is still a model 1896 rifle.

"Legitimate" Model 1898 Krag rifles have been found with model 1896, 1898, 1901, or 1902 rear-sights on them.

It appears your Model 1898 Krag rifle is wearing the hand-guard that was used with both the model 1898 and model 1902 rear-sights.

My 'hunch' is that you likely have a model 1902 sight.
A well focused picture, smaller than 768 KB, will allow positive identification.

FWIW - A model 1902 rear-sight "Slide" will work on both a model 1898 and model 1902 rear sight "Leaf". The 1898 sight "Leaf" has smooth edges. The 1902 sight "Leaf" is serrated on its left side.

'Granpas-gun parts' or S&S Firearms will likely have compatible parts for your sight 

In 1898 the U.S. Ordnance Department increased the velocity of the .30-40 cartridge from 2,000 feet per second to 2,200 fps.

Along with the adoption of this 'hotter' cartridge came the new "Dickson"rear-sight, (the model 1898 sight), for U.S. Krag rifles and model 1899 carbines.

The Dickson rear-sight had three sighting 'notches' and was adjustable for windage. (The two outside 'notches' were to compensate for strong cross winds at distant ranges.

This sight was fitted to model 1898 rifles and model 1899 carbines in the approximate 218,000 to 253,000 serial number range.

The "Dickson" rear-sight was not particularly popular and the 2,200 fps ammo was blamed for the sudden appearance of cracked Bolt Locking-Lugs in U.S. Krags.

The "Dickson" sight and higher velocity ammo were eventually abandoned. There was a return to the previous 2,000 fps .30-40 cartridge.

The base of the 1898 sights was eventually sold off as scrap. Its curved ramp was designed for the high velocity ammo. The base was too low for 2,000 fps ammo.
(A similar looking sight and base would be adopted as the model 1902 sight. This base was taller and was machined for a simpler style of flat Spring for the Leaf).

The model 1898 sight Leaf and eye-piece were retained in stores at Springfield. Ultimately, the Leaf received serrations on the left-side and the three-notch eye-piece had its square corners 'rounded-off', which removed the out-side "V" notches.
These altered parts were used to build one of the variations of the model 1902 sight.

The actual "model 1898 sight" is relatively rare, was only put on a narrow range of 1898 rifle serial numbers, and it was intended they be taken-off and salvaged or scrapped.