Battis, I used a 1903A3 sight on my Krag.  I used a short set screw to hold it firmly in place.  Did you get a 1903A3 front sight or a 1903 front sight base?
The 1903 front sight base has a horizontal hole through it.  If that base is put on the rifle and you carefully run a drill through that hole, it will cut a groove across the top of the barrel so a pin can be used to hold the base in place.
The 1903A3's horizontal hole intersects the key that would go into a slot milled in the barrel, as you suspected.  I didn't want to try to cut a slot for that key.  That's why I used a set screw instead.  I guess I could've soldered the sight in place but didn't trust my soldering skill.