I just spent about an hour looking into this, because I can't let something like this go either once I've seen it. I thought it was something with the font, but the font widths appear to be perfect. It's not the bearing, and it's not the kernings, and every single point on every single glyph is at the correct horizontal location to the integer...
I'll try to keep this short by telling you where I stopped for the night... I stopped when I realized that setting different font sizes would change whether the numbers moved to the left or right...
I'll see if I can find someone who knows fonts better than I do to look at this, but in the meantime, I hope you don't mind that this is not particularly high on the priority list. I assure you that it will hurt me deep inside every time I see it now, though.