Flow Rate vs Kpa.. Correction factor?

[Todd157k]

I'm dialing in my 150 shot Dry system with SVO 30s and everything is looking real good thanks to the help on here. The question is:
When I try to put in the exact Flow Rate values (calculated from Redhotsupra's xls) I type in 34.64163 (for example on 0 kpa) and hit "=" but the number that appears in the chart is 33.48293. A difference of 1.1587. (this same percentage difference for all the fields). If I type in "1" and hit "=" it gives me .99209 (for all kpa fields). I opened up a stock file from the repository and try it but come out exact.. 1=1.
Where is it getting this "correction factor" from and how do I get rid of it?

[Todd157k]

Bueller???



Bueller???

[5_Liter_Eater]

This is normal although I can't remember the reason why this happens.

[chrisj6]

Its something about the resolution in the OS....I could be wrong though.

[Todd157k]

That was my first thought however when I downloaded someone else's file and looked, I can set the Flow rates to any number I wish.. without any percentage differences. hmmmm

[Bill@HPTuners]

its the incriments in which the pcm can enter.

[Todd157k]

its the incriments in which the pcm can enter.

So, I guess the real question (since I can't get rid of it) is... do I adjust the input values so that the final values are approximately correct or just leave it alone since it's close?

[Bill@HPTuners]

your within .01 of the number your entering, I'd say that qualifies as "close enough"