Rounding error?

Hi there,

I'm an absolute beginner in Ruby...

Here's my problem:

print("Length: ")
s = gets()
length = s.to_f
print("Width: ")
s = gets()
width = s.to_f
surface = length * width
puts("Surface = #{length} x #{width} = #{surface}")

When running this code, it asks for Length and Width as expected, but
when I type the values 4.9 and 5.9 it gives 28.910000000000004 as
The values 3.9 and 3.9 give 15.20999999999999 as result.
What do I need to do to avoid these rounding-errors?


Re: Rounding error?

By Peter Vandenabeele at 12/16/2011 - 08:10

On Fri, Dec 16, 2011 at 12:51 PM, Jan Hendrickx < ... at vlaamsemolshoop dot be>wrote:

[Dutch: Dag Jan, welkom op de lijst]

Try to use BigDecimal.

<a href="" title=""></a>"4.90") will create an "exact" fractional number in
the decimal system.

Take care that you feed a _string_ to BigDecimal:

Trying this

b = would still cause the rounding error
so it is not supported.

When ready with calculations, use

big_decimal.to_s to convert back to a string.

A small demo:

peterv@e6500:~$ irb
ruby-1.9.3-p0 :001 > a ="4.90")
NameError: uninitialized constant BigDecimal
from (irb):1
from /home/peterv/.rvm/rubies/ruby-1.9.3-p0/bin/irb:16:in `<main>'
ruby-1.9.3-p0 :002 > require 'bigdecimal'
=> true
ruby-1.9.3-p0 :003 > a ="4.90")
=> #<BigDecimal:946ad1c,'0.49E1',18(18)>
ruby-1.9.3-p0 :004 > b =
ArgumentError: can't omit precision for a Rational.
from (irb):4:in `new'
from (irb):4
from /home/peterv/.rvm/rubies/ruby-1.9.3-p0/bin/irb:16:in `<main>'
ruby-1.9.3-p0 :005 > c ="0.245")
=> #<BigDecimal:94238a4,'0.245E0',9(18)>
ruby-1.9.3-p0 :006 > d = a/c
=> #<BigDecimal:941d828,'0.2E2',9(45)>
ruby-1.9.3-p0 :007 > d.to_s
=> "0.2E2"
ruby-1.9.3-p0 :008 > d.to_s('F')
=> "20.0"



Re: Rounding error?

By Vimal R. at 12/16/2011 - 08:09


I don't face this problem.

When I follow the same code in my machine, I get proper result for
4.9 x 5.9 = 28.91
3.9 x 3.9 = 15.21

I use Ruby 1.8.7. What version are you using?


Re: Rounding error?

By Peter Vandenabeele at 12/16/2011 - 08:20

You most probably _do_ face the problem, but don't see it because the
_presentation_ of the result masks the small remainder errors. And what's
worse, the result might be dependent on the position of the moon (sorry,
joke, but the result will differ, based on the exact Ruby version, CPU
etc. that is used when the code is ran).

On ruby 1.9.3

peterv@e6500:~$ irb
ruby-1.9.3-p0 :001 > 4.9 * 5.9
=> 28.910000000000004
ruby-1.9.3-p0 :002 > "%35.30f"%(4.9 * 5.9)
=> " 28.910000000000003694822225952521"

On ruby 1.8.7

peterv@e6500:~$ rvm use 1.8.7-p330
Using /home/peterv/.rvm/gems/ruby-1.8.7-p330
peterv@e6500:~$ irb
no such file to load -- wirble
ruby-1.8.7-p330 :001 > 4.9 * 5.9
=> 28.91
ruby-1.8.7-p330 :002 > "%35.30f"%(4.9 * 5.9)
=> " 28.910000000000003694822225952521"



Re: Rounding error?

By Mike Stok at 12/16/2011 - 08:25

It you're interested in some reading about floating point then <a href="" title=""></a> is worth looking at.


Re: Rounding error?

By Jan Hendrickx at 12/16/2011 - 09:22

WOW! You guys are realy fast :)))

Vimal R. wrote in post #1037007:
I'm using Ruby 1.9.3; downloaded it yesterday, so I suppose that's the
latest stable version.

Peter Vandenabeele wrote in post #1037008:
[Dutch: Hoi Peter, bedankt voor de verwelkoming :)]

I changed the test-program like this:

require 'bigdecimal'
print("Length: ")
s = gets()
length =
print("Width: ")
s = gets()
width =
surface = length * width
puts("Surface = #{length.to_f} x #{width.to_f} = #{surface.to_f}")

It works fine now, but, well... I don't think it's very 'elegant'...
I haven't had the time yet to read the page on BigDecimal, but I'll do
that right now.

Mike Stok wrote in post #1037010:
I'll go and have a look at that too...

Greetings, and thanks to all!

Re: Rounding error?

By Peter Vandenabeele at 12/16/2011 - 10:29

On Fri, Dec 16, 2011 at 2:22 PM, Jan Hendrickx < ... at vlaamsemolshoop dot be>wrote:

I believe there is no need to convert to float (with to_f).

A problem that I did face is that the default to_s on BigDecimal
gives engineering notation ( the #{length} will execute length.to_s ).


<a href="" title=""></a>

to_s(s) click to toggle source
Converts the value to a string.

The default format looks like 0.xxxxEnn.

I find that non-optimal. I see BigDecimal used mainly for business
(money, sizes, amounts of goods etc.) and there the 'g/G' format specifier
seems most suited as default ...

If the Float (or G) notation was the default, you could simply write your
original code and it would do the

puts("Surface = #{length} x #{width} = #{surface}")

For reference, the "g/G" format specifier (e.g. in C printf):

g, G double in either normal or exponential notation, whichever is more
appropriate for its magnitude. 'g' uses lower-case letters, 'G' uses
upper-case letters. This type differs slightly from fixed-point notation in
that insignificant zeroes to the right of the decimal point are not
included. Also, the decimal point is not included on whole numbers.