I've just started to work with MPFR arbitrary precision library and quite soon encounter very weird behaviour. The main goal of using it was to improve precision of large argument 'trigs' and this works extremely good in MPFR.
But then I decided to check out simple math and it was unbelievable - even in straightforward examples with strict answers rounding errors exist and doesn't depends on used precision.
Even in the example like 1.1 * 1 the result is 1.10000000000000008881784... And this result given at insane 2000 bit precision (53 in normal double)!
Maybe it is my system's issue, but even in online MPFR similar problems exist. You can try such an example online: http://ex-cs.sist.ac.jp/~tkouya/try_mpfr.html
1 * 1.1 @ 64 bits = 1.10000000000000000002
But online version moves error further with increase of precision but in my installation - no.
My system: Ubuntu 9.10 + gmp 5.0.0.1 + mpfr 2.4.2
This appears to be the canonical answer to floating-point precision issues. It includes an explanation of why more bits doesn't solve the problem.
Old Answer
See question 7 of the MPFR FAQ. Anyway, there will always be some error, since many numbers are not exactly representable with binary mantissas no matter how many bits you use. See this answer to a related SO question for a good explanation.