This is Problem 3 from Project Euler site
I'm not out after the solution, but I probably guess you will know what my approach is. To my question now, how do I handle numbers exceeding unsigned int?
Is there a mathematical approach for this, if so where can I read about it?
Have you tried unsigned long long or even more better/specifically uint64_t?
If you want to work with numbers bigger than the range of uint64_t [264-1] [64 bit integer, unsigned], then you should look into bignum: http://en.wikipedia.org/wiki/Arbitrary-precision_arithmetic.
600,851,475,143 is the number given by the question and 264-1 is equal to 18,446,744,073,709,551,615. It is definitely big enough.