I just started learning Ruby, and for my sandbox I use latest Rubygems version, according to examples and tutorials.
At the same time I have Redmine, which requires Rubygems 1.6.2.
To manage gem versions I use bundler, but can I use it to manage Rubygems version too?
I dont think so - I believe it uses your current rubygem from the environment.
However you can use rvm to manage the rubygems version, so if you install it and have a project specific gemset in your .rvmrc file.
I think rvm is OSX/unix only - there is an alternative for Windows I believe.