I am going to start learning Abstract Algebra- Groups, Rings,etc. I am interested to know any programming language, if at all which can help me learn/try the concepts I learn in theory.
EDIT: I am not really looking at implementing what I learn. I am interested to know any language which already supports them.
The text that you want here is Abstract Algebra, A Computational Approach by Chuck Sims. The author will recommend that you use the APL programming language. The book is out of print, but you can probably find it in your library.
There is also the GAP Computer Algebra System which is fun to use for group theory.
Advanced resources:
Magma: fairly sophisticated CAS that can handle a wide range of concepts from abstract algebra.
Macaulay 2: used to study polynomial rings, especially Gröbner bases.
SINGULAR: used to study polynomial rings.
A key fact in studying group theory is that every group is isomorphic to a permutation group by the regular action. Thus, the key to understanding group theory is to understand the permutation groups.
Finally, any language that supports defining your own objects and defining a binary operator on two instances of the object will support learning the theory through programming.