Is MPI implementation of matrix inversion a standard library routine or good MPI implementation critically depends on your setup(software/hardware) so have to be coded by yourself.
ScaLAPACK provides implementations of lots of linear algebra functionality, including the solution to least squares problems. See for example the ScaLAPACK users' guide. (I don't think ScaLAPACK has a general matrix inverse - hopefully your application needs the inverse only to solve a linear system - in which case, using the functionality to do that directly will be more accurate than attempting to find an inverse).