Many times I play with me Emacs config and break it a lot. Is it possible, in case of a mistake in the config file, to load the previous one that was ok and display some kind of informative error message?
You can't trivially "undo" a partially-loaded config, any more than you can trivially undo any other partially-executed software program (which is what your config is -- a software program), and so any such roll-back is going to mean killing Emacs and re-running it with a replacement config.
There's a very common manual solution to your problem -- version control. If you keep your config under version control then, if you manage to break it, you can roll back to the previous working revision (which in general is something you can establish better than Emacs).
Those steps -- killing Emacs, rolling back the config, and starting Emacs with the modified config -- need to happen anyhow. It's probably not much bother to do it manually.
I don't have any semi-automated solution for you, but if all you care about is "we reached emacs-startup-hook without any errors", then you might use that hook to trigger an rsync of your config directory to your backup directory, and then write yourself a script to execute a rollback by swapping configs.