I have never used a ioc-Container but from what I have read, the objects suffixed with "Container" seem to me a factory that reads from configuration/context, does its magic and returns an object of concrete type according to the read configuration/context.
What key aspects am I missing? Using containers seems to add complexity (which is time consuming for development, refactoring and configuration), where to use and which are the key benefits?
Both IoC containers and factories build other objects, but that's where the similarities end.
A factory is an object which knows how to build other objects. It encapsulates extra logic, perhaps configuration time information etc. It can build several of a related type of objects, such as from a hierarchy of objects. But the number of objects it can build is part of the structure of the factory. So you might have a SqlConnectionFactory or a RequestHandlerFactory, which know what sort of SqlConnections or RequestHandlers to build, but if you want to build InputValidators you're going to need to make a new factory.
An IoC container is a tool used to help with the inversion of control pattern. But it's not strictly needed. IoC makes you be explicit in what dependencies your objects have, and makes you provide those dependencies when the object is created/initialized. When done right, this helps with separation of concerns, decoupling between components, unit testing etc. and generally makes the resulting code "better". But because manually wiring up object graphs is a hassle, a number of frameworks for doing this automatically, either from external configuration or from code annotations have appeared. These are called IoC containers. The way they initialize objects is much more generic than a factory. You can ask it to build any type of object, and as long as it knows something about those objects (the signature of the constructor, in the simplest case), it can build them.
It's more usual to see an IoC container use a factory, than the other way around. In much of the code I've seen, the need for factories for all but the most complex objects is greatly alleviated though.