Every docker run command, or every RUN command inside a Dockerfile, creates a container. If the container is no longer running it can still be seen with docker ps -a.
Should I be concerned with having an enormous list of non-running containers? Should I be issuing docker rm on non-running containers?
I am unsure of what performance or memory/storage penalties these non-running containers incur.
The containers that are not running are not taking any system resources besides disk space.
It is usually good to clean up after yourself, but if you have a lot of them sitting around it shouldn't slow down performance at all.
If you do notice a slow down when running docker commands with lots of stopped containers, it might be a bug in docker, and you should submit a bug.