I have three EC2 instances, each of which has its own MongoDB, and they are connected in a replica set.
because an EFS, can be shared across multiple EC2 instances, I am wondering three things...
would the MongoDB instances still need to be connected in to a replica set if the data on disk is effectively already shared between all three instances? (my concern is what's in Mongo DB's RAM, and that each Mungo DB instance would not 'know', what data it had, or if directly plopping the data to disk would omit an important 'indexing' step)
from a performance standpoint, it is better to not have MongoDB writing the same thing N times to N, different instances if we only needed to write once to the file system?
to make sure everything is "safe", should I have each EC2 instance mounted to its own EFS and just not concern myself with the above 2 questions? (in which case the cost would be N times the cost per GB)
To start with, never do this. Database systems generally lock the data files they are working with. This is so they can control when and how data is actually written to the disk. You may be able to get around this locking mechanism by using a networked file system, but you shouldn't. You will open yourself up to data corruption. If you can even get this to work in the first place.
See point 1, you will corrupt your data. Replicas are intended to improve the read performance and availability of your data.
You really don't need to use EFS for Mongo. Using replicas, and/or shards if you need more write performance works well.