Does anyone know a good way to read/write files to my hdfs from within MPI? I've done a fair amount of digging trying to figure this out, and just need a general direction to pursue.
There is a full chapter of the MPI Standard about MPI I/O. I'd start by reading there.
MPI implementations have this implemented, usually using ROMIO. You can also take a look at that.