What are some best practises for prototyping a filesystem?
I've had an attempt in Python using fusepy, and now I'm curious:
In the long run, should any respectable filesystem implementation be in C? Will not being in C hamper portability, or eventually cause performance issues?
Not necessarily, there are plenty of performing languages different to C (O'Caml, C++ are the first that come to mind.) In fact, I expect NTFS to be written in C++. Thing is you seem to come from a Linux background, and as the Linux kernel is written in C, any filesystem with hopes to be merged into the kernel has to be written in C as well.
Are there other implementations like FUSE?
There are a couple for Windows, for example, http://code.google.com/p/winflux/ and http://dokan-dev.net/en/ in various maturity levels
Evidently core filesystem technology moves slowly (fat32, ext3, ntfs, everything else is small fish), what debugging techniques are employed?
Again, that is mostly true in Windows, in Solaris you have ZFS, and in Linux ext4 and btrfs exist. Debugging techniques usually involve turning machines off in the middle of various operations and see in what state data is left, storing huge amounts of data and see performance.
What is the general course filesystem development takes in arriving at a highly optimized, fully supported implementation in major OSs?
Again, this depends on which OS, but it does involve a fair amount of testing, especially making sure that failures do not lose data.