I have got a Western Digital 500 GB internal SSD (WDS500G1B0A). Its specification states 525 mbps of sequential write speeds. I formatted this SSD using f2fs.
Then I performed sequential writes of size 4k on the file system using O_SYNC flag (to ensure that the writes get committed to disk when the write returns). My test program is single threaded.
I understand that the throughput of this is going to be a combination of IOPs and latency ratings of the underlying SSD. What throughput should I expect with such execution? I got only around 8k IOPS.
I can also share my test program in case I am missing something basic.
Am I also missing something else with formatting SSD etc.?
When I run the same program without O_SYNC flag on the same formatted SSD, I get 10 times better results (around 80k IOPS). What additional tuning or testing I can do to get a better handle on what is happening?
Let me first admit that I was baffled by the 80 k IOPS without O_SYNC flag as they happened to coincide with the SSD IOPS (525 MB / 4k blocksize ~ 100 k IOPS). I should have stuck to the throughput specification of 525 MB for my analysis. What I observed is that the SSD throughput indeed gets close to this specification even with O_SYNC flag albeit with very large block size (around 16 MB) which is understandable.
Since without O_SYNC flag, linux must be committing the writes in in-memory kernel buffers we cannot really use it for underlying SSD IO performance analysis. It must be dependent on other factors of the hardware like memory, CPU etc. E.g. on a more powerful server I could even see 2000 MBps which is unrealistic at the SSD level.
SSD can perform better than HDD even without O_SYNC flag while the writes are getting flushed to disk periodically since that would be a reasonable model for the IO application on top. I can share my numbers with this analysis in case anyone is interested.
Sorry for going on a different tangent with my original question. However it was some good learning anyway. Hence I thought of posting my answer here.