One of the distinguishing features of SAS is a full-duplex connection between the HBA and device. SATA provides only half-duplex connections. I used to think that SAS drives could simultaneously ingest and retrieve data using both directions of a full-duplex connection.
However, Wikipedia claims that:
The SAS transport layer can transmit data at the full speed of the link in both directions at once, so a SCSI command executing over the link can transfer data to and from the device simultaneously. However, because SCSI commands that can do that are rare, and a SAS link must be dedicated to an individual command at a time, this is generally not an advantage with a single device.[12]
What are these rare SCSI commands mentioned above?
Does it mean that SAS drive cannot transfer data in both directions simultaneously, for example, serving write and read requests at the same time?
I think that Wikipedia page is offering too narrow a reading of SAS's capabilities.
It's a serial protocol. Commands and their data are transmitted through the same communications channel (the SAS link). By way of example, imagine a magical SAS drive and controller that have zero added latency, so that you only see the effects of the link speed limit.
If the initiator sends a WRITE16
command for 1 MiB of data, then follows it up with the 1 MiB of data to write, it's going to take at least 874 microseconds to make it over the link. That's 1 MiB of data + 16 bytes of command, with 8/10 encoding == 10,485,920 bits
, transmitted at 12 Gbit/s.
There might be some additional framing in there, but you get the idea.
If the initiator follows that WRITE
up immediately with a READ
request for 1 MiB, the drive won't even see the READ
request until it has ingested the last of the WRITE
data. When it encounters the READ
, the drive transmits the 1 MiB of read back to the initiator, which takes 874us.
If you do the math on that, it says that we moved 2 MiB of data in 1,748 microseconds, which is 12 Gbit/s. That certainly makes it look like you can't read and write at the same time, and the naysayers were right. But, hold on.
The real win for SAS is when there's a lot of I/O happening, and the controller can arrange things such that it interleaves read and write commands. If the initiator issues a sequence of WRITE
, READ
, WRITE
, READ
, etc., then the drive can be replying with read data while it's ingesting the next write and its data. Once you get a pipeline like that rocking, from an application perspective you can write and read at 12Gb/s simultaneously.
In practice, it actually sometimes works out like that! Full duplex.
I admit I don't know what that article is getting at with rare full-duplex SCSI commands. I think it may simply be confused. Here are two possibilities:
EXTENDED COPY
is an offloaded target-side copy, capable of reading and writing data. That is typically implemented by a storage array and consumed by something like VMware.
There are bi-directional SCSI commands like XDWRITEREAD
, which reads data into a buffer, XOR's it with write data sent by the initator, then returns it back to the initator. Though it transmits data both ways, it's kind of a half-duplex operation at best.